CN115861293A

CN115861293A - Defect contour extraction method, defect contour extraction device, storage medium, defect contour extraction device, and program product

Info

Publication number: CN115861293A
Application number: CN202310080248.2A
Authority: CN
Inventors: 请求不公布姓名
Original assignee: Chengdu Shulian Cloud Computing Technology Co ltd
Current assignee: Chengdu Shulian Cloud Computing Technology Co ltd
Priority date: 2023-02-08
Filing date: 2023-02-08
Publication date: 2023-03-28

Abstract

The embodiment of the application discloses a defect contour extraction method, a defect contour extraction device, a storage medium, equipment and a program product, which relate to the technical field of image processing and comprise the following steps: performing mask processing on the defects according to the position information of the defects on the target image to obtain a mask image; repairing the mask image to obtain a repaired image; and (4) performing pixel level difference on the repaired image and the target image to obtain defect outline information. The method comprises the steps of positioning the position of a defect, carrying out accurate mask processing, focusing on the region except the mask on an image in the presence of the mask, directly eliminating defect interference, restoring the mask image into a corresponding product non-defective background image, and subtracting a target image with the defect from a corresponding non-defective background image at the pixel level.

Description

Defect contour extraction method, defect contour extraction device, storage medium, defect contour extraction device, and program product

Technical Field

The present application relates to the field of image processing technologies, and in particular, to a method, an apparatus, a storage medium, a device, and a program product for extracting a defect contour.

Background

In the production of precision manufacturing such as a PCB, the manufacturing process flow is composed of a plurality of segments, various product defects are easily introduced in a complex and tedious manufacturing process, and in the face of diversified service scenes, further post-processing is often required to be performed on the defects, such as the outline and background segmentation of the defects, but different product types often have different circuit structures and back panel colors, and the types of the defects are rich and diverse and are different in color, shape and depth, so that the defect outline is difficult to extract by the conventional image processing technology.

Disclosure of Invention

The present application mainly aims to provide a defect contour extraction method, apparatus, storage medium, device and program product, and aims to solve the problem in the prior art that a defect contour on a complex backplane is difficult to extract.

In order to achieve the above purpose, the embodiments of the present application adopt the following technical solutions:

in a first aspect, an embodiment of the present application provides a defect contour extraction method, including the following steps:

performing mask processing on the defects according to the position information of the defects on the target image to obtain a mask image;

repairing the mask image to obtain a repaired image; the restored image is a non-defective background image of a product corresponding to the target image;

and (4) performing pixel level difference on the repaired image and the target image to obtain defect outline information.

By positioning the defect position and carrying out accurate mask processing on the defect, the method can focus on the area except the mask on the image in the presence of the mask, directly eliminate the interference of the defect, repair the mask image into the corresponding product non-defective background image, solve the problem that the real non-defective image in the actual state is difficult to obtain, and then under the condition of pixel level difference, the target image with the defect is subjected to difference with the corresponding non-defective background image, because the background of the area without the defect is consistent, the interference of a complex background can be completely eliminated after the difference is made, and the area where the defect is located is highlighted after the difference is made, so that the defect contour information can be quickly extracted.

In a possible implementation manner of the first aspect, before performing mask processing on a defect according to position information of the defect on a target image and obtaining a mask image, the defect contour extraction method further includes:

obtaining bbox coordinate information of the defects on the target image according to the defect information on the target image;

according to the position information of the defect on the target image, performing mask processing on the defect to obtain a mask image, wherein the mask processing comprises the following steps:

and performing mask processing on the defects according to the bbox coordinate information of the defects on the target image to obtain a mask image.

bbox coordinate is defect position area frame coordinate, construct a rectangle frame and cover the defect position, the rectangle frame is more convenient to be fixed a position, its position and size can all be obtained according to the vertex coordinate, because the influence of backplate, concrete defect position can't be drawed temporarily, also be the outline of defect draws the difficulty, utilize bbox mode to carry out position determination, compromise speed and the algorithm degree of difficulty of seeking, can effectively promote the processing speed, and defect position rectangle frame can also provide the position basis for the setting of mask, mask processing also can be more accurate, swift.

In a possible implementation manner of the first aspect, performing mask processing on a defect according to bbox coordinate information of the defect on a target image to obtain a mask image includes:

generating a mask area according to bbox coordinate information of defects on the target image;

and performing mask processing on the defects according to the mask area to obtain a mask image.

bbox is a defect position rectangular frame, so that mask regions with the same size are correspondingly generated to realize accurate coverage, defect omission is avoided during subsequent image restoration, the defect position rectangular frame is defined according to the limit positions of defects in four directions, the size of a mask is reduced as far as possible, mask processing is performed according to the mask regions with the matched sizes, the processing difficulty can be reduced, the difficulty in restoration and restoration is reduced, and the extraction of the defect outline is more efficient.

In one possible implementation manner of the first aspect, the subtracting the repair image and the target image at a pixel level to obtain defect contour information includes:

respectively subtracting corresponding pixel points of the restored image and the target image to obtain pixel difference value information;

taking the pixel difference information as the pixel value of a new pixel point to generate a difference image;

and acquiring defect contour information according to the difference image.

The image is processed by using the minimum unit pixel, so that the defect salient on the image is more clear, the pixel difference value is the result of difference of pixel values of a group of corresponding pixel points, the absolute value is taken to represent that the pixel values of the positions without the defects are the same, the pixel values of the pixel points at the positions with the defects are zero after the difference is made, the area is represented as black, the pixel values of the pixel points at the positions with the defects are different, the pixel values of the pixel points at the positions with the defects are not zero after the difference is made, the pixel values are used as the pixel values of the corresponding points to reconstruct an image, namely a difference image, and the area with the defects is highlighted because the pixels are not zero.

In a possible implementation manner of the first aspect, after the pixel difference information is used as a pixel value of a new pixel point and a difference image is generated, the defect contour extraction method further includes:

carrying out binarization processing on the difference image to obtain a binarization difference image;

obtaining defect contour information according to the difference image, comprising:

and acquiring defect contour information according to the binary difference image.

In order to make the defect information more clear, the pixel points in the area where the defect is located after the defect is made to be poor can more clearly reflect the defect position, the situation that the pixel values of the pixel points in the area where the corresponding defect is located are different, the defect is in a state that the outline is not clear and the depth is not uniform is avoided, the difference image is subjected to binarization processing, namely the difference image is processed into a black-white image, the defect is represented as white, and other areas are reset into black, so that the defect is more clearly highlighted, and the defect outline can be accurately extracted.

In one possible implementation manner of the first aspect, obtaining the defect contour information according to the difference image includes:

obtaining a defect coverage area according to the difference image;

and extracting the outline of the defect coverage area to obtain defect outline information.

The extraction of the defect outline can be realized by adopting an outline searching function of OpenCV (open CV), a region covered by the defect is extracted from the difference image to obtain a complete defect, then the edge of the defect image is extracted by utilizing an edge operator, or point position information on the edge outline is obtained one by one, and when enough points are obtained, the outline of the defect can be naturally and clearly extracted according to the point-action line principle.

In a possible implementation manner of the first aspect, repairing the mask image to obtain a repaired image includes:

removing the mask on the mask image to obtain an image to be repaired;

generating a repairing material based on the background information of the area except the mask on the image to be repaired;

and (5) placing the repairing material in the area where the mask is located to obtain a repairing image.

Removing the mask on the image, for example, the mask adding and removing function of OpenCV is adopted to complete, and obtaining the image to be repaired after the mask is removed; background information such as patterns, colors, textures and the like on a PCB backboard is extracted from other areas on the image, and then the distribution rule of the image background at the mask can be known through the information, so that a repairing material for repairing, namely a small image with the background information is generated, the background on the image is matched with the original image, and the image is repaired only by placing the repairing material with the size matched with the mask area in the area where the original mask is located.

inputting the mask image into a trained image restoration model to obtain a restoration image; the image restoration model can remove the mask on the image and restore the image into a corresponding product defect-free background image.

In order to reduce the time for image restoration and improve the efficiency of defect contour extraction, the model is trained in advance to avoid the time consumption in the restoration process in each flow, and the model can restore the area where the mask is located on the image into the corresponding defect-free background image of the product as in the restoration operation of the previous embodiment due to the processing of the training sample.

In a possible implementation manner of the first aspect, before inputting the mask image into the trained image inpainting model and obtaining the inpainting image, the defect contour extraction method further includes:

obtaining a plurality of background images of the target product without defects;

respectively overlapping the mask material with the background image to obtain a plurality of overlapped images;

and training to obtain an image restoration model based on the plurality of superposed images, the background image and the mask material.

The background image and the mask material are overlapped, a plurality of overlapped images, namely the images with the masks, are obtained through artificial manufacturing, then the overlapped images, the background image and the mask material are used as training samples, the model is made to learn the combination of the overlapped images under supervision training, the mask material on the overlapped images is removed, and therefore the overlapped images are restored into the background image.

In one possible implementation manner of the first aspect, after obtaining the background maps of the defect-free target products, the defect contour extraction method further includes:

performing data enhancement on the background image to obtain a target background image;

and overlapping the mask material with the background image respectively to obtain a plurality of overlapped images, wherein the steps of:

and overlapping the mask materials with the target background image respectively to obtain a plurality of overlapped images.

Because the number of the background images is possibly insufficient, more sample images are obtained by data enhancement, and geometric transformation operations such as symmetry, rotation, cutting, turning and the like are carried out on the background images, so that the sample image set is expanded, the image quality is enhanced, and the robustness and the generalization capability of the model are favorably improved.

In a second aspect, an embodiment of the present application provides a defect contour extraction apparatus, including:

the mask module is used for performing mask processing on the defects according to the position information of the defects on the target image to obtain a mask image;

the restoration module is used for restoring the mask image to obtain a restored image; wherein, the repaired image is a non-defective background image of the product corresponding to the target image

And the difference making module is used for making difference between the repaired image and the target image at a pixel level to obtain defect contour information.

In a third aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is loaded and executed by a processor, the method for extracting a defect contour as provided in any one of the above first aspects is implemented.

In a fourth aspect, an embodiment of the present application provides an electronic device, including a processor and a memory, wherein,

the memory is used for storing a computer program;

the processor is configured to load and execute the computer program, so as to enable the electronic device to execute the defect contour extraction method provided by any one of the above first aspects.

In a fifth aspect, the present application provides a computer program product, which includes a computer program, when being executed, is configured to execute the defect contour extraction method provided in any one of the above first aspects.

Compared with the prior art, the beneficial effects of this application are:

the embodiment of the application provides a defect contour extraction method, a defect contour extraction device, a storage medium, equipment and a program product, wherein the method comprises the following steps: performing mask processing on the defects according to the position information of the defects on the target image to obtain a mask image; repairing the mask image to obtain a repaired image; the restored image is a non-defective background image of a product corresponding to the target image; and (5) performing pixel level difference on the repaired image and the target image to obtain defect outline information. The method of the application carries out accurate mask processing on the defect by positioning the defect position, can focus on the area except the mask on the image more intensively under the condition that the mask exists, directly eliminates the interference of the defect, restores the mask image into the corresponding product non-defective background image, and then under the condition that the pixel level is poor, the target image with the defect is poor with the corresponding non-defective background image, because the background of the area without the defect is consistent, the interference of a complex background can be completely eliminated after the difference is made, the area where the defect is located is highlighted after the difference is made, and the defect outline information can be extracted quickly.

Drawings

Fig. 1 is a schematic structural diagram of an electronic device in a hardware operating environment according to an embodiment of the present application;

fig. 2 is a schematic flowchart of a defect contour extraction method according to an embodiment of the present application;

FIG. 3 is a block diagram of a defect profile extraction apparatus according to an embodiment of the present disclosure;

fig. 4 is a target image in the defect contour extraction method provided in the embodiment of the present application;

fig. 5 is a schematic flow chart of obtaining bbox coordinate information of a defect in the defect contour extraction method provided in the embodiment of the present application;

fig. 6 is a mask image obtained by performing a mask process on the target image shown in fig. 4;

fig. 7 is a restored image obtained by restoring the mask image shown in fig. 6;

fig. 8 is a binarized difference image in the defect contour extraction method according to the embodiment of the present application;

the mark in the figure is: 101-processor, 102-communication bus, 103-network interface, 104-user interface, 105-memory.

Detailed Description

It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

The main solution of the embodiment of the application is as follows: a method, an apparatus, a storage medium, a device and a program product for defect contour extraction are provided, the method comprising: according to the position information of the defects on the target image, carrying out mask processing on the defects to obtain a mask image; repairing the mask image to obtain a repaired image; the restored image is a non-defective background image of a product corresponding to the target image; and (4) performing pixel level difference on the repaired image and the target image to obtain defect outline information.

In the production of precision manufacturing such as PCB printed circuit board, the manufacturing process flow is composed of a plurality of sections, various product defects are easily introduced in the complicated and fussy manufacturing process, the defect types are rich and various, and the defects are different in color, shape and depth; meanwhile, different product types often have different circuit structures and backboard colors, and illumination and definition interference caused by different shooting devices, and a series of change factors bring huge challenges to defect detection tasks.

Many panel or semiconductor manufacturers have introduced intelligent defect detection systems such as AOI and ADC, which can implement real-time detection of products produced on line by learning the defect characteristics of historical products, but currently, only the positions of defects can be located, and in the face of diversified service scenes, further post-processing is often required to be performed on the defects, such as contour and background segmentation of the defects, and the conventional image processing technology is difficult to segment the defect contours under the condition of a complex background, and in addition, the defect types are influenced, which results in inaccurate defect identification requiring special processing according to the contours.

Therefore, the defect position is located, the defect is accurately masked, the region except the mask on the image can be focused under the condition that the mask exists, the interference of the defect is directly eliminated, the mask image is repaired into the corresponding product non-defective background image, then under the condition that the pixel level is poor, the target image with the defect is poor with the corresponding non-defective background image, the background of the region without the defect is consistent, the interference of the complex background can be completely eliminated after the difference is made, the region where the defect is located is highlighted after the difference is made, and the defect outline information can be quickly extracted.

Referring to fig. 1, fig. 1 is a schematic structural diagram of an electronic device in a hardware operating environment according to an embodiment of the present application, where the electronic device may include: a processor 101, such as a Central Processing Unit (CPU), a communication bus 102, a user interface 104, a network interface 103, and a memory 105. Wherein the communication bus 102 is used for enabling connection communication between these components. The user interface 104 may comprise a Display screen (Display), an input unit such as a Keyboard (Keyboard), and the optional user interface 104 may also comprise a standard wired interface, a wireless interface. The network interface 103 may optionally include a standard wired interface, a WIreless interface (e.g., a WIreless-FIdelity (WI-FI) interface). The Memory 105 may be a storage device independent from the processor 101, and the Memory 105 may be a high-speed Random Access Memory (RAM) Memory or a Non-Volatile Memory (NVM), such as at least one disk Memory; the processor 101 may be a general-purpose processor including a central processing unit, a network processor, etc., and may also be a digital signal processor, an application specific integrated circuit, a field programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components.

Those skilled in the art will appreciate that the configuration shown in fig. 1 does not constitute a limitation of the electronic device and may include more or fewer components than those shown, or some components may be combined, or a different arrangement of components.

As shown in fig. 1, the memory 105, which is a storage medium, may include therein an operating system, a data storage module, a network communication module, a user interface module, and an electronic program.

In the electronic device shown in fig. 1, the network interface 103 is mainly used for data communication with a network server; the user interface 104 is mainly used for data interaction with a user; the processor 101 and the memory 105 in the present application may be disposed in an electronic device, and the electronic device calls the defect contour extraction device stored in the memory 105 through the processor 101 and executes the defect contour extraction method provided in the embodiment of the present application.

Referring to fig. 2, based on the hardware device of the foregoing embodiment, an embodiment of the present application provides a defect contour extraction method, including the following steps:

s10: and performing mask processing on the defects according to the position information of the defects on the target image to obtain a mask image.

In the implementation process, the target image is a PCB panel image, and the image has a defect, as shown in fig. 4, the defect location information may be that the panel is divided into a plurality of blocks, and the location information of the defect indicates which blocks and positions of the blocks the defect occurs in, or the center point coordinates are used to indicate the location of the defect. The Mask is also called Mask, and is often used for matting or other local processing on an image, after the Mask is set, a focus point in processing the image can be changed according to the Mask, as shown in fig. 6, that is, an image with a Mask set according to a position of a defect, so that the defect on the image is completely covered by the Mask.

In order to reduce the difficulty in finding the defect position, an embodiment of the present application provides a manner of representing defect position information by using a bbox coordinate, and specifically, before performing mask processing on a defect according to the position information of the defect on a target image and obtaining a mask image, the defect contour extraction method further includes:

and obtaining the bbox coordinate information of the defect on the target image according to the defect information on the target image.

In the specific implementation process, a rectangular frame is constructed to cover the defect position by bbox coordinates, namely defect position area frame coordinates, as shown in the attached drawing 5, the rectangular frame is convenient to position, the position and the size of the rectangular frame can be obtained according to vertex coordinates, the specific defect position can not be extracted temporarily due to the influence of a back plate, namely the outline of the defect is difficult to extract, the position is determined by using a bbox mode, the searching speed and the algorithm difficulty are considered, the processing speed can be effectively increased, the defect position rectangular frame can also provide position basis for the setting of a mask, and the mask processing can be more accurate and faster.

Based on the step of representing the defect position information by adopting the bbox coordinate, the step S10 is to perform mask processing on the defect according to the position information of the defect on the target image to obtain a mask image, and comprises the following steps:

In one embodiment, according to bbox coordinate information of a defect on a target image, performing mask processing on the defect to obtain a mask image, and the method comprises the following steps:

In the specific implementation process, bbox is a defect position rectangular frame, so that mask regions with the same size are correspondingly generated to realize accurate coverage, defect omission is avoided during subsequent image restoration, the defect position rectangular frame is defined according to the limit positions of defects in four directions, the size of the mask is reduced as much as possible, mask processing is performed according to the mask regions with the matched sizes, the processing difficulty can be reduced, the difficulty in restoration and restoration is reduced, and the extraction of the defect outline is more efficient.

S20: repairing the mask image to obtain a repaired image; the restored image is a non-defective background image of a product corresponding to the target image.

In the specific implementation process, the image is repaired, and broadly speaking, the lost or damaged image is restored to the state of the original image, and the embodiment of the application specifically refers to that the area where the mask is located is repaired to be a non-defective product image corresponding to the target image; because the precision degree of the PCB is higher, a defect-free product image in a real state cannot be obtained, the defect-free product image is obtained by adopting the repairing means, and a foundation is laid for obtaining defect contour information by performing difference on a target image and a corresponding defect-free image. As shown in fig. 7, which is a restored image corresponding to the target image shown in fig. 4. Specifically, step S20: repairing the mask image to obtain a repaired image, comprising:

s201: removing the mask on the mask image to obtain an image to be repaired;

s202: generating a repairing material based on the background information of the area except the mask on the image to be repaired;

s203: and (5) placing the repairing material in the area where the mask is located to obtain a repairing image.

In the specific implementation process, the mask on the image is removed, for example, the mask adding and removing functions of OpenCV are adopted, and the image to be repaired is obtained after the mask is removed; background information such as patterns, colors, textures and the like on a PCB backboard is extracted from other areas on the image, and then the distribution rule of the image background at the mask can be known through the information, so that a repairing material for repairing, namely a small image with the background information is generated, the background on the image is matched with the original image, and the image is repaired only by placing the repairing material with the size matched with the mask area in the area where the original mask is located.

S30: and (4) performing pixel level difference on the repaired image and the target image to obtain defect outline information.

In the specific implementation process, the pixel level difference is made, that is, the pixel values of all the points are made to be different, because the repaired image is a non-defective image obtained based on the target image, all the pixel points on the image correspond one to one, the area without defects returns to zero after the difference processing, the image is represented as the same pixel value, the pixel points of the area with defects also obviously have different pixels after the difference processing, the natural defects are highlighted, and the defect contour information can be extracted. Specifically, step S30: and subtracting the repaired image and the target image at a pixel level to obtain defect contour information, wherein the defect contour information comprises:

s301: respectively subtracting corresponding pixel points of the restored image and the target image to obtain pixel difference value information;

s302: taking the pixel difference information as the pixel value of a new pixel point to generate a difference image;

s303: and acquiring defect contour information according to the difference image.

In the specific implementation process, the image is processed by using the minimum unit pixel, so that the defect on the image is more clearly highlighted, the pixel difference value is the result of difference between pixel values of a group of corresponding pixel points, the absolute value is taken to represent that the pixel values of the positions where no defect exists are the same, zero is obtained after the difference is made, the region is represented as black, the pixel values of the pixel points of the positions where the defect exists are different, the pixel values of the pixel points of the positions where the defect exists are not zero after the difference is made, and the pixel values are used as the pixel values of the corresponding points to reconstruct an image, namely a difference image, as shown in fig. 8, and the region with the defect is highlighted because the pixel is not zero.

In the embodiment, the defect position is positioned, the defect is accurately masked, the region except the mask on the image can be more intensively concerned under the condition that the mask exists, the interference of the defect is directly eliminated, the mask image is repaired into the corresponding product non-defective background image, the problem that the real non-defective image under the actual state is difficult to obtain is solved, then under the condition that the pixel level is poor, the target image with the defect is poor with the corresponding non-defective background image, because the background of the region without the defect is consistent, the interference of the complex background can be completely eliminated after the difference is made, and the region where the defect is located is highlighted after the difference is made, so that the defect outline information can be quickly extracted.

In an embodiment, after the pixel difference information is used as the pixel value of a new pixel point and a difference image is generated, the defect contour extraction method further includes:

and carrying out binarization processing on the difference image to obtain a binarization difference image.

In the specific implementation process, in order to make the defect information more clear, the pixel points in the region where the defect is located after the defect is made can more clearly reflect the defect position, the condition that the pixel values of the pixel points in the region where the corresponding defect is located are different, so that the defect has states of unclear outline and inconsistent depth is avoided, the difference image is subjected to binarization processing, namely processed into a black-and-white image, as shown in the attached drawing 8, the defect is represented as white, and other regions are reset into black, so that the defect is more clearly highlighted, and the defect outline can be accurately extracted.

Based on the binarization operation, obtaining defect contour information according to the difference image, including:

and acquiring defect contour information according to the binarization difference image.

In one embodiment, obtaining defect profile information from the difference image comprises:

acquiring a defect coverage area according to the difference image;

In the specific implementation process, the extraction of the defect outline can be realized by adopting an outline searching function such as OpenCV (open CV), an area covered by the defect is extracted from the difference image to obtain a complete defect, then the edge of the defect image is extracted by utilizing an edge operator, or point position information on the edge outline is obtained one by one, and when enough points are obtained, the outline of the defect can be naturally and clearly extracted according to the point-action line principle.

In one embodiment, repairing the mask image to obtain a repaired image includes:

In a specific implementation process, in order to reduce the time for image restoration and improve the efficiency of defect contour extraction, the model is trained in advance to avoid time consumption in the restoration process in each flow, and the training sample is processed, so that the model can restore the area where the mask on the image is located into the corresponding defect-free background image of the product as in the restoration operation of the previous embodiment. Specifically, before inputting the mask image into the trained image inpainting model and obtaining the inpainting image, the defect contour extraction method further includes:

In the specific implementation process, the background image and the mask material are overlapped, the overlapped images, namely the images with the masks, are obtained through artificial manufacturing, then the overlapped images, the background image and the mask material are used as training samples, the model is learned to the combination of the overlapped images under supervision training, the mask material on the overlapped images is removed, and therefore the overlapped images are restored into the background image.

In one embodiment, after obtaining the background images of the defect-free target products, the defect contour extraction method further includes:

and performing data enhancement on the background image to obtain a target background image.

In the specific implementation process, as the number of the background images is possibly insufficient, more sample images are obtained by data enhancement, and geometric transformation operations such as symmetry, rotation, clipping, turning and the like are performed on the background images, so that the sample image set is expanded, the image quality is enhanced, and the robustness and the generalization capability of the model are favorably improved.

Based on the data enhancement operation, the mask material is respectively overlapped with the background image to obtain a plurality of overlapped images, which comprises the following steps:

Referring to fig. 3, based on the same inventive concept as the previous embodiment, the present application further provides a defect contour extraction apparatus, including:

It should be understood by those skilled in the art that the division of each module in the embodiment is only a division of a logic function, and all or part of the modules may be integrated onto one or more actual carriers in actual application, and all of the modules may be implemented in a form called by a processing unit through software, or implemented in a form of hardware, or implemented in a form of combination of software and hardware, and it should be noted that each module in the defect contour extraction apparatus in the embodiment corresponds to each step in the defect contour extraction method in the foregoing embodiment one to one, therefore, the specific implementation manner of the embodiment may refer to the implementation manner of the defect contour extraction method, and is not described herein again.

Based on the same inventive concept as that in the foregoing embodiments, embodiments of the present application further provide a computer-readable storage medium, which stores a computer program, and when the computer program is loaded and executed by a processor, the defect contour extraction method provided by the embodiments of the present application is implemented.

Based on the same inventive concept as the foregoing embodiments, embodiments of the present application further provide an electronic device, including a processor and a memory, wherein,

the memory is used for storing a computer program;

the processor is used for loading and executing a computer program to enable the electronic device to execute the defect contour extraction method provided by the embodiment of the application.

Furthermore, based on the same inventive concept as in the previous embodiments, embodiments of the present application also provide a computer program product comprising a computer program for executing the defect contour extraction method as provided by the embodiments of the present application when the computer program is executed.

In some embodiments, the computer-readable storage medium may be memory such as FRAM, ROM, PROM, EPROM, EEPROM, flash, magnetic surface memory, optical disk, or CD-ROM; or may be various devices including one or any combination of the above memories. The computer may be a variety of computing devices including intelligent terminals and servers.

In some embodiments, executable instructions may be written in any form of programming language (including compiled or interpreted languages), in the form of programs, software modules, scripts or code, and may be deployed in any form, including as a stand-alone program or as a module, component, subroutine, or other unit suitable for use in a computing environment.

By way of example, executable instructions may correspond, but do not necessarily have to correspond, to files in a file system, and may be stored in a portion of a file that holds other programs or data, such as in one or more scripts in a hypertext Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple coordinated files (e.g., files that store one or more modules, sub-programs, or portions of code).

By way of example, executable instructions may be deployed to be executed on one computing device or on multiple computing devices at one site or distributed across multiple sites and interconnected by a communication network.

It should be noted that, in this document, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or system. Without further limitation, an element defined by the phrases "comprising a," "8230," "8230," or "comprising" does not exclude the presence of other like elements in a process, method, article, or system comprising the element.

The above-mentioned serial numbers of the embodiments of the present application are merely for description and do not represent the merits of the embodiments.

Through the description of the foregoing embodiments, it is clear to those skilled in the art that the method of the foregoing embodiments may be implemented by software plus a necessary general hardware platform, and certainly may also be implemented by hardware, but the former is a better embodiment in many cases. Based on such understanding, the technical solutions of the present application may be embodied in the form of a software product, which is stored in a storage medium (e.g., a rom/ram, a magnetic disk, an optical disk) and includes instructions for enabling a multimedia terminal (e.g., a mobile phone, a computer, a television receiver, or a network device) to execute the method according to the embodiments of the present application.

In summary, the present application provides a method, an apparatus, a storage medium, a device and a program product for defect contour extraction, where the method includes: performing mask processing on the defects according to the position information of the defects on the target image to obtain a mask image; repairing the mask image to obtain a repaired image; the repaired image is a non-defective background image of a product corresponding to the target image; and (4) performing pixel level difference on the repaired image and the target image to obtain defect outline information. The method and the device have the advantages that the defect position is located, the defect is accurately masked, the region except the mask on the image can be focused under the condition that the mask exists, the interference of the defect is directly eliminated, the mask image is restored into the corresponding product non-defective background image, the problem that the real non-defective image under the actual state is difficult to obtain is solved, then under the condition that the pixel level difference is made, the target image with the defect is subjected to the difference with the corresponding non-defective background image, the interference of a complex background can be completely eliminated after the difference is made because the background of the region without the defect is consistent, and the region where the defect is located is highlighted after the difference is made, so that the defect outline information can be quickly extracted.

The above description is only exemplary of the present application and should not be taken as limiting the present application, as any modification, equivalent replacement, or improvement made within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A defect contour extraction method is characterized by comprising the following steps:

according to the position information of the defects on the target image, performing mask processing on the defects to obtain a mask image;

repairing the mask image to obtain a repaired image; the repaired image is a product defect-free background image corresponding to the target image;

and performing pixel level difference on the repaired image and the target image to obtain defect outline information.

2. The method of claim 1, wherein before the step of masking the defect according to the position information of the defect on the target image to obtain the masked image, the method further comprises:

the mask processing is performed on the defect according to the position information of the defect on the target image to obtain a mask image, and the mask processing comprises the following steps:

and performing mask processing on the defects according to bbox coordinate information of the defects on the target image to obtain a mask image.

3. The method for extracting the defect contour according to claim 2, wherein the step of performing mask processing on the defect according to bbox coordinate information of the defect on the target image to obtain a mask image comprises the steps of:

4. The method for extracting defect contour according to claim 1, wherein the step of subtracting the repaired image from the target image at a pixel level to obtain defect contour information comprises:

taking the pixel difference value information as the pixel value of a new pixel point to generate a difference value image;

and acquiring defect contour information according to the difference image.

5. The method of claim 4, wherein after the generating the difference image by using the pixel difference information as the pixel value of the new pixel, the method further comprises:

the obtaining of the defect contour information according to the difference image includes:

6. The method of claim 4, wherein the obtaining defect contour information from the difference image comprises:

obtaining the defect coverage area according to the difference image;

7. The method for extracting a defect contour according to claim 1, wherein the repairing the mask image to obtain a repaired image comprises:

removing the mask on the mask image to obtain an image to be repaired;

and placing the repair material in the area where the mask is located to obtain a repair image.

8. The method for extracting a defect contour according to claim 1, wherein the repairing the mask image to obtain a repaired image comprises:

9. The method of claim 8, wherein before inputting the mask image into a trained image inpainting model and obtaining an inpainting image, the method further comprises:

overlapping the mask materials with the background images respectively to obtain a plurality of overlapped images;

and training to obtain the image restoration model based on the plurality of superposed images, the background image and the mask material.

10. The method of claim 9, wherein after obtaining the background images of the defect-free target products, the method further comprises:

the overlapping the mask material with the background image respectively to obtain a plurality of overlapped images comprises:

11. A defect contour extraction device, comprising:

the repairing module is used for repairing the mask image to obtain a repaired image; wherein the repaired image is a defect-free background image of the product corresponding to the target image

12. A computer-readable storage medium, storing a computer program, wherein the computer program, when loaded and executed by a processor, implements a defect profile extraction method as claimed in any one of claims 1 to 10.

13. An electronic device comprising a processor and a memory, wherein,

the memory is used for storing a computer program;

the processor is configured to load and execute the computer program to cause the electronic device to perform the defect contour extraction method according to any one of claims 1 to 10.

14. A computer program product, comprising a computer program for performing the defect contour extraction method of any one of claims 1-10 when the computer program is executed.