CN115375617A

CN115375617A - Defect detection and training method and device, storage medium and equipment

Info

Publication number: CN115375617A
Application number: CN202210749606.XA
Authority: CN
Inventors: 马麟; 杨路; 汪恺璇
Original assignee: Robotics Robotics Shenzhen Ltd
Current assignee: Robotics Robotics Shenzhen Ltd
Priority date: 2022-06-28
Filing date: 2022-06-28
Publication date: 2022-11-22

Abstract

The application relates to a defect detection and training method, a defect detection and training device, a storage medium and equipment. The defect detection method comprises the following steps: acquiring an initial image; inputting the initial image into a reconstruction module to obtain a reconstructed image close to a normal sample; and inputting the initial image and the reconstructed image into an abnormal segmentation module for comparison so as to obtain a defect detection result of the initial image. By adopting the technical scheme, the stability of the defect detection performance can be improved.

Description

Defect detection and training method and device, storage medium and equipment

Technical Field

The present application relates to the field of artificial intelligence technologies, and in particular, to a method, an apparatus, a storage medium, and a device for defect detection and training.

Background

In recent years, with the development of artificial intelligent AI techniques such as machine learning and deep learning, automated detection of defects using AI techniques has been widely studied.

However, in general, most of the defect detection problems now require a large number of sample data sets and manual calibration as support, the task is completed at high cost, and in addition, in the task of abnormal segmentation, the number of abnormal samples is small, namely rarity of defects, which causes data imbalance when a model is built, but more seriously, some defect types are unpredictable, and even the defect samples are never collected before the defect is met, namely, the unknown of the defect, which causes the failure of the supervised learning scheme.

Therefore, the defect detection is realized by adopting a semi-supervised and unsupervised learning idea, and particularly, only normal samples are used for constructing a model in a training stage, and any manual marking is not needed in the process. But the defect detection based on the thought of semi-supervised and unsupervised learning still has the problems of poor detection stability and the like.

Disclosure of Invention

In view of the above, the present application provides a defect detection and training method, apparatus, storage medium and device.

A first aspect of the present application provides a defect detection method, including:

acquiring an initial image;

inputting the initial image into a reconstruction module to obtain a reconstructed image close to a normal sample;

and inputting the initial image and the reconstructed image into an abnormal segmentation module for comparison so as to obtain a defect detection result of the initial image.

Further, in one embodiment, the reconstruction module includes a GAN or VAE network model.

Further, in one embodiment, the reconstruction module comprises a network model built based on the combination of the GAN and the U-net.

Further, in one embodiment, the anomaly segmentation module includes a network model built based on the ResNet50 and the upsampling structure.

Further, in one embodiment, the reconstruction module includes an encoder and a decoder; in the reconstruction module, only feature fusion is carried out between the last layer jump connection and the last layer up-sampling; wherein, the first and the second end of the pipe are connected with each other,

the last layer-skip connection is a layer-skip connection between the penultimate layer of the encoder and the layer corresponding to the decoder; the last layer upsampling is an upsampling of a last layer of the encoder.

A second aspect of the present application provides a defect detection training method, including:

acquiring a positive training sample and a synthetic defect training sample; wherein the synthesized defect training sample is a defect sample synthesized on the basis of the positive training sample;

obtaining an initial reconstruction model of the reconstruction model;

training the initial reconstruction model by taking the positive training sample as a reconstruction target based on the positive training sample and the synthetic defect training sample and combining with the constraint of a reconstruction loss function to obtain the reconstruction model; and/or

Acquiring a positive training sample, a synthetic defect training sample and a reconstruction training sample; wherein, the reconstructed training sample is a sample which is output by a reconstruction module respectively from the positive training sample and the synthetic defect training sample; the synthetic defect training sample is a defect sample synthesized on the basis of the positive training sample;

obtaining an initial segmentation model of the segmentation model;

and training the initial segmentation model based on the positive training sample, the synthetic defect training sample and the reconstruction training sample in combination with the constraint of a segmentation loss function to obtain the segmentation model.

A third aspect of the present application provides a defect detecting apparatus, including:

the image acquisition module is used for acquiring an initial image;

the image reconstruction module is used for inputting the initial image into the reconstruction module to obtain a reconstructed image close to a normal sample;

an abnormal segmentation module for comparing the initial image and the reconstructed image input into the abnormal segmentation module to obtain the defect detection result of the initial image

The present application fourth aspect provides a defect detection training apparatus, the defect detection training apparatus includes:

the first sample acquisition module is used for acquiring a positive training sample and a synthetic defect training sample; wherein the synthetic defect training sample is a defect sample synthesized on the basis of the positive training sample;

the first model acquisition module is used for acquiring an initial reconstruction model of the reconstruction model;

the first model training module is used for training the initial reconstruction model by taking the positive training sample as a reconstruction target based on the positive training sample and the synthetic defect training sample and combining with the constraint of a reconstruction loss function to obtain the reconstruction model; and/or

The second sample acquisition module is used for acquiring a positive training sample, a synthetic defect training sample and a reconstructed training sample; the reconstructed training samples are samples output by a reconstruction module respectively from the positive training sample and the synthetic defect training sample; the synthetic defect training sample is a defect sample synthesized on the basis of the positive training sample;

the second model acquisition module is used for acquiring an initial segmentation model of the segmentation model;

and the second model training module is used for training the initial segmentation model based on the positive training sample, the synthetic defect training sample and the reconstruction training sample and combined with the constraint of a segmentation loss function to obtain the segmentation model.

A fifth aspect of the invention provides a computer device comprising a memory storing a computer program and a processor implementing the defect detection method and/or the defect detection training method of any of the above when the computer program is executed by the processor.

A sixth aspect of the invention provides a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the defect detection method and/or the defect detection training method of any of the above.

The reconstruction module is used for reconstructing the image and the segmentation module is used for better completing a defect detection task aiming at the initial image by comparing the reconstructed image with the initial image, thereby improving the stability of the defect detection performance.

Drawings

FIG. 1 is a first block diagram of a tag optimization system in one embodiment;

FIG. 2 is a first block diagram of a computer device in one embodiment;

FIG. 3A is a first block diagram of a method for defect detection in one embodiment;

FIG. 3B is a block diagram of a second exemplary embodiment of a method for defect detection;

FIG. 4 is a first block diagram of a reconstructed model in one embodiment;

FIG. 5 is a first block diagram of a segmentation model in one embodiment;

FIG. 6 is a first flowchart of a method for defect detection according to an embodiment;

FIG. 7 is a first flowchart of a defect detection training method according to an embodiment;

FIG. 8 is a diagram illustrating a second process of the defect detection training method according to an embodiment;

FIG. 9 is a first block diagram of an apparatus for defect detection in an embodiment;

FIG. 10 is a first block diagram of a defect detection training apparatus according to an embodiment;

FIG. 11 is a block diagram of a first configuration of a defect detection training apparatus in accordance with an embodiment.

Detailed Description

In order to make the objects, technical solutions and advantages of the present application more clearly understood, the present application is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the present application and are not intended to limit the present application.

At present, the defect detection based on learning ideas such as supervision, semi-supervision and unsupervised by people still has the problems of difficulty in collecting training samples, poor defect detection stability and the like. In view of the above, the present application provides a defect detection method, a defect training method, a defect detection apparatus, a storage medium, and a device. The reconstruction module is used for reconstructing the image, and the segmentation module is used for comparing the reconstructed image with the initial image, so that the defect detection task aiming at the initial image can be better completed, and the stability of the defect detection performance is improved.

For ease of understanding, some of the basic concepts involved in this application are first presented.

And the reconstruction module is used for reconstructing the input initial image to obtain a normal reconstructed image. That is, the initial image may be defective or non-defective, and a normal reconstructed image can be output after the initial image is input to the reconstruction module, that is, the reconstructed image is a normal image without defects.

And the abnormal segmentation module is used for comparing the input initial image with the reconstructed normal image so as to obtain the defect detection result of the initial image.

The following embodiments of the reconstruction module and the anomaly segmentation module will be described in further detail.

The defect detection and defect detection training method provided by the embodiment of the invention can be applied to a Computer terminal (PC), an Industrial control Computer terminal (IPC), a mobile terminal, a server, a system (as shown in fig. 1) comprising a terminal 110 and a server 120, and can be implemented in a Controller similar to a Programmable Logic Controller (PLC), a Field Programmable Gate Array (FPGA), a Digital Signal Processor (DSP), a Micro Control Unit (MCU) or the like through interaction between the terminal 110 and the server 120. The controller generates program instructions according to a pre-fixed program in combination with manually input information, parameters, or data collected by an external image sensor or the like. The specific limitations of the controller can be seen in the limitations of the defect detection and defect detection training methods in the following embodiments.

In particular, the method can be applied to a computer device as shown in fig. 2, where the computer device may be a terminal or a server. The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected by a system bus. Wherein the processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device comprises a nonvolatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used for communicating with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, an operator network, NFC (near field communication) or other technologies. The computer program is executed by a processor to implement a marker optimization method. The display screen of the computer equipment can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer equipment can be a touch layer covered on the display screen, can also be a security check, a track ball or a touch pad arranged on the shell of the computer equipment, and can also be an external keyboard, a touch pad or a mouse and the like.

The server may be an independent physical server, a server cluster or a distributed system formed by a plurality of physical servers, or a cloud server providing basic cloud computing services such as a cloud service, a cloud database, cloud computing, a cloud function, cloud storage, a Network service, cloud communication, a middleware service, a domain name service, a security service, a CDN (Content Delivery Network), a big data and artificial intelligence platform, and the like. The terminal may be, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart audio, a smart watch, and the like. The terminal and the server may be directly or indirectly connected through wired or wireless communication, which is not limited herein.

As shown in fig. 6, in an embodiment, a defect detection method is provided, which may include the following steps, taking the computer terminal 110 as an example, and the method is applied in the system shown in fig. 1:

step S110, acquiring an initial image;

step S120, inputting the initial image into a reconstruction module to obtain a reconstructed image close to a normal sample;

step S130 inputs the initial image and the reconstructed image into the abnormal segmentation module for comparison, so as to obtain a defect detection result of the initial image.

The reconstruction module is used for reconstructing images, and the segmentation module is used for comparing the reconstructed images with the initial images and mutually matching the reconstruction module and the image segmentation module, so that a defect segmentation task aiming at the initial images can be better completed, and the stability of the defect detection performance is improved.

For ease of understanding, the individual steps of the above-described method are described in further detail below.

Step S110 acquires an initial image.

In one embodiment, the computer terminal may access a storage space corresponding to the storage address, extract an initial image of the target object captured and transmitted by the image sensor before that from the accessed storage space, or acquire the initial image transmitted by the server.

The initial image may be an image directly acquired by the image sensor, or an image obtained after some image preprocessing (e.g., cropping, contrast enhancement).

Specifically, the image sensor may be a camera, a scanner, or other devices (mobile phone, computer) with related functions, besides the camera.

It should be noted that, in one embodiment, the initial image may be an image of the surface of the object captured by the image sensor, and thus may be an image with a defect or a normal image without a defect of the object according to the difference of the actual surface of the object (which may or may not have a defect).

Step S120 inputs the initial image to the reconstruction module to obtain a reconstructed image close to a normal sample.

Specifically, the reconstruction module may include a neural Network model such as a generated countermeasure Network (GAN) or a Variance Automatic Encoder (VAE), or other existing or future developed neural Network models with similar reconstruction functions or traditional image processing software.

Further, in one embodiment, the reconstruction module may include a semantic segmentation network (e.g., U-net). For example, the embodiment of the present application takes the case that the reconstruction module partitions the network based on semantics, and partially combines with a residual structure (e.g., resNet 50) for further details.

As shown in fig. 3A or 3B, in one embodiment, the GAN may include a generator and a discriminator.

A generator may be configured to generate a reconstructed image that approximates a normal sample from an input initial image.

The discriminator may be configured to determine whether an input of the discriminator is a defect image synthesized by a true positive sample (e.g., an initial image or a positive training sample)/a true positive sample (e.g., a positive training sample), or a reconstructed image obtained by the defect image synthesized by a true positive sample/a true positive sample through the generator.

Fig. 4 is a block diagram of a GAN generator in one embodiment.

As shown in fig. 4, in one embodiment, the reconstruction module may include a network model built based on GAN (taking DCGAN as an example) and in conjunction with a semantic segmentation network U-net. The generator mainly comprises an encoder and a decoder, wherein the encoder maps an input initial image into a one-dimensional space, and the decoder maps the one-dimensional code into an original picture space to obtain a reconstructed image. Such as: downsampling may be performed using a convolutional layer, the input picture may be encoded, and then upsampling may be performed using a transposed convolution.

In one embodiment, feature fusion may be performed only between the last hop layer connection and the last layer upsampling; wherein, the last layer jump connection is the layer jump connection between the last but one layer of the coder and the corresponding layer of the decoder; the last layer upsampling is the upsampling of the last layer of the encoder. Only one layer of features is used for fusion, so that the defect region features can be prevented from being reconstructed, and the quality of a reconstructed image is improved.

Illustratively, as shown in fig. 4, in principle, an encoder (left region of fig. 4, stage in which the spatial size is gradually reduced) and a decoder (right region of fig. 4, stage in which the spatial size is gradually increased). Corresponding feature layers of the same spatial size between the encoder and the decoder, in principle, each layer may have the existence of that skip layer connection (as shown by the dashed line L in the figure), in this example, only the skip layer connection of the penultimate feature layer of the encoder is used, and other skip layer connections are not needed, because the closer to the feature map of the bottom layer, the more global information is expressed, the closer to the feature layer of the shallow layer (with large spatial size), the more local information is expressed, and the defects, usually belonging to local information, in the generator generation process, we want to input the defect picture (synthesized based on real positive samples), output (corresponding real) positive samples, that is, we do not want the generator to reconstruct the defect region (mostly with local information), therefore, in the example of fig. 4, the skip layer connection between the shallow layer of the encoder and the corresponding shallow layer of the decoder is cancelled, and only the skip layer connection between the penultimate feature layer of the network encoder and the corresponding layer of the decoder is used.

VAE: the main structure is an automatic encoder, which encodes the input and decodes the encoding result to generate the output similar to the sample.

Step S130 compares the initial image and the reconstructed image input into the segmentation module to obtain a defect detection result of the initial image.

As shown in fig. 3A or 3B, the segmentation module may illustratively include a segmentation model built based on ResNet50 and an upsampling structure. Specifically, the segmentation model body can use the U-net structure for reference, and meanwhile, the ResNet50 is used for feature extraction.

As shown in fig. 5, for example, each block of the ResNet50 performs feature extraction on an input image, the output features of each block perform feature fusion in an upsampling module, and a two-classification unit (for example, a sigmoid layer) performs a two-classification operation on the finally obtained image, thereby completing abnormal region segmentation.

As shown in fig. 7, in an embodiment, a training method for defect detection is provided, which is applied to the computer terminal 110 in the system shown in fig. 1 as an example, and may include the following method steps:

step S210, acquiring a positive training sample and a synthetic defect training sample; wherein, the synthesized defect training sample is a defect sample synthesized on the basis of the positive training sample.

Step S220 obtains an initial reconstruction model of the reconstruction model.

Step S230 trains an initial reconstruction model based on the positive training sample and the synthetic defect training sample, in combination with the constraint of the reconstruction loss function, with the positive training sample itself as the reconstruction target, to obtain a reconstruction model.

The defect detection model obtained by training through the method can better complete defect detection and improve the stability of the defect detection performance.

For the sake of an easy understanding, the above-mentioned method steps are explained in further detail below.

Step S210, acquiring a positive training sample and a synthetic defect training sample; wherein the synthesized defect training sample is a defect sample synthesized on the basis of the positive training sample.

In one implementation, the training samples of the reconstructed model may only include a plurality of positive training samples, some defects may be randomly added to the positive training samples as input of the reconstructed model, the non-abnormal region is used as a label, and the reconstructed model may reconstruct an input picture by learning the distribution of normal samples to obtain a reconstructed normal image.

Illustratively, a positive training sample may be represented as follows:

A＝{am|m＝1，2，......，M}

wherein, M is the number of normal images, am is the normal image of the mth target surface, the size of the normal image am is H × W, H is the row number of image pixels, and W is the column number of image pixels.

For example, in the normal sample image data set a, the size of each normal image may be set to be 128 × 128, i.e., W = H =128.

Step S220 obtains an initial reconstruction model of the reconstruction model.

The initial model of the reconstructed model may refer to an untrained reconstructed model with initial parameters set.

For the description of the network structure of the reconstruction model, reference is made to the foregoing embodiments, and the description is not repeated here. In one embodiment, when the reconstruction module includes a generator, the generator may use a Structural Similarity Index Measure (SSIM) and an L2 loss (loss) as loss functions to ensure the quality of the reconstructed image. In one embodiment, when the reconstruction module includes an arbiter, the arbiter may employ a combination of binary cross entropy (BEC) and L2 penalty.

For example, see the following equations 1-1, 1-2, 1-3. Wherein, X is the initial image,

for the output reconstructed image, z is the feature map of the normal sample in the discriminator,

to generate a feature map of the sample in the discriminator.

In one implementation, taking the case that the reconstruction model includes a GAN network structure as an example, step S230 may include: and alternately carrying out the training steps of the discriminator and the generator.

Further, in one embodiment, the parameters of the generator need to be fixed when performing the discriminant training. Inputting training samples (positive training samples and/or synthetic defect training samples) into a generator to obtain reconstructed samples, and then inputting both the reconstructed samples and the training samples into a discriminator for judgment, wherein the discriminator can use the aforementioned L _adv As a function of the loss.

Further, in one embodiment, the parameters of the arbiter need to be fixed when performing the generator training. The training samples are input to the generator to obtain reconstructed samples, and then both the reconstructed samples and the training samples are input to the discriminator for judgment. By the training method, on one hand, the difference between the reconstructed sample and the original sample is limited, and on the other hand, the characteristic difference between the two samples is also limited.

As shown in fig. 8, in an embodiment, a training method for defect detection is provided, which is applied to the computer terminal 110 in the system shown in fig. 1 as an example, and may include the following method steps:

step S310, acquiring a positive training sample, synthesizing a defect training sample and reconstructing the training sample; the reconstruction training sample is a positive training sample and a synthetic defect training sample which are respectively output by a reconstruction module; the synthesized defect training samples are defect samples synthesized on the basis of the positive training samples.

Step S320 acquires an initial segmentation model of the segmentation model.

Step S330 trains an initial segmentation model based on the positive training sample, the synthetic defect training sample and the reconstruction training sample in combination with the constraint of the segmentation loss function to obtain a segmentation model.

In one embodiment, for the segmentation model described in the above embodiment, since the abnormal region has less occupation in the whole picture, when segmenting the pixel point, the loss function cannot directly use the binary cross entropy loss function (BCE loss), which may cause an imbalance problem, and this problem can be solved by using the Focal loss. In addition, in one embodiment, a KL divergence loss function may be used to limit the sparsity of the anomaly region.

For example, the abnormal segmentation loss function may adopt a form of combining focal loss and sparsity loss (as shown in the following formulas 2-1 and 2-2), where Mask is a predicted abnormal region segmentation map, and ρ is a constant to limit sparsity.

In one embodiment, the positive training sample, the synthetic defect training sample, and the reconstructed training samples of the positive training sample and the synthetic defect training sample together constitute the training samples of the segmentation model. Step S330 may comprise the following method steps:

splicing the positive training sample and the reconstructed training sample output by the reconstruction module, and then using the spliced positive training sample and the reconstructed training sample as the input of a segmentation model, wherein the segmented training sample is marked as completely black (because no abnormal area exists); in addition, based on the synthetic defect training sample and the reconstructed training sample output by the reconstruction module, the synthetic defect training sample and the reconstructed training sample are spliced and used as the input of the segmentation model, and at the moment, the model is trained by being marked as a defect area (because the defect is synthesized, the position of the defect area is known in advance). Specifically, the above splicing may refer to channel dimension splicing, that is, splicing two 3-channel pictures into one 6-channel picture.

It should be understood that although the various steps in the flowcharts of fig. 6-8 are shown in order as indicated by the arrows, the steps are not necessarily performed in order as indicated by the arrows. The steps are not performed in the exact order shown and described, and may be performed in other orders, unless explicitly stated otherwise. Also, at least some of the steps of fig. 6-8 may include multiple sub-steps or stages that are not necessarily performed at the same time, but may be performed at different times, and the order of performance of the sub-steps or stages is not necessarily sequential, but may be performed in turn or alternating with other steps or at least some of the sub-steps or stages of other steps.

As shown in fig. 9, in one embodiment, there is provided a defect detecting apparatus including:

an image acquisition module 110, configured to acquire an initial image;

an image reconstruction module 120, configured to input the initial image into a reconstruction module to obtain a reconstructed image close to a normal sample;

an abnormal segmentation module 130, configured to input the initial image and the reconstructed image into the abnormal segmentation module for comparison, so as to obtain a defect detection result of the initial image

As shown in fig. 10, in one embodiment, there is provided a defect detection training apparatus, including:

a first sample obtaining module 210, configured to obtain a positive training sample and a synthetic defect training sample; wherein the synthesized defect training sample is a defect sample synthesized on the basis of the positive training sample;

a first model obtaining module 220, configured to obtain an initial reconstructed model of the reconstructed model;

the first model training module 230 is configured to train an initial reconstruction model based on the positive training sample and the synthetic defect training sample, in combination with the constraint of the reconstruction loss function, with the positive training sample itself as a reconstruction target, to obtain a reconstruction model.

As shown in fig. 11, in one embodiment, a defect detection training apparatus is provided, the apparatus comprising:

a second sample obtaining module 310, configured to obtain a positive training sample, a synthetic defect training sample, and a reconstructed training sample; the reconstructed training sample is a positive training sample and a synthesized defect training sample which are respectively output by the reconstruction module; synthesizing a defect training sample, namely synthesizing the defect sample on the basis of the positive training sample;

a second model obtaining module 320, configured to obtain an initial segmentation model of the segmentation model;

the second model training module 330 is configured to train the initial segmentation model based on the positive training sample, the synthetic defect training sample, and the reconstruction training sample, in combination with the constraint of the segmentation loss function, to obtain a segmentation model.

For the specific limitations of the defect detection and defect detection training apparatus, reference may be made to the limitations of the defect detection and defect detection training method, which are not described herein again. All or part of each module in the defect detection and defect detection training device can be realized by software, hardware and a combination thereof. The modules can be embedded in a hardware form or independent from a processor in the computer device, and can also be stored in a memory in the computer device in a software form, so that the processor can call and execute operations corresponding to the modules.

In one embodiment, as shown in fig. 2, a computer device is provided, which comprises a memory and a processor, the memory storing a computer program, the processor implementing the steps of the defect detection and/or defect detection training method described above when executing the computer program.

In an embodiment, a computer-readable storage medium is provided, on which a computer program is stored, which computer program, when being executed by a processor, carries out the above-mentioned steps of the defect detection and/or defect detection training method

It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by hardware instructions of a computer program, which can be stored in a non-volatile computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. Any reference to memory, storage, database, or other medium used in the embodiments provided herein may include non-volatile and/or volatile memory, among others. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically Programmable ROM (EPROM), electrically Erasable Programmable ROM (EEPROM), or flash memory. Volatile memory can include Random Access Memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in a variety of forms such as Static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double Data Rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous Link DRAM (SLDRAM), rambus (Rambus) direct RAM (RDRAM), direct Rambus Dynamic RAM (DRDRAM), and Rambus Dynamic RAM (RDRAM), among others.

It should be noted that the image sensor, the controller, and the like mentioned in the above method, apparatus, and system may be a real object in a real environment, or may be a virtual image sensor and a virtual image controller in a simulation platform, and the effect of connecting the real object is achieved through a simulation environment. The control unit which is trained by depending on the virtual environment is transplanted to the real environment to control or retrain the real object, so that the resources and time in the training process can be saved.

It will be appreciated by those skilled in the art that the configurations shown in fig. 1 and 2 are only block diagrams of some configurations relevant to the present disclosure, and do not constitute a limitation on the systems, computer devices, etc. to which the present disclosure may be applied, and that a particular system, computer device, etc. may include more or less components than those shown in the figures, or may combine certain components, or have a different arrangement of components.

The technical features of the embodiments described above may be arbitrarily combined, and for the sake of brevity, all possible combinations of the technical features in the embodiments described above are not described, but should be considered as being within the scope of the present specification as long as there is no contradiction between the combinations of the technical features.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described or recited in detail in a certain embodiment, reference may be made to the descriptions of other embodiments.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one position, or may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

The terms "first," "second," "third," "S110," "S120," "S130," and the like in the description and in the drawings above, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged under appropriate circumstances or may occur concurrently in some cases, such that the embodiments described herein may be implemented in other sequences than those illustrated or described herein. Furthermore, the terms "comprises," "comprising," and any variations thereof, are intended to cover non-exclusive inclusions. For example: a process, method, system, article, or robot that comprises a list of steps or modules is not necessarily limited to those steps or modules explicitly listed, but includes other steps or modules not explicitly listed or inherent to such process, method, system, article, or robot.

It should be noted that the embodiments described in the specification are preferred embodiments, and the structures and modules involved are not necessarily essential to the invention, as will be understood by those skilled in the art.

The above-mentioned embodiments only express several embodiments of the present invention, and the description thereof is more specific and detailed, but not construed as limiting the scope of the invention. It should be noted that various changes and modifications can be made by those skilled in the art without departing from the spirit of the invention, and these changes and modifications are all within the scope of the invention. Therefore, the protection scope of the present patent shall be subject to the appended claims.

Claims

1. A defect detection method, comprising:

acquiring an initial image;

2. The defect detection method of claim 1 wherein the reconstruction module comprises a GAN or VAE network model.

3. The defect detection method of claim 1, wherein the reconstruction module comprises a network model built based on a combination of GAN and U-net.

4. The defect detection method according to any one of claims 1 to 3, wherein the anomaly segmentation module comprises a network model built based on ResNet50 and an upsampling structure.

5. The defect detection method of any of claims 1-3, wherein the reconstruction module comprises an encoder and a decoder; in the reconstruction module, only feature fusion is carried out between the last layer jump connection and the last layer up-sampling; wherein the content of the first and second substances,

the last layer jump connection is a layer jump connection between the penultimate layer of the encoder and the layer corresponding to the decoder; the last layer upsampling is upsampling of a last layer of the encoder.

6. A defect detection training method, characterized by comprising:

obtaining an initial reconstruction model of the reconstruction model;

Acquiring a positive training sample, a synthetic defect training sample and a reconstructed training sample; the reconstructed training sample is a sample which is output by a reconstruction module respectively from the positive training sample and the synthetic defect training sample; the synthetic defect training sample is a defect sample synthesized on the basis of the positive training sample;

obtaining an initial segmentation model of the segmentation model;

and training the initial segmentation model based on the positive training sample, the synthetic defect training sample and the reconstruction training sample and combining with the constraint of a segmentation loss function to obtain the segmentation model.

7. A defect detection apparatus, characterized in that the defect detection apparatus comprises:

the image acquisition module is used for acquiring an initial image;

and the abnormal segmentation module is used for inputting the initial image and the reconstructed image into the abnormal segmentation module for comparison so as to obtain a defect detection result of the initial image.

8. A defect inspection training apparatus, characterized in that the defect inspection training apparatus comprises:

the first sample acquisition module is used for acquiring a positive training sample and a synthetic defect training sample; wherein the synthesized defect training sample is a defect sample synthesized on the basis of the positive training sample;

The second sample acquisition module is used for acquiring a positive training sample, a synthetic defect training sample and a reconstructed training sample; the reconstructed training sample is a sample which is output by a reconstruction module respectively from the positive training sample and the synthetic defect training sample; the synthetic defect training sample is a defect sample synthesized on the basis of the positive training sample;

the second model obtaining module is used for obtaining an initial segmentation model of the segmentation model;

9. A computer device comprising a memory and a processor, the memory storing a computer program, wherein the processor implements the defect detection method of any one of claims 1-5 when executing the computer program; and/or the defect detection training method of claim 6.

10. A computer-readable storage medium, on which a computer program is stored, which, when being executed by a processor, carries out the defect detection method according to any one of claims 1 to 5; and/or the defect detection training method of claim 6.