CN113112442A

CN113112442A - Defect detection method and device and terminal equipment

Info

Publication number: CN113112442A
Application number: CN201911358035.1A
Authority: CN
Inventors: 孔庆杰; 林姝
Original assignee: Riseye Intelligent Technology Shenzhen Co ltd
Current assignee: Riseye Intelligent Technology Shenzhen Co ltd
Priority date: 2019-12-25
Filing date: 2019-12-25
Publication date: 2021-07-13

Abstract

The application is applicable to the technical field of detection, and provides a defect detection method, a defect detection device and terminal equipment, wherein the defect detection method comprises the following steps: acquiring an image sample of a device to be detected; carrying out defect detection on the image sample through the trained defect detection network to obtain a detection result of the device to be detected; acquiring an image sample of which the detection result meets a preset condition and marking the image sample; and based on the labeled image sample after labeling, performing defect detection on the subsequent image sample through the trained defect detection network. According to the method and the device, the trained defect detection network does not need to be updated, the required mark quantity is small, the robustness of the defect detection network can be enhanced, and the defect detection efficiency is improved.

Description

Defect detection method and device and terminal equipment

Technical Field

The application belongs to the technical field of detection, and particularly relates to a defect detection method, a defect detection device and terminal equipment.

Background

In industrial production, defect identification for industrial components is an important guarantee for product quality. The method has many researches aiming at the defect detection method of the industrial components, and the high-quality defect detection has great significance for improving the production quality and the efficiency. At present, the efficiency of manual detection is low, the detection is easily influenced by individual states, the precision is not high, and the traditional detection method based on deep learning needs larger labeling quantity.

Disclosure of Invention

In order to overcome the problems in the related art, the embodiments of the present application provide a defect detection method, apparatus, and terminal device.

The application is realized by the following technical scheme:

in a first aspect, an embodiment of the present application provides a defect detection method, including:

acquiring an image sample of a device to be detected;

carrying out defect detection on the image sample through the trained defect detection network to obtain a detection result of the device to be detected;

acquiring an image sample of which the detection result meets a preset condition and marking the image sample;

and based on the labeled image sample after labeling, performing defect detection on the subsequent image sample through the trained defect detection network.

In a possible implementation manner of the first aspect, the labeling, performed on the image sample whose detection result meets a preset condition, includes:

and selecting the image sample with the confidence coefficient of the detection result larger than the confidence coefficient threshold value for labeling.

In a possible implementation manner of the first aspect, the performing, by the trained defect detection network, defect detection on a subsequent image sample based on an annotated image sample includes:

inputting the subsequent image sample and the labeled image sample into the trained defect detection network;

calculating the prediction probability that the subsequent image sample and each marked image sample belong to the same type of defect;

determining a detection result for the subsequent image sample based on the respective prediction probabilities.

In a possible implementation manner of the first aspect, the determining a detection result of the subsequent image sample according to each prediction probability includes:

determining the defect types of the marked image samples corresponding to the prediction probabilities;

calculating the average value of the prediction probabilities belonging to the same defect category;

and determining the detection result of the subsequent image sample according to the average value corresponding to each defect type.

In a possible implementation manner of the first aspect, the method further includes:

acquiring a plurality of training image samples of a device to be detected, wherein each training image sample contains artificial labeling information;

and training the defect detection network after pre-training through the plurality of training image samples to obtain the trained defect detection network.

In a possible implementation manner of the first aspect, the training the pre-trained defect detection network by using the plurality of training image samples includes:

dividing the plurality of training image samples into a plurality of image sample pairs, each image sample pair comprising two training image samples;

for each image sample pair, acquiring feature layers of two training image samples through the defect detection network which is pre-trained;

and after connecting the two characteristic layers, predicting the probability that the two training image samples belong to the same type of defects through the full-connection layer of the pre-trained defect detection network.

In a possible implementation manner of the first aspect, the acquiring an image sample of a device to be detected includes:

and acquiring an image sample of the device to be detected by an industrial camera.

In a second aspect, an embodiment of the present application provides a defect detecting apparatus, including:

the image sample acquisition module is used for acquiring an image sample of the device to be detected;

the first defect detection module is used for carrying out defect detection on the image sample through the trained defect detection network to obtain a detection result of the device to be detected;

the image sample labeling module is used for obtaining the image sample of which the detection result meets the preset condition and labeling the image sample;

and the second defect detection module is used for carrying out defect detection on the subsequent image sample through the trained defect detection network based on the labeled image sample.

In a third aspect, an embodiment of the present application provides a terminal device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, where the processor implements the defect detection method according to any one of the first aspect when executing the computer program.

In a fourth aspect, the present application provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the computer program implements the defect detection method according to any one of the first aspect.

In a fifth aspect, the present application provides a computer program product, which when run on a terminal device, causes the terminal device to execute the defect detection method described in any one of the above first aspects.

It is understood that the beneficial effects of the second aspect to the fifth aspect can be referred to the related description of the first aspect, and are not described herein again.

Compared with the prior art, the embodiment of the application has the advantages that:

according to the embodiment of the application, the image sample of the device to be detected is obtained, the defect detection is carried out on the image sample through the trained defect detection network to obtain the detection result of the device to be detected, the image sample of which the detection result meets the preset condition is marked, the defect detection is carried out on the subsequent image sample through the trained defect detection network based on the marked image sample, the defect detection is carried out on the subsequent image sample through the trained defect detection network, the defect detection network does not need to be updated after the training, the defect detection can be carried out on the subsequent image sample through the trained defect detection network based on the marked image sample, the required marked amount is small, and the marked image sample is the image sample with better quality, so that the robustness of the defect detection network can be enhanced, and the defect detection efficiency is improved.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the specification.

Drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the embodiments or the prior art descriptions will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present application, and it is obvious for those skilled in the art to obtain other drawings based on these drawings without inventive exercise.

Fig. 1 is a schematic view of an application scenario of a defect detection method according to an embodiment of the present application;

FIG. 2 is a schematic flow chart illustrating a defect detection method according to an embodiment of the present application;

FIG. 3 is a schematic flow chart illustrating a defect detection method according to an embodiment of the present application;

FIG. 4 is a schematic flowchart of a defect detection method according to an embodiment of the present application;

FIG. 5 is a schematic flowchart of a defect detection method according to an embodiment of the present application;

FIG. 6 is a schematic flowchart of a defect detection method according to an embodiment of the present application;

FIG. 7 is a flowchart illustrating a defect detection method according to an embodiment of the present application;

FIG. 8 is a schematic structural diagram of a defect detection apparatus provided in an embodiment of the present application;

FIG. 9 is a schematic structural diagram of a defect detection apparatus according to an embodiment of the present application;

fig. 10 is a schematic structural diagram of a terminal device provided in an embodiment of the present application;

fig. 11 is a schematic structural diagram of a computer to which the defect detection method provided in the embodiment of the present application is applied.

Detailed Description

In the following description, for purposes of explanation and not limitation, specific details are set forth, such as particular system structures, techniques, etc. in order to provide a thorough understanding of the embodiments of the present application. It will be apparent, however, to one skilled in the art that the present application may be practiced in other embodiments that depart from these specific details. In other instances, detailed descriptions of well-known systems, devices, circuits, and methods are omitted so as not to obscure the description of the present application with unnecessary detail.

It will be understood that the terms "comprises" and/or "comprising," when used in this specification and the appended claims, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof.

It should also be understood that the term "and/or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

As used in this specification and the appended claims, the term "if" may be interpreted contextually as "when", "upon" or "in response to" determining "or" in response to detecting ". Similarly, the phrase "if it is determined" or "if a [ described condition or event ] is detected" may be interpreted contextually to mean "upon determining" or "in response to determining" or "upon detecting [ described condition or event ]" or "in response to detecting [ described condition or event ]".

Furthermore, in the description of the present application and the appended claims, the terms "first," "second," "third," and the like are used for distinguishing between descriptions and not necessarily for describing or implying relative importance.

Reference throughout this specification to "one embodiment" or "some embodiments," or the like, means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," or the like, in various places throughout this specification are not necessarily all referring to the same embodiment, but rather "one or more but not all embodiments" unless specifically stated otherwise. The terms "comprising," "including," "having," and variations thereof mean "including, but not limited to," unless expressly specified otherwise.

Based on the above problems, in the defect detection method in the embodiment of the present application, an image sample of a device to be detected is obtained, a detection result of the device to be detected is obtained by performing defect detection on the image sample through a trained defect detection network, then, the image sample with the detection result satisfying a preset condition is labeled, and defect detection is performed on a subsequent image sample through the trained defect detection network based on the labeled image sample.

Fig. 1 is a schematic view of an application system of a defect detection method provided in an embodiment of the present application, and referring to fig. 1, the defect detection method may be used for detecting defects of components in an industrial production process. The image capturing device 10 is used for capturing an image sample of the device to be detected 30, and then inputting the image sample into the defect detecting device 20 for defect detection processing. The device 30 to be detected may be one device to be detected or a plurality of devices to be detected, and the defect detection method according to the embodiment of the present application may be implemented.

Specifically, the defect detecting device 20 may specifically perform defect detection on the image sample sent by the image acquisition device 10 through the trained defect detection network to obtain a detection result of the device to be detected, label the image sample whose detection result meets a preset condition, and perform defect detection on the subsequent image sample through the trained defect detection network based on the labeled image sample, thereby implementing defect detection on the device.

The defect detection method according to the embodiment of the present application is described in detail below with reference to fig. 1.

Fig. 2 is a schematic flow chart of a defect detection method according to an embodiment of the present application, and referring to fig. 2, the defect detection method is described in detail as follows:

in step 101, an image sample of a device under test is acquired.

The image sample of the device to be detected may be acquired by the image acquisition device, for example, the image sample of the device to be detected acquired by the image acquisition device may be directly acquired to perform processing in subsequent steps, or an image acquisition instruction may be sent to the image acquisition device, and the image sample of the device to be detected acquired by the image acquisition device based on the image acquisition instruction is acquired, which is not limited in this embodiment of the present application.

In some embodiments, an image sample of the device to be tested can be captured by an industrial camera. The industrial camera has the advantages of high image stability, high transmission capability, high interference resistance, and the like, and may be, for example, an industrial camera based on a CCD (Charge Coupled Device) chip or a CMOS (Complementary Metal Oxide Semiconductor) chip.

The embodiment of the present application is not limited to this, and may be specifically determined according to the structure and the type of the device, and the detection requirement.

It should be noted that the image sample of the device to be detected in this step may be one image sample of one device to be detected, may also be multiple image samples of one device to be detected, and may also be multiple image samples of multiple devices to be detected, which is not limited in this embodiment of the present application.

In step 102, the trained defect detection network is used to perform defect detection on the image sample, so as to obtain a detection result of the device to be detected.

In this step, the defect detection network that has been pre-trained may be trained to obtain a trained defect detection network, and a specific training process is described in detail later and will not be described in detail here.

Specifically, the image sample obtained in step 101 may be input into the trained defect detection network to obtain a detection result of the device to be detected, where the detection result may include information such as a defect position, a defect type, and a confidence of a predicted value of each detection result of the device to be detected.

For example, the defect detection network may be an RCNN network, an RFCN network, or an SSD network, and specifically, which kind of defect detection network is used may be selected according to the situation, which is not limited in this embodiment of the present application.

The following takes the RCNN network as an example to illustrate the embodiments of the present application, but the present application is not limited thereto.

For example, for input k image samples x_iThe RCNN network can output k detection results corresponding to

The confidence degree corresponding to each detection result can be s₁,…,s_k. Each image sample corresponds to one detection result, and each detection result corresponds to one confidence coefficient.

It should be noted that, in this step, the trained defect detection network may be used to detect the image sample input for the first time, so as to obtain the detection result of the device to be detected; in this step, a trained defect detection network may also be used to detect a non-first-time input image sample to obtain a detection result of a device to be detected, which is not limited in this embodiment of the present application.

For example, the image samples may be input to the trained defect detection network one by one for defect detection, and the number of the image samples input at each time is not limited. For example, there may be multiple image samples of one device to be tested, or multiple image samples of multiple devices to be tested. The condition that the trained defect detection network is used for detecting the image sample which is not input for the first time can be specifically that the image sample which is input into the trained defect detection network is input for defect detection before. For example, the image samples in this step are the nth batch of image samples input into the trained defect detection network, where N is greater than or equal to 2.

In this embodiment, the image sample in step 102 is an image sample that is first input to the trained defect detection network, and the scheme of the present application is described by taking this as an example, but not limited thereto.

In step 103, the image sample with the detection result meeting the preset condition is obtained for labeling.

The marked range may include the type of the defect and the position of the defect, for example, the position of the defect may be marked with a rectangular frame.

For example, the labeling of the image sample whose detection result meets the preset condition may be: and selecting the image sample with the confidence coefficient of the detection result larger than the confidence coefficient threshold value for labeling. The specific value of the confidence threshold may be set according to actual needs, wherein under the condition that the confidence threshold is higher, an image sample with a confidence greater than the confidence threshold may not be obtained in each detection result, that is, the confidence of each image sample is less than the confidence threshold.

For example, for image sample x_iAnd corresponding detection results

The image samples with the confidence degrees larger than the confidence degree threshold value can be selected for labeling, the image samples with the higher confidence degrees are selected for labeling, and the types and the positions of the defects are determined for performing defect identification on the subsequent image samples.

In the step, the image samples meeting the preset conditions can be selected to be labeled according to the detection results, instead of labeling all the image samples, and the image samples meeting the preset conditions are fewer than all the image samples, so that the labeling quantity can be reduced, the labeling efficiency can be improved, and the labeling information of the image samples meeting the preset conditions is adopted to identify the defects of the subsequent image samples, and the robustness of the defect monitoring network can also be improved.

In step 104, based on the labeled image sample after labeling, defect detection is performed on the subsequent image sample through the trained defect detection network.

Specifically, the annotated image sample and the subsequent image sample may be input to a trained defect detection network, and defect identification may be performed on the subsequent image sample based on the annotation information of the annotated image sample.

Referring to fig. 3, step 104 may be implemented by the following process:

in step 1041, the subsequent image samples and the annotated image samples are input to the trained defect detection network.

In step 1042, the prediction probability that the following image sample and each annotated image sample belong to the same type of defect is calculated.

The annotated image sample and the subsequent image sample can be input into the trained defect detection network in pairs, and the prediction probability that the subsequent image sample and each annotated image sample belong to the same type of defect is calculated.

For example, whether the subsequent image sample and the annotation image sample belong to the same type of defect can be determined according to the defect type and the defect position. Specifically, if the defect type and defect position of a certain subsequent image sample are the same as those of a certain labeled image sample, the subsequent image sample and the labeled image sample are determined to be the image sample with the same type of defects.

In step 1043, the detection result for the subsequent image sample is determined based on the respective prediction probabilities.

The number of the annotated image samples may be multiple, so that each annotated image sample corresponds to one prediction probability with a subsequent image sample, and therefore, a detection result of the subsequent image sample can be determined according to each prediction probability.

Specifically, referring to fig. 4, the above-mentioned determining the detection result of the subsequent image sample based on the respective prediction probabilities may specifically include the following processes:

in step 201, the defect type of the labeled image sample corresponding to each prediction probability is determined.

For example, the annotated image sample comprises N second image samples, which for any second image sample corresponds to a prediction probability relative to a subsequent image sample, the prediction probability characterizing a similarity probability of a defect of the second image sample to a defect of the subsequent image sample. And each labeled image sample corresponding to the prediction probability corresponds to one defect type and can be obtained according to the labeling information of the labeled image sample.

It should be noted that the N second image samples may correspond to M defect types, where M is less than or equal to N, that is, the defect types of the second image samples may be different, or the defect types of the second image samples may be partially the same.

In step 202, the average of the prediction probabilities belonging to the same defect class is calculated.

For example, for N second image samples, M defect classes, the 1 st defect class corresponds to N₁A second image sample, the 2 nd defect type corresponds to N₂A second image sample, … …, with the M defect class corresponding to N_MA second image sample, wherein N₁、N₂、……、N_MThe sum is N. Then, for the 1 st defect, the average of the corresponding prediction probabilities is N₁A second image sampleAn average of the sums of the corresponding prediction probabilities; for the 2 nd defect, the average value of the corresponding prediction probability is N₂The average value of the sum of the prediction probabilities corresponding to the second image samples; for the M defect, the average value of the corresponding prediction probability is N_MThe average value of the sum of the prediction probabilities corresponding to the second image samples.

In step 203, the detection result of the subsequent image sample is determined according to the average value corresponding to each defect type.

After calculating the average value of the prediction probabilities corresponding to the defect types, determining the detection result of the subsequent image sample according to the average value of the prediction probabilities of the subsequent image sample and the defect types. For example, the detection result may be a defect type of a subsequent image sample and a corresponding prediction probability value, which is an average value of the prediction probabilities.

In a possible implementation manner, the defect detection method may further include a step of training a defect detection network.

Specifically, referring to fig. 5, the defect detection method may further include:

in step 105, a plurality of training image samples of a device to be detected are obtained, wherein each training image sample contains artificial labeling information.

In step 106, the defect detection network that is pre-trained is trained through the plurality of training image samples, so as to obtain the trained defect detection network.

The training image samples all contain information such as manually labeled defect types, defect positions and the like, and are used for training a defect detection network after pre-training. For example, the pre-trained defect detection network may be a pre-trained defect detection network using the ImageNet data set.

In the step, the defect detection network which is pre-trained is trained through the training image samples, so that the number of required training image samples can be greatly reduced, and further, the workload of carrying out manual labeling on the training image samples can be effectively reduced.

Illustratively, referring to fig. 6, step 106 may include the steps of:

in step 1061, the plurality of training image samples are divided into a plurality of image sample pairs, each of the image sample pairs including two training image samples.

In step 1062, for each of the image sample pairs, feature layers of two training image samples are obtained through the pre-trained defect detection network.

In step 1063, after connecting the two feature layers, predicting the probability that the two training image samples belong to the same type of defect through the fully-connected layer of the pre-trained defect detection network.

For example, a plurality of training image samples may be randomly divided into a plurality of image sample pairs (x)_i,x_j) Wherein x is_iAnd x_jRepresenting two training image samples, for each image sample pair (x)_i,x_j) After data enhancement processing, each image sample pair (x) is obtained by utilizing a defect detection network after pre-training_i,x_j) After the two characteristic layers of the two training image samples are connected, the probability that the two training image samples belong to the same type of defect is predicted through the full-connection layer of the defect detection network which is pre-trained, so that the defect detection network is trained, and the trained defect detection network is obtained.

According to the defect detection method, the deep learning is used for replacing the artificial defect identification, so that the interference of artificial subjective factors can be reduced, and the product quality is ensured; training the defect detection network after pre-training, and effectively reducing the workload of manually labeling the training image sample by a transfer learning mode; the image samples with higher confidence coefficient are labeled, and the subsequent image samples are subjected to defect detection based on the labeling information, so that the robustness of a defect detection network can be enhanced, the defect detection network does not need to be updated, and the defect detection efficiency can be improved.

Fig. 7 is a schematic flowchart of a defect detection method according to an embodiment of the present application, and referring to fig. 7, the defect detection method is described in detail as follows:

in step 301, a plurality of training image samples of a device to be detected are obtained, where each training image sample includes artificial labeling information.

In step 302, the defect detection network that has been pre-trained is trained by a plurality of training image samples, so as to obtain a trained defect detection network.

In step 303, an image sample of the device under test is acquired.

In step 304, the trained defect detection network is used to perform defect detection on the image sample, so as to obtain a detection result of the device to be detected.

In step 305, image samples with the confidence level of the defect detection result greater than the confidence level threshold are selected for labeling.

In step 306, an image sample of the device under test is acquired. The image sample in this step is a subsequent image sample with respect to the image sample in step 303.

In step 307, based on the labeled image sample, defect detection is performed on the subsequently acquired image sample through the trained defect detection network.

According to the defect detection method, the trained defect detection network is not required to be updated, the trained defect detection network can be used for detecting the defects of the subsequent image sample based on the labeled image sample, the required labeling amount is small, and the labeled image sample is the image sample with high confidence coefficient, so that the robustness of the defect detection network can be enhanced, and the defect detection efficiency is improved.

It should be understood that, the sequence numbers of the steps in the foregoing embodiments do not imply an execution sequence, and the execution sequence of each process should be determined by its function and inherent logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

Fig. 8 and 9 show block diagrams of the defect detection apparatus provided in the embodiment of the present application, corresponding to the defect detection method described in the above embodiment, and only the relevant parts to the embodiment of the present application are shown for convenience of description.

Referring to fig. 8, the defect detecting apparatus in the embodiment of the present application may include an image sample acquiring module 401, a first defect detecting module 402, an image sample labeling module 403, and a second defect detecting module 404.

The image sample acquiring module 401 is configured to acquire an image sample of a device to be detected;

a first defect detection module 402, configured to perform defect detection on the image sample through the trained defect detection network to obtain a detection result of the device to be detected;

an image sample labeling module 403, configured to obtain an image sample with a detection result meeting a preset condition and label the image sample;

and a second defect detection module 404, configured to perform defect detection on a subsequent image sample through the trained defect detection network based on the labeled image sample after labeling.

Optionally, the image sample labeling module 403 may be specifically configured to:

Referring to fig. 9, the second defect detecting module 404 may include an input unit 4041, a probability calculating unit 4042, and a determining unit 4043.

The input unit 4041 is configured to input the subsequent image sample and the labeled image sample into the trained defect detection network;

a probability calculating unit 4042, configured to calculate a prediction probability that the subsequent image sample and each labeled image sample belong to the same type of defect;

a determining unit 4043, configured to determine a detection result for the subsequent image sample based on the respective prediction probabilities.

For example, the determining unit 4043 may specifically be configured to:

Referring to fig. 9, in a possible implementation, the apparatus may further include a training image sample obtaining module 405 and a network training module 406.

The device comprises a training image sample acquisition module 405, a comparison module and a comparison module, wherein the training image sample acquisition module 405 is used for acquiring a plurality of training image samples of a device to be detected, and each training image sample contains artificial labeling information;

and a network training module 406, configured to train the pre-trained defect detection network through the plurality of training image samples, so as to obtain the trained defect detection network.

Optionally, referring to fig. 9, the network training module 406 may include:

an image sample dividing unit 4061, configured to divide the plurality of training image samples into a plurality of image sample pairs, where each image sample pair includes two training image samples;

a feature layer obtaining unit 4062, configured to obtain, for each image sample pair, feature layers of two training image samples through the pre-trained defect detection network;

and the predicting unit 4063 is configured to predict the probability that the two training image samples belong to the same type of defect through the fully-connected layer of the pre-trained defect detection network after the two feature layers are connected.

Optionally, the image sample acquiring module 401 may be specifically configured to:

It should be noted that, for the information interaction, execution process, and other contents between the above-mentioned devices/units, the specific functions and technical effects thereof are based on the same concept as those of the embodiment of the method of the present application, and specific reference may be made to the part of the embodiment of the method, which is not described herein again.

It will be apparent to those skilled in the art that, for convenience and brevity of description, only the above-mentioned division of the functional units and modules is illustrated, and in practical applications, the above-mentioned function distribution may be performed by different functional units and modules according to needs, that is, the internal structure of the apparatus is divided into different functional units or modules to perform all or part of the above-mentioned functions. Each functional unit and module in the embodiments may be integrated in one processing unit, or each unit may exist alone physically, or two or more units are integrated in one unit, and the integrated unit may be implemented in a form of hardware, or in a form of software functional unit. In addition, specific names of the functional units and modules are only for convenience of distinguishing from each other, and are not used for limiting the protection scope of the present application. The specific working processes of the units and modules in the system may refer to the corresponding processes in the foregoing method embodiments, and are not described herein again.

An embodiment of the present application further provides a terminal device, and referring to fig. 10, the terminal device 500 may include: at least one processor 510, a memory 520, and a computer program stored in the memory 520 and operable on the at least one processor 510, wherein the processor 510, when executing the computer program, implements the steps of any of the above-described method embodiments, such as the steps S101 to S104 in the embodiment shown in fig. 2. Alternatively, the processor 510, when executing the computer program, implements the functions of the modules/units in the above-described device embodiments, for example, the functions of the modules 401 to 404 shown in fig. 8.

Illustratively, the computer program may be divided into one or more modules/units, which are stored in the memory 520 and executed by the processor 510 to accomplish the present application. The one or more modules/units may be a series of computer program segments capable of performing specific functions, which are used to describe the execution of the computer program in the terminal device 500.

Those skilled in the art will appreciate that fig. 10 is merely an example of a terminal device and is not limiting of terminal devices and may include more or fewer components than shown, or some components may be combined, or different components such as input output devices, network access devices, buses, etc.

The Processor 510 may be a Central Processing Unit (CPU), other general purpose Processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field-Programmable Gate Array (FPGA) or other Programmable logic device, discrete Gate or transistor logic, discrete hardware components, etc. A general purpose processor may be a microprocessor or the processor may be any conventional processor or the like.

The memory 520 may be an internal storage unit of the terminal device, or may be an external storage device of the terminal device, such as a plug-in hard disk, a Smart Media Card (SMC), a Secure Digital (SD) Card, a Flash memory Card (Flash Card), and the like. The memory 520 is used for storing the computer programs and other programs and data required by the terminal device. The memory 520 may also be used to temporarily store data that has been output or is to be output.

The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended ISA (EISA) bus, or the like. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, the buses in the figures of the present application are not limited to only one bus or one type of bus.

The defect detection method provided by the embodiment of the application can be applied to terminal equipment such as a computer, a tablet computer, a notebook computer, a netbook, a Personal Digital Assistant (PDA), a mobile phone and the like, and the embodiment of the application does not limit the specific type of the terminal equipment.

Take the terminal device as a computer as an example. Fig. 11 is a block diagram showing a partial structure of a computer provided in an embodiment of the present application. Referring to fig. 11, the computer includes: a communication circuit 610, a memory 620, an input unit 630, a display unit 640, an audio circuit 650, a wireless fidelity (WiFi) module 660, a processor 670, and a power supply 680. Those skilled in the art will appreciate that the computer architecture shown in FIG. 11 is not intended to be limiting of computers, and may include more or fewer components than those shown, or some components in combination, or a different arrangement of components.

The following describes each component of the computer in detail with reference to fig. 11:

the communication circuit 610 may be used for receiving and transmitting signals during a message transmission or communication process, and in particular, receives and processes an image sample transmitted by the image capturing device to the processor 670; in addition, the image acquisition instruction is sent to the image acquisition device. Typically, the communication circuit includes, but is not limited to, an antenna, at least one Amplifier, a transceiver, a coupler, a Low Noise Amplifier (LNA), a duplexer, and the like. In addition, the communication circuit 610 may also communicate with networks and other devices via wireless communication. The wireless communication may use any communication standard or protocol, including but not limited to Global System for Mobile communication (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE)), e-mail, Short Messaging Service (SMS), and the like.

The memory 620 may be used to store software programs and modules, and the processor 670 executes various functional applications of the computer and data processing by operating the software programs and modules stored in the memory 620. The memory 620 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required by at least one function (such as a sound playing function, an image playing function, etc.), and the like; the storage data area may store data (such as audio data, a phonebook, etc.) created according to the use of the computer, etc. Further, the memory 620 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device.

The input unit 630 may be used to receive input numeric or character information and generate key signal inputs related to user settings and function control of the computer. Specifically, the input unit 630 may include a touch panel 631 and other input devices 632. The touch panel 631, also referred to as a touch screen, may collect touch operations of a user (e.g., operations of the user on the touch panel 631 or near the touch panel 631 by using any suitable object or accessory such as a finger or a stylus) thereon or nearby, and drive the corresponding connection device according to a preset program. Alternatively, the touch panel 631 may include two parts of a touch detection device and a touch controller. The touch detection device detects the touch direction of a user, detects a signal brought by touch operation and transmits the signal to the touch controller; the touch controller receives touch information from the touch sensing device, converts the touch information into touch point coordinates, sends the touch point coordinates to the processor 670, and can receive and execute commands sent by the processor 670. In addition, the touch panel 631 may be implemented using various types, such as resistive, capacitive, infrared, and surface acoustic wave. The input unit 630 may include other input devices 632 in addition to the touch panel 631. In particular, other input devices 632 may include, but are not limited to, one or more of a physical keyboard, function keys (such as volume control keys, switch keys, etc.), a trackball, a mouse, a joystick, and the like.

The display unit 640 may be used to display information input by a user or information provided to the user and various menus of the computer. The Display unit 640 may include a Display panel 641, and optionally, the Display panel 641 may be configured in the form of a Liquid Crystal Display (LCD), an Organic Light-Emitting Diode (OLED), or the like. Further, the touch panel 631 can cover the display panel 641, and when the touch panel 631 detects a touch operation thereon or nearby, the touch panel is transmitted to the processor 670 to determine the type of the touch event, and then the processor 670 provides a corresponding visual output on the display panel 641 according to the type of the touch event. Although in fig. 11, the touch panel 631 and the display panel 641 are two separate components to implement the input and output functions of the computer, in some embodiments, the touch panel 631 and the display panel 641 may be integrated to implement the input and output functions of the computer.

The audio circuit 650 may provide an audio interface between a user and a computer. The audio circuit 650 may transmit the received electrical signal converted from the audio data to a speaker, and convert the electrical signal into an audio signal for output; on the other hand, the microphone converts the collected sound signal into an electric signal, which is received by the audio circuit 650 and converted into audio data, which is then processed by the audio data output processor 670 and transmitted to, for example, another computer via the communication circuit 610, or the audio data is output to the memory 620 for further processing.

WiFi belongs to short-distance wireless transmission technology, and the computer can help a user to receive and send e-mails, browse webpages, access streaming media and the like through the WiFi module 660, and provides wireless broadband internet access for the user. Although fig. 11 shows the WiFi module 660, it is understood that it does not belong to the essential constitution of the computer, and may be omitted entirely as needed within the scope not changing the essence of the invention.

The processor 670 is a control center of the computer, connects various parts of the entire computer using various interfaces and lines, performs various functions of the computer and processes data by operating or executing software programs and/or modules stored in the memory 620 and calling data stored in the memory 620, thereby monitoring the computer as a whole. Alternatively, processor 670 may include one or more processing units; alternatively, processor 670 may integrate an application processor that handles primarily the operating system, user interface, and applications, etc., and a modem processor that handles primarily wireless communications. It will be appreciated that the modem processor described above may not be integrated into processor 670.

The computer may also include a power supply 680 (e.g., a battery) to provide power to the various components, where the power supply 680 may be logically coupled to the processor 670 via a power management system to manage charging, discharging, and power consumption management functions via the power management system.

The embodiment of the present application further provides a computer-readable storage medium, where a computer program is stored, and when the computer program is executed by a processor, the computer program implements the steps in the embodiments of the defect detection method described above.

The embodiment of the present application provides a computer program product, which when running on a mobile terminal, enables the mobile terminal to implement the steps in the embodiments of the defect detection method when executed.

The integrated unit, if implemented in the form of a software functional unit and sold or used as a stand-alone product, may be stored in a computer readable storage medium. Based on such understanding, all or part of the processes in the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium and can implement the steps of the embodiments of the methods described above when the computer program is executed by a processor. Wherein the computer program comprises computer program code, which may be in the form of source code, object code, an executable file or some intermediate form, etc. The computer readable medium may include at least: any entity or device capable of carrying computer program code to a photographing apparatus/terminal apparatus, a recording medium, computer Memory, Read-Only Memory (ROM), Random Access Memory (RAM), an electrical carrier signal, a telecommunications signal, and a software distribution medium. Such as a usb-disk, a removable hard disk, a magnetic or optical disk, etc. In certain jurisdictions, computer-readable media may not be an electrical carrier signal or a telecommunications signal in accordance with legislative and patent practice.

In the above embodiments, the descriptions of the respective embodiments have respective emphasis, and reference may be made to the related descriptions of other embodiments for parts that are not described or illustrated in a certain embodiment.

Those of ordinary skill in the art will appreciate that the various illustrative elements and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware or combinations of computer software and electronic hardware. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the implementation. Skilled artisans may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present application.

In the embodiments provided in the present application, it should be understood that the disclosed apparatus/network device and method may be implemented in other ways. For example, the above-described apparatus/network device embodiments are merely illustrative, and for example, the division of the modules or units is only one logical division, and there may be other divisions when actually implementing, for example, a plurality of units or components may be combined or integrated into another system, or some features may be omitted, or not implemented. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, devices or units, and may be in an electrical, mechanical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

The above-mentioned embodiments are only used for illustrating the technical solutions of the present application, and not for limiting the same; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; such modifications and substitutions do not substantially depart from the spirit and scope of the embodiments of the present application and are intended to be included within the scope of the present application.

Claims

1. A method of defect detection, comprising:

acquiring an image sample of a device to be detected;

2. The defect detection method of claim 1, wherein the obtaining of the image sample with the detection result satisfying the preset condition for labeling comprises:

3. The method of claim 1 or 2, wherein the performing defect detection on the subsequent image sample through the trained defect detection network based on the labeled image sample comprises:

4. The defect detection method of claim 3, wherein said determining the detection of said subsequent image samples based on respective prediction probabilities comprises:

5. The defect detection method of claim 1, further comprising:

6. The defect detection method of claim 5, wherein training the pre-trained defect detection network with the plurality of training image samples comprises:

7. The defect detection method of claim 1, wherein said obtaining an image sample of the device under test comprises:

8. A defect detection apparatus, comprising:

9. A terminal device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that the processor implements the method according to any of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, in which a computer program is stored which, when being executed by a processor, carries out the method according to any one of claims 1 to 7.