CN116402771A

CN116402771A - Defect detection method and device and model training method and device

Info

Publication number: CN116402771A
Application number: CN202310297964.6A
Authority: CN
Inventors: 张午阳; 曹旸
Original assignee: Omron China Co ltd
Current assignee: Omron China Co ltd
Priority date: 2023-03-24
Filing date: 2023-03-24
Publication date: 2023-07-07

Abstract

The embodiment of the application provides a defect detection method and device and a model training method and device. The defect detection device of the appearance of the product comprises: the first processing unit is used for detecting text lines of characters in the original image to obtain a first mask image corresponding to the text lines; a second processing unit, which performs binarization processing on the region of the Chinese character line in the original image to obtain a second mask image corresponding to the character outline; a third processing unit that masks text in the original image based on the first mask image and the second mask image; and a fourth processing unit that performs defect detection on an area other than the shielded text in the original image. The method and the device can accurately shield characters in the image and improve the accuracy of defect detection.

Description

Defect detection method and device and model training method and device

Technical Field

The embodiment of the application relates to the technical field of image processing.

Background

In industrial production, it is necessary to perform appearance inspection of the produced product to ensure that defective products are not delivered to the customer. For production purposes, the information of the product is often printed on the surface of the product.

It should be noted that the foregoing description of the background art is only for the purpose of facilitating a clear and complete description of the technical solutions of the present application and for the convenience of understanding by those skilled in the art, and is not to be construed as merely illustrative of the background art section of the present application.

Disclosure of Invention

In production practice, engineers find that text printed on the surface of a product sometimes interferes with the appearance detection of the product, e.g., text on the surface of the product sometimes is detected as a defect in the appearance of the product, etc. Therefore, it is necessary to adaptively mask the characters on the surface of the product.

The inventors of the present application have found that, although the conventional technique can detect a region where a line of text in an image is located or a region where a single character is located, it is difficult to accurately divide the shape of the character, and therefore, when the conventional technique is used to mask characters on the surface of a product, a region corresponding to the shape of the characters cannot be accurately masked, for example, the masked region may have a larger range than the region corresponding to the shape of the characters, and thus, there is a case where a defect in the masked region is missed.

Aiming at least one of the technical problems, the embodiment of the application provides a defect detection method, a device, a model training method, a device and an electronic device, in the defect detection method, binarization processing is carried out on a text line area to obtain a mask image corresponding to character outlines, characters in the image are shielded based on the mask image, and defects are detected in the shielded area, so that the characters in the image can be accurately shielded, and the defect detection accuracy is improved.

According to an aspect of the embodiments of the present application, there is provided a defect detection apparatus for product appearance, the apparatus including:

the first processing unit is used for detecting text lines of characters in the original image to obtain a first mask image corresponding to the text lines;

a second processing unit, which performs binarization processing on the region of the Chinese character line in the original image to obtain a second mask image corresponding to the character outline;

a third processing unit that masks text in the original image based on the first mask image and the second mask image; and

and a fourth processing unit for detecting defects in areas outside the shielded characters in the original image.

According to another aspect of an embodiment of the present application, there is provided a model training apparatus, the apparatus including:

a fifth processing unit, which detects text lines of characters in the training image to obtain a first mask image corresponding to the text lines;

a sixth processing unit, which performs binarization processing on the region of the Chinese character line in the training image to obtain a second mask image corresponding to the character outline; and

and a seventh processing unit for training a model for detecting the outline of the character based on at least the second mask image.

According to another aspect of the embodiments of the present application, there is provided a defect detection method for product appearance, the method including:

performing text line detection on characters in an original image to obtain a first mask image corresponding to the text line;

performing binarization processing on the region of the Chinese character line in the original image to obtain a second mask image corresponding to the character outline;

shielding characters in the original image based on the first mask image and the second mask image; and

and performing defect detection on the area outside the shielded characters in the original image.

According to another aspect of an embodiment of the present application, there is provided a model training method, the method including:

text line detection is carried out on characters in the training image, and a first mask image corresponding to the text line is obtained;

performing binarization processing on the region of the Chinese character row in the training image to obtain a second mask image corresponding to the character outline; and

training a model for detecting text contours based at least on the second mask image.

According to another aspect of embodiments of the present application, there is provided an electronic device comprising a memory storing a computer program and a processor configured to execute the computer program to implement a method for defect detection and/or a method for model training of the appearance of a product as described above.

One of the beneficial effects of the embodiment of the application is that: in the defect detection method, binarization processing is carried out on the text line area to obtain a mask image corresponding to the outline of the characters, the characters in the image are shielded based on the mask image, and the defects are detected in the shielded area, so that the characters in the image can be accurately shielded, and the accuracy of defect detection is improved.

Specific implementations of the embodiments of the present application are disclosed in detail with reference to the following description and drawings, indicating the manner in which the principles of the embodiments of the present application may be employed. It should be understood that the embodiments of the present application are not limited in scope thereby. The embodiments of the present application include many variations, modifications and equivalents within the spirit and scope of the appended claims.

Drawings

The accompanying drawings, which are included to provide a further understanding of the embodiments of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. It is obvious that the drawings in the following description are only examples of the present application, and that other embodiments may be obtained from these drawings without inventive work for a person of ordinary skill in the art. In the drawings:

FIG. 1 is a schematic diagram of a method for detecting defects in the appearance of a product according to an embodiment of the present application;

FIG. 2 is a schematic illustration of an original image, a first mask image, a second mask image, and a first fused mask;

FIG. 3 is a schematic illustration of masking text in an original image based on a first mask image and a second mask image;

FIG. 4 is a schematic diagram of an example of a defect detection method for the appearance of the product of the present application;

FIG. 5 is a schematic illustration of a fusion operation based on a first mask image, a second mask image, and a third mask image;

FIG. 6 is a schematic diagram of a model training method of an embodiment of the second aspect;

FIG. 7 is a schematic diagram of a defect detection device for product appearance;

FIG. 8 is a schematic diagram of a model training apparatus;

fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present application.

Detailed Description

The foregoing and other features of embodiments of the present application will become apparent from the following description, taken in conjunction with the accompanying drawings. In the specification and drawings, there have been specifically disclosed specific embodiments of the present application which are indicative of some of the ways in which the principles of the embodiments of the present application may be employed, it being understood that the present application is not limited to the described embodiments, but, on the contrary, the embodiments of the present application include all modifications, variations and equivalents falling within the scope of the appended claims.

In the embodiments of the present application, the terms "first," "second," and the like are used to distinguish between different elements from each other by reference, but do not denote a spatial arrangement or a temporal order of the elements, and the elements should not be limited by the terms. The term "and/or" includes any and all combinations of one or more of the associated listed terms. The terms "comprises," "comprising," "including," "having," and the like, are intended to reference the presence of stated features, elements, components, or groups of components, but do not preclude the presence or addition of one or more other features, elements, components, or groups of components.

In the embodiments of the present application, the singular forms "a," an, "and" the "include plural referents and should be construed broadly to mean" one "or" one type "and not limited to" one "or" another; furthermore, the term "comprising" is to be interpreted as including both the singular and the plural, unless the context clearly dictates otherwise. Furthermore, the term "according to" should be understood as "at least partially according to … …", and the term "based on" should be understood as "based at least partially on … …", unless the context clearly indicates otherwise.

Features that are described and/or illustrated with respect to one embodiment may be used in the same way or in a similar way in one or more other embodiments in combination with or instead of the features of the other embodiments. The term "comprises/comprising" when used herein refers to the presence of a feature, integer, step or component, but does not exclude the presence or addition of one or more other features, integers, steps or components.

Example of the first aspect

The embodiment of the application provides a defect detection method for product appearance.

Fig. 1 is a schematic diagram of a defect detection method for product appearance according to an embodiment of the present application. As shown in fig. 1, the defect detection method for the appearance of the product comprises the following steps:

operation 101, detecting text lines of characters in an original image to obtain a first mask image corresponding to the text lines;

an operation 102, performing binarization processing on the region of the Chinese character line in the original image to obtain a second mask image corresponding to the character outline;

operation 103, shielding characters in the original image based on the first mask image and the second mask image; and

and 104, performing defect detection on the area outside the shielded characters in the original image.

In the defect detection method, binarization processing is carried out on the text line area to obtain a mask image corresponding to the outline of the characters, the characters in the image are shielded based on the mask image, and the defects are detected in the shielded area, so that the characters in the image can be accurately shielded, and the accuracy of defect detection is improved.

Next, a defect detection method of the product appearance of the present application will be described with reference to the drawings.

Fig. 2 is a schematic diagram of an original image, a first mask image, a second mask image, and a first fused mask. Where (a) of fig. 2 represents a first original image, (b) of fig. 2 represents a schematic view of applying a first mask image to the original image, (c) of fig. 2 represents a second mask image, and (d) of fig. 2 represents a schematic view of applying a first fusion mask to the original image.

In some embodiments, the original image may be an image of a printed text surface of a product, such as a printed wiring board, electronic component, metal element, or the like. As shown in fig. 2 (a), the original image has a letter 21 therein.

In operation 101, text line detection is performed on text in an original image, thereby detecting at least one line of text. As shown in fig. 2 (b), the region 22 corresponds to a region of a text line. The area 22 of the text line may be the foreground and the other areas in the original image may be the background. The specific method of text line detection in operation 101 may refer to related art, for example, text line detection may be performed using visual image processing techniques, or text line detection may be performed using an Artificial Intelligence (AI) model obtained based on deep learning.

As shown in fig. 2 (b), the text line detection can specify the text line region 22, but it is difficult to accurately detect the region outside the outline of the text 21.

In operation 102, a binarization process may be performed for the region of the text line, thereby obtaining a second mask image corresponding to the text outline.

In some embodiments of operation 102, the pixel values of the foreground pixels of the first mask image obtained in operation 101 may be multiplied by the pixel values of the pixels at corresponding locations in the original image, for example, so that the background outside the area of the text line may be masked, leaving only the image of the area of the text line (e.g., area 22 of fig. 2 (b)); then, the image of the region of the text line is subjected to a binarization operation (i.e., local binarization) to obtain a second mask image corresponding to the text outline 23. In the binarization operation, a pixel value of a pixel point having a pixel value or a gradation value lower than a predetermined value may be set to 0 (black), and a pixel value of a pixel point having a pixel value or a gradation value greater than or equal to a predetermined value may be set to 255 (white).

As shown in fig. 2 (c), in the second mask image, the text line has a text outline 23 in the region.

In operation 103, text in the original image may be masked based on the first mask image and the second mask image.

Fig. 3 is a schematic diagram of masking text in an original image based on a first mask image and a second mask image for implementing operation 103.

As shown in fig. 3, the method for masking text in an original image based on a first mask image and a second mask image includes:

operation 301, fusing the first mask image and the second mask image to obtain a first fused mask image; and

operation 302 applies a first fused mask image to the original image to mask regions of the original image corresponding to text contours.

In operation 301, an exclusive or operation at a pixel unit level may be performed on the first mask image and the second mask image, thereby obtaining a first fusion mask. The exclusive or operation at the pixel unit level is, for example, an exclusive or operation of pixel values is performed on a pixel unit of the first mask image and a pixel unit of the same position of the second mask image. Wherein each pixel unit may include one pixel, or more than one pixel.

In addition, in operation 301, the exclusive or operation is not limited, and other types of operations, such as and operation, are also possible, and the present application is not limited thereto.

As shown in fig. 2 (d), in operation 302, a first fused mask image is applied to the original image, so that the region corresponding to the outline 24 of the text in the first fused mask image is masked from the original image, thereby determining the region 25 other than the text in the original image.

In operation 104, defect detection is performed on an area (e.g., an area 25 shown in fig. 2 (d)) other than text in the original image, for example, contamination, breakage, foreign matter, or the like in the area is detected. The method for performing defect detection may refer to the related art, and for example, detection may be performed by a visual detection method, or detection may be performed by an AI model based on deep learning, or the like.

Further, in operation 302, both the third mask image and the first fused mask image may be applied to the original image, wherein the third mask image may correspond to an area within the border of the product in the original image, for example, 20 within the border of the product shown in fig. 2 (a). For example, the third mask image is fused with the first mask image and the second mask image to obtain a fused mask image, and characters in the original image and areas outside the frames of products in the original image are shielded by using the fused mask image. Thus, in operation 104, defect detection can be performed on the region except for the text in the region inside the frame of the product, so that the region range of defect detection can be reduced, and the detection efficiency can be improved.

Fig. 4 is a schematic diagram of an example of a defect detection method of the product appearance of the present application, corresponding to

operations

101, 102, and 103 described above.

As shown in fig. 4, text line detection 42 is performed on the original image 41, thereby obtaining a first mask image 43;

applying the first mask image 43 to the original image 41, for example, multiplying 44 the foreground of the first mask image 43 (e.g., white pixels in the first mask image 43) with the corresponding pixels on the original image 41, thereby obtaining an intermediate image 45, in which intermediate image 45 the text within the text line area is displayed;

binarizing (i.e., locally binarizing) 46 the text line area of the intermediate image 45 to obtain a second mask image 47;

performing a fusion operation 49 based on the first mask image 43 and the second mask image 47, thereby obtaining a first fused mask image 50;

applying the first fused mask image 50 to the original image 41 to obtain a masked image 51, and performing defect detection in a region 512 of the masked image 51;

in addition, in the fusion operation 49 of the present application, fusion may be performed based on the first mask image 43, the second mask image 47, and the third mask image 48, and thus, in the fusion operation 49, other fused mask images than the first fused mask image 50 may be generated, and for example, the other fused mask images may include a third fused mask image described later. Wherein the third mask image 48 may correspond to an area within the border of the product in the original image, for example, 20 within the border of the product shown in fig. 2 (a).

Fig. 5 is a schematic diagram of a fusion operation based on the first mask image, the second mask image, and the third mask image for implementing the fusion operation 49 of fig. 4. As shown in fig. 5, the fusing operation 49 may include the following operations:

performing an exclusive or operation 52 at a pixel unit level (e.g., pixel level) on the first mask image 43 and the second mask image 47 to obtain a first fused mask image 50;

the pixel values of each pixel unit (for example, one pixel unit has one pixel) in the first fused mask image 50 are subjected to a negation process 53, so as to obtain a first fused inverse mask image 54;

the third mask image 48 is xored 55 at a pixel unit level (e.g., pixel level) with the first fused inverse mask image 54 to obtain a second fused mask image 56.

The first 50 and second 56 fused mask images generated by the fusion operation 49 of fig. 5, as well as the third mask image 48, may be used to mask text in the original image; in addition, at least one of the first, second, and third fused

mask images

50, 56, 48 may also be used for training to obtain a model that detects text in the images.

In the method for detecting defects in the first aspect of the embodiments of the present application, binarization processing is performed on a text line region to obtain a mask image corresponding to a text outline, and text in the image is shielded based on the mask image, so that defects are detected in the shielded region, and therefore, text in the image can be accurately shielded, and accuracy of defect detection is improved.

The above only describes each step or process related to the present application, but the present application is not limited thereto. The method of defect detection may also comprise other steps or processes, for the details of which reference may be made to the prior art.

The above embodiments are merely illustrative of the embodiments of the present application, but the present application is not limited thereto, and appropriate modifications may be made on the basis of the above embodiments. For example, each of the above embodiments may be used alone, or one or more of the above embodiments may be combined.

Embodiments of the second aspect

Embodiments of the second aspect of the present application provide a model training method. The model training method is used for training a model, and the model is used for detecting characters in an image.

FIG. 6 is a schematic diagram of a model training method of an embodiment of the second aspect, as shown in FIG. 6, comprising:

operation 601, performing text line detection on characters in a training image to obtain a first mask image corresponding to the text line;

operation 602, performing binarization processing on the region of the Chinese character line in the training image to obtain a second mask image corresponding to the character outline; and

operation 603 trains a model for detecting text contours based at least on the second mask image.

The

description regarding operations

601, 602 may refer to the

description regarding operations

101, 102 in the embodiment of the first aspect, except that

operations

101, 102 are directed to original images, and

operations

601, 602 of the present application are directed to training images. The training image may be from a training set, for example.

In operation 603, at least one of the first fused mask image and the second fused mask image may be input into a model to be trained, thereby training the model to be trained. In addition, in operation 603, a third mask image may also be input into the model to be trained, thereby training the model to be trained.

Wherein the first fused mask image is based on the second mask image; the second fused mask image is obtained based on the first fused mask image and a third mask image corresponding to the region to be detected in the training image.

In this application, the method for obtaining the first fused mask image and the second fused mask image may refer to an embodiment of the first aspect.

For example, the first mask image and the second mask image are subjected to an exclusive or operation at a pixel unit level (for example, one pixel unit has one pixel therein), thereby obtaining a first fused mask image.

For another example, pixel values of each pixel unit (for example, one pixel unit has one pixel) in the first fused mask image are subjected to inverse processing, so as to obtain a first fused inverse mask image; and then, performing exclusive OR operation on a third mask image corresponding to the region to be detected in the training image and the first fused inverse mask image to obtain a second fused mask image.

In operation 603, the model to be trained may be a neural network model, which may have the structure of a transducer model, for example, a SegFormer structure. For a detailed description of the SegFormer structure, reference may be made to the relevant art, for example https:// arxiv. Org/pdf/2105.15203.Pdf.

In operation 603, the neural network has a global receptive field based on the transducer structure, and can fully extract text line features; in addition, various image enhancements (e.g., shading, random cropping, random scaling, etc.) may be performed on the training data during the training process of operation 603 to enhance the robustness of the model; in operation 603, an average cross-over-the-sum (m-IOU) score on the training set may be used as a model preference index.

Further, the online hard-case mining cross entropy (OHEM cross entropy) may be employed as a loss (loss) function in operation 603. For example, for a plurality of first fused mask images, respectively calculating a loss function, and averaging to obtain a first average loss function for the plurality of first fused mask images; for another example, the loss function is calculated for each of the plurality of second fused mask images, and the second average loss function for the plurality of second fused mask images is obtained by averaging. Among them, reference is made to the related art as to how to train with the loss function.

In the application, the mask of the character shape can be automatically obtained through the

operations

601 and 602, so that the workload of manually marking the character shape is greatly reduced, and the training efficiency can be improved when the first fusion mask and/or the second fusion mask are used as training elements for training the model, and the detection accuracy of the trained model on the characters can be improved.

The above only describes each step or process related to the present application, but the present application is not limited thereto. The model training method may also comprise other steps or processes, for the details of which reference may be made to the prior art.

Embodiments of the third aspect

An embodiment of the third aspect relates to a defect detection device for product appearance, which corresponds to the defect detection method for product appearance according to the embodiment of the first aspect.

Fig. 7 is a schematic diagram of a defect detecting apparatus for product appearance, and as shown in fig. 7, a defect detecting apparatus 700 for product appearance includes:

a first processing unit 701, which detects text lines of characters in an original image to obtain a first mask image corresponding to the text lines;

a second processing unit 702, performing binarization processing on the region of the Chinese character line in the original image to obtain a second mask image corresponding to the character outline;

a third processing unit 703 that masks characters in the original image based on the first mask image and the second mask image; and

and a fourth processing unit 704 for performing defect detection on the area outside the shielded text in the original image.

In some embodiments, masking text in the original image based on the first mask image and the second mask image comprises:

fusing the first mask image and the second mask image to obtain a first fused mask image; and

and applying the first fusion mask image to the original image so as to shield the area corresponding to the text outline in the original image.

In some embodiments, fusing the first mask image and the second mask image includes:

and performing pixel unit-level exclusive-or operation on the first mask image and the second mask image.

It should be noted that only the respective components or modules related to the present application are described above, but the present application is not limited thereto. The defect detecting device 700 of the product appearance may further include other components or modules, and regarding the specific contents of these components or modules, reference may be made to the related art.

For simplicity, the connection relationships or signal trends between the various components or modules are shown only by way of example in fig. 7, but it should be apparent to those skilled in the art that various related techniques such as bus connections may be employed. The above-described respective components or modules may be implemented by hardware means such as a processor, a memory, or the like; the embodiments of the present application are not limited in this regard.

Embodiments of the fourth aspect

An embodiment of the fourth aspect relates to a model training apparatus corresponding to the model training method of the embodiment of the second aspect.

Fig. 8 is a schematic diagram of a model training apparatus, as shown in fig. 8, the model training apparatus 800 includes:

a fifth processing unit 801, which detects text lines of characters in the training image to obtain a first mask image corresponding to the text lines;

a sixth processing unit 802, configured to perform binarization processing on the region of the Chinese character line in the training image, so as to obtain a second mask image corresponding to the text outline; and

a seventh processing unit 803 that trains a model for detecting a text outline based on at least the second mask image.

In some embodiments, training the model based at least on the second mask image includes:

inputting at least one of the first fused mask image and the second fused mask image into the model, training the model,

wherein the first fused mask image is obtained based on the second mask image,

the second fused mask image is obtained based on the first fused mask image and a third mask image corresponding to the region to be detected in the training image.

In some embodiments, training the model based at least on the second mask image further comprises:

and carrying out inversion processing on pixel values of each pixel unit in the first fusion mask image to obtain a first fusion inverse mask image.

and performing exclusive OR operation on the third mask image corresponding to the region to be detected in the training image and the first fusion inverse mask image to obtain a second fusion mask image.

It should be noted that only the respective components or modules related to the present application are described above, but the present application is not limited thereto. Model training apparatus 800 may also include other components or modules, for the details of which reference may be made to the related art.

For simplicity, the connection relationships or signal trends between the various components or modules are shown only by way of example in fig. 8, but it should be apparent to those skilled in the art that various related techniques such as bus connections may be employed. The above-described respective components or modules may be implemented by hardware means such as a processor, a memory, or the like; the embodiments of the present application are not limited in this regard.

Embodiments of the third aspect

Embodiments of the present application provide an electronic device comprising a defect detection device 700 for product appearance according to embodiments of the third aspect and/or a model training device 800 according to embodiments of the fourth aspect, the contents of which are incorporated herein. The electronic device may be, for example, a computer, server, workstation, laptop, smart phone, etc.; embodiments of the present application are not so limited.

Fig. 9 is a schematic diagram of an electronic device according to an embodiment of the present application. As shown in fig. 9, the electronic device 900 may include: a processor (e.g., central processing unit, CPU) 910 and a memory 920; memory 920 is coupled to central processor 910. Wherein the memory 920 may store various data; further, a program 921 for information processing is stored, and the program 921 is executed under the control of the processor 910.

In some embodiments, the functionality of the defect detection device 700 and/or the model training device 800 for the appearance of the product is integrated into the processor 910 for implementation. Wherein the processor 910 is configured to implement the method as described in the embodiments of the first aspect and/or the embodiments of the second aspect.

In some embodiments, the defect detection device 700 and/or the model training device 800 for the product appearance are configured separately from the processor 910, for example, the defect detection device 700 and/or the model training device 800 for the product appearance may be configured as a chip connected to the processor 910, and the functions of the defect detection device 700 and/or the model training device 800 for the product appearance are implemented by the control of the processor 910.

In addition, as shown in fig. 9, the electronic device 900 may further include: input output (I/O) devices 930 and a display 940; wherein, the functions of the above components are similar to the prior art, and are not repeated here. It is noted that the electronic device 900 need not include all of the components shown in fig. 9; in addition, the electronic device 900 may further include components not shown in fig. 9, and reference may be made to the related art.

Embodiments of the present application also provide a computer readable program, wherein the program, when executed in an electronic device, causes the computer to perform the method for detecting defects in the appearance of a product as described in the embodiments of the first aspect and/or the method for model training as described in the embodiments of the second aspect in the electronic device.

Embodiments of the present application also provide a storage medium storing a computer readable program, where the computer readable program causes a computer to execute the defect detection method of the product appearance according to the embodiments of the first aspect and/or the model training method according to the embodiments of the second aspect in an electronic device.

The apparatus and method of the present application may be implemented by hardware, or may be implemented by hardware in combination with software. The present application relates to a computer readable program which, when executed by a logic means, enables the logic means to carry out the apparatus or constituent means described above, or enables the logic means to carry out the various methods or steps described above. The present application also relates to a storage medium such as a hard disk, a magnetic disk, an optical disk, a DVD, a flash memory, or the like for storing the above program.

The methods/apparatus described in connection with the embodiments of the present application may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. For example, one or more of the functional blocks shown in the figures and/or one or more combinations of the functional blocks may correspond to individual software modules or individual hardware modules of the computer program flow. These software modules may correspond to the individual steps shown in the figures, respectively. These hardware modules may be implemented, for example, by solidifying the software modules using a Field Programmable Gate Array (FPGA).

A software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. A storage medium may be coupled to the processor such that the processor can read information from, and write information to, the storage medium; or the storage medium may be integral to the processor. The processor and the storage medium may reside in an ASIC. The software modules may be stored in the memory of the mobile terminal or in a memory card that is insertable into the mobile terminal. For example, if the apparatus (e.g., mobile terminal) employs a MEGA-SIM card of a relatively large capacity or a flash memory device of a large capacity, the software module may be stored in the MEGA-SIM card or the flash memory device of a large capacity.

One or more of the functional blocks described in the figures and/or one or more combinations of functional blocks may be implemented as a general purpose processor, a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any suitable combination thereof for use in performing the functions described herein. One or more of the functional blocks described with respect to the figures and/or one or more combinations of functional blocks may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP communication, or any other such configuration.

The present application has been described in connection with specific embodiments, but it should be apparent to those skilled in the art that these descriptions are intended to be illustrative and not limiting. Various modifications and adaptations of the disclosure may occur to those skilled in the art and are within the scope of the disclosure.

The application also provides the following supplementary notes:

1. a method for detecting defects in the appearance of a product, the method comprising:

2. The method of appendix 1, wherein,

based on the first mask image and the second mask image, shielding characters in the original image comprises the following steps:

3. The method of supplementary note 2, wherein,

fusing the first mask image and the second mask image, including:

4. A method of model training, the method comprising:

5. The method of supplementary note 4, wherein training the model based at least on the second mask image includes:

wherein the first fused mask image is obtained based on the second mask image,

6. The method of supplementary note 5, wherein training the model based at least on the second mask image, further comprises:

7. The method of supplementary note 6, wherein,

fusing the first mask image and the second mask image, including:

8. The method of supplementary note 6, wherein training the model based at least on the second mask image, further comprises:

9. A storage medium storing a computer readable program, wherein the computer readable program causes a processor coupled to the storage medium to perform the method of any one of appendix 1 to appendix 8.

Claims

1. A defect detection device for product appearance, the device comprising:

2. The apparatus of claim 1, wherein,

3. The apparatus of claim 2, wherein,

fusing the first mask image and the second mask image, including:

4. A model training apparatus, the apparatus comprising:

5. The apparatus of claim 4, wherein training the model based at least on the second mask image comprises:

wherein the first fused mask image is obtained based on the second mask image,

6. The apparatus of claim 5, wherein training the model based at least on the second mask image further comprises:

7. The apparatus of claim 6, wherein,

fusing the first mask image and the second mask image, including:

8. The apparatus of claim 6, wherein training the model based at least on the second mask image further comprises:

9. A method for detecting defects in the appearance of a product, the method comprising:

10. A method of model training, the method comprising: