CN117523300A - Product abnormality detection method, device, equipment and medium - Google Patents

Abstract

The invention discloses a method, a device, equipment and a medium for detecting abnormality of a product, and relates to the technical field of abnormality detection. The method comprises the following steps: obtaining an appearance image of a product to be detected; performing target detection on the appearance image by adopting a target detection model to obtain at least one target area of the appearance image and target categories of the target areas; querying candidate abnormal detection results matched with the target category of the target area aiming at each target area; and determining a target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category. According to the technical scheme provided by the embodiment of the invention, the final detection result of the product to be detected is comprehensively determined according to each abnormal region, the abnormal type of each abnormal region and the candidate abnormal detection result of each abnormal type, and compared with the method for taking the abnormal detection result with the highest confidence as the final detection result in the prior art, the method has the advantage that the accuracy of abnormal detection is improved.

Description

Product abnormality detection method, device, equipment and medium

Technical Field

The present invention relates to the field of anomaly detection technologies, and in particular, to a method, an apparatus, a device, and a medium for anomaly detection of a product.

Background

AOI (Automatic Optic Inspection, automated optical inspection) inspection technology is a method that utilizes optical systems and image processing to inspect and evaluate products such as circuit boards and electrical cells, as well as soldering processes. AOI detection technology plays an important role in improving production efficiency, reducing human errors and improving quality control. In the production process and the welding process of the product, the method can be used for ensuring the consistency and the reliability of the product and reducing the defect rate.

However, the traditional AOI detection technology has lower accuracy when carrying out abnormal detection on complex product images, so that the overskill Rate of the products is higher, a large number of qualified products are misjudged as bad products, the production line needs to consume excessive manpower and time to reprocess the qualified products, the productivity of the products is reduced, and the unnecessary production cost is increased.

Disclosure of Invention

The invention provides a method, a device, equipment and a medium for detecting abnormality of a product, which are used for improving the accuracy of detecting the abnormality of the product.

In a first aspect, the present invention provides a method for detecting anomalies in a product, comprising:

obtaining an appearance image of a product to be detected;

performing target detection on the appearance image by adopting a target detection model to obtain at least one target area of the appearance image and target categories of the target areas;

Querying candidate abnormal detection results matched with the target category of the target area aiming at each target area;

and determining a target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category.

In a second aspect, the present invention also provides an abnormality detection apparatus for a product, including:

the image acquisition module is used for acquiring an appearance image of the product to be detected;

the target category determining module is used for carrying out target detection on the appearance image by adopting a target detection model to obtain at least one target area of the appearance image and the target category of each target area;

the candidate result determining module is used for querying candidate abnormal detection results matched with the target category of the target area aiming at each target area;

the target result determining module is used for determining a target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category.

In a third aspect, an embodiment of the present invention further provides an electronic device, including:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein the method comprises the steps of

The memory stores instructions executable by the at least one processor to enable the at least one processor to perform the anomaly detection method for the product provided by any one of the embodiments of the present invention.

In a fourth aspect, embodiments of the present invention further provide a computer readable storage medium storing computer instructions for causing a processor to execute the method for detecting an anomaly of a product according to any one of the embodiments of the present invention.

The embodiment of the invention acquires the appearance image of the product to be detected; performing target detection on the appearance image by adopting a target detection model to obtain at least one target area of the appearance image and target categories of the target areas; querying candidate abnormal detection results matched with the target category of the target area aiming at each target area; and determining a target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category. Compared with the technical scheme that the detection result with the highest confidence is used as the final detection result of the product in the prior art, the technical scheme provided by the embodiment of the invention can determine the appearance of the product to be detected, a plurality of target areas and the target categories of the target areas, comprehensively determine the target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category, and improve the abnormality detection accuracy.

It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.

FIG. 1 is a flow chart of a method for anomaly detection of a product according to a first embodiment of the present invention;

FIG. 2 is a flow chart of a method for detecting anomalies in a product according to a second embodiment of the present invention;

fig. 3 is a schematic structural view of an abnormality detecting device for a product according to a third embodiment of the present invention;

fig. 4 is a schematic structural diagram of an electronic device implementing a method for detecting abnormality of a product according to an embodiment of the present invention.

Detailed Description

In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.

It should be noted that the terms "first" and "second" and the like in the description and the claims of the present invention and the above drawings are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

In the technical scheme of the embodiment of the invention, the acquisition, storage, application and the like of the related appearance images and the like all accord with the regulations of related laws and regulations, and the prior art is not violated.

Example 1

Fig. 1 is a flowchart of a method for detecting an abnormality of a product according to an embodiment of the present invention, where the method may be performed by an apparatus for detecting an abnormality of a product, and the apparatus for detecting an abnormality of a product may be implemented in hardware and/or software, and specifically configured in an electronic device, such as a server.

Referring to the abnormality detection method of the product shown in fig. 1, the abnormality detection method includes:

s101, obtaining an appearance image of a product to be detected.

In this embodiment, the product to be detected is a product waiting for abnormal detection, and the product may be, for example, a battery cell, a circuit board, a welded product, etc.; the welding product may be a product obtained by welding using a welding material. The method for acquiring the appearance image of the product to be detected is not limited, and for example, the appearance image of the product to be detected can be directly acquired through an image acquisition device such as a camera or a camera, or the appearance image of the product to be detected can be acquired from an image library for storing the appearance images of all the products.

S102, performing target detection on the appearance image by adopting a target detection model to obtain at least one target area of the appearance image and target categories of the target areas.

In this embodiment, the target detection model may be used to perform target detection on an appearance image of a product; the object detection models of various types of products are different. The target region may be a region in which a target exists in the image to be detected. The target category is the category of the target in the target area. The target may refer to a defect on the outer surface of the product.

The target detection model is an exemplary target detection model of the battery cell if the product to be detected is the battery cell; the target categories may include, but are not limited to, at least one of dirt categories, hole categories, crack categories, stain categories, protrusion categories, pit categories, and the like; if the product to be detected is a welding product, the target detection model is a target detection model of the welding product; the target categories may include, but are not limited to, at least one of an overspray category, an underspray category, and the like.

Specifically, a target detection model corresponding to a product to be detected is adopted to carry out target detection on an appearance image of the product to be detected, and at least one target area of the appearance image and target types of all target areas are obtained.

S103, inquiring candidate abnormal detection results matched with the target category of the target area aiming at each target area.

In the present embodiment, the candidate abnormality detection results may include, but are not limited to, normal, abnormal, and the like.

Specifically, for each target area, in the preset matching information of the target category and the candidate abnormality detection result, the candidate abnormality detection result matched with the target category of the target area is queried. It should be noted that, the matching information between the target category and the candidate anomaly detection result may be set by the technician according to the actual requirement or practical experience, which is not limited in the present invention. Illustratively, the normal is taken as a candidate abnormal detection result of the stain category matching; and taking the abnormality as a candidate abnormality detection result of other target categories besides the stain removal point category.

S104, determining a target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category.

In this embodiment, the target abnormality detection result may be a final abnormality detection result of the product to be detected. Specifically, a certain algorithm is adopted, and a target abnormality detection result of a product to be detected is determined according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category.

In an alternative embodiment, before the appearance image of the product to be detected is acquired, the method further comprises: acquiring at least one sample abnormal image and labeling data of the sample abnormal image; the labeling data comprises a labeling target area and a labeling target category of the labeling target area; rotating the sample abnormal image to obtain a rotated sample image, and taking the labeling data of the sample abnormal image as the labeling data of the rotated sample image; and taking the sample abnormal image and the rotation sample image as candidate abnormal images, and training a preset deep learning model according to the labeling data of each candidate abnormal image and each candidate abnormal image to obtain a target detection model.

The sample abnormal image may be an appearance image of a product with abnormal appearance, and the number of the sample abnormal images is at least one, usually a plurality of sample abnormal images. The labeling target region may be a region of the existing target labeled on the sample abnormal image. The labeling target category may be a category of targets in the labeling target region. The rotated sample image may be a rotated sample anomaly image.

Specifically, for each sample abnormal image, the sample abnormal image is rotated according to a preset angle, and at least one rotation abnormal image is obtained. It should be noted that, the number and the numerical value of the preset angles can be set independently by a technician according to actual requirements or practical experience, and the invention is not limited to this; illustratively, the preset angles may be 90 degrees, 180 degrees, and 270 degrees, respectively, or the preset angles may be 60 degrees and 120 degrees, respectively; the type of the deep learning model may be set autonomously by the technician, and for example, the deep learning model may be an SSD (Single Shot MultiBox Detector, single-pass polygon detection) model, a YOLO (You Only Live Once) model, or the like.

It should be noted that, each abnormal image of the sample is an abnormal image of the sample of the same product, so as to obtain a target detection model of the product. Exemplary, obtaining a sample abnormal image of at least one cell and labeling data of the sample abnormal image of the cell; rotating the sample abnormal image of the battery cell to obtain a rotating sample image of the battery cell, and taking the labeling data of the sample abnormal image of the battery cell as the labeling data of the rotating sample image of the battery cell; and training a preset deep learning model according to the labeling data of each candidate abnormal image of the battery cell and each candidate abnormal image of the battery cell to obtain a target detection model of the battery cell.

It can be appreciated that by adopting the technical scheme, the sample abnormal image is rotated to obtain a rotated sample image; the rotating sample image and the sample abnormal image are used as candidate abnormal images, and the preset deep learning model is trained, so that the target detection model obtained through training can detect defects of the outer surfaces of products with different angles in the appearance images of the products, and the accuracy of target detection of the appearance images of the products to be detected is improved.

Optionally, after obtaining the target detection model, the method further includes: performing anomaly detection on each candidate anomaly image by adopting a target detection model, determining detection data of each candidate anomaly image, and checking whether the candidate anomaly images with different detection data and labeling data exist or not; if the checking result is that the candidate abnormal images with different detection data and labeling data exist, the candidate abnormal images with the same detection data and labeling data are used as correct candidate images, and the candidate abnormal images with different detection data and labeling data are used as error candidate images; generating prompt information of the error candidate image so as to prompt a technician to correct the labeling data of the error candidate image and input the corrected labeling data of the error candidate image; receiving correction labeling data of an error abnormal image, updating the candidate abnormal image into a correct candidate image and an error candidate image, training a preset deep learning model according to the labeling data of the candidate abnormal image and the candidate abnormal image to obtain a corrected abnormal detection model, and updating the target detection model into the corrected abnormal detection model; and returning to execute the step of adopting the target detection model to perform anomaly detection on each candidate abnormal image, determining detection data of each candidate abnormal image, and checking whether the candidate abnormal image with different detection data and different labeling data exists or not until the candidate abnormal image with different detection data and different labeling data does not exist.

The detection data may include, but is not limited to, detection target areas and detection target categories. The detection target area may be a target area in the appearance image detected by the target detection model, and the detection target category may be a category of a target in the target area detected by the target detection model. The corrected annotation data may be corrected annotation data.

It can be understood that by adopting the above technical scheme, whether the detection data of the candidate abnormal image is different from the labeling data of the candidate abnormal image or not is obtained by checking the target detection model, the detection accuracy of the target detection model is determined, when the candidate abnormal image with different detection data and labeling data exists, the prompting information of the candidate abnormal image is generated, so that a technician corrects the labeling data of the wrong candidate image, trains a preset deep learning model according to the labeling data of the candidate abnormal image and the labeling data of the candidate abnormal image, and the corrected abnormal detection model is obtained, thereby improving the accuracy of the target detection model.

In an alternative embodiment, after the candidate abnormal image having the detection data different from the labeling data is used as the error candidate image, the method further includes: for each error candidate image, determining a region phase difference distance between the region position of the detection target region of the error candidate image and the region position of the labeling target region of the error candidate image, and determining a category similarity between the detection region category of the error candidate image and the labeling region category of the error candidate image; according to the difference distance and the category similarity of the areas, determining the similarity between the detection data and the labeling data of the error candidate images; if the similarity is greater than or equal to a preset similarity threshold, generating first prompt information to prompt a technician to check whether the labeling data of the error candidate image is wrong, correcting the labeling data, and inputting corrected labeling data of the error candidate image; if the similarity is smaller than a preset similarity threshold, generating second prompt information to prompt a technician to check whether the detection data of the error candidate image is the illusion of the target detection model, correcting the labeling target category of the error candidate image into the illusion category, and inputting the corrected labeling data of the error candidate image. It should be noted that, the preset similarity threshold may be set by the technician according to the actual requirement or practical experience.

In one embodiment, the detection region class is converted into a first vector, the labeling region class is converted into a second vector, and cosine similarity between the first vector and the second vector is determined to be class similarity; according to the difference distance and the category similarity of the areas, determining the similarity between the detection data and the labeling data of the error candidate images; and carrying out weighted summation on the regional phase difference distance and the category similarity to obtain the similarity between the detection data and the labeling data of the error candidate image.

It can be understood that, by adopting the above technical scheme, if the similarity is smaller than the preset similarity threshold, the second prompt message is generated to prompt the technician to check whether the detection data of the error candidate image is the illusion of the target detection model, and correct the labeling target category of the error candidate image into the illusion category, so that the probability of generating the illusion result of the error of the target detection model can be reduced, and the accuracy of target detection is improved.

In an alternative embodiment, the second prompt is further used to prompt the technician for: if the detection data of the error candidate image is the illusion of the target detection model, checking whether the error candidate image is a rotation sample image, and if the error candidate image is the rotation sample image, removing the error candidate image from the candidate sample image and removing the labeling data of the error candidate image; and inputting the candidate sample image with the error candidate image removed and the labeling data of the candidate sample image with the error candidate image removed so as to train the deep learning model.

The embodiment of the invention acquires the appearance image of the product to be detected; performing target detection on the appearance image by adopting a target detection model to obtain at least one target area of the appearance image and target categories of the target areas; querying candidate abnormal detection results matched with the target category of the target area aiming at each target area; and determining a target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category. Compared with the technical scheme that the detection result with the highest confidence is used as the final detection result of the product in the prior art, the technical scheme provided by the embodiment of the invention can determine the appearance of the product to be detected, a plurality of target areas and the target categories of the target areas, comprehensively determine the target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category, and improve the abnormality detection accuracy.

Example two

Fig. 2 is a flowchart of a method for detecting an abnormality of a product according to a second embodiment of the present invention, where the determining operation of the target abnormality detection result of the product to be detected is optimized and improved based on the technical solution of the foregoing embodiment.

Further, the method comprises the steps of determining the target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category, and refining the target abnormality detection result of the product to be detected into the target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category, so as to perfect the determination operation of the target abnormality detection result of the product to be detected.

In the embodiments of the present invention, the details are not described, and reference may be made to the description of the foregoing embodiments.

Referring to fig. 2, the abnormality detection method of the product includes:

s201, obtaining an appearance image of a product to be detected.

S202, performing target detection on the appearance image by adopting a target detection model to obtain at least one target area of the appearance image and target categories of the target areas.

S203, inquiring candidate abnormal detection results matched with the target category of the target area aiming at each target area.

S204, determining an auxiliary target area according to the target category of each target area and the candidate abnormal detection result of the matching of each target category, and taking the target category of the auxiliary target area as the auxiliary target category.

In this embodiment, the auxiliary target area may be a target area having the greatest influence on the target abnormality detection result. The auxiliary target category is the category of the target in the auxiliary target area. Specifically, a certain algorithm is adopted, an auxiliary target area is determined according to the target category of each target area and the candidate abnormal detection result matched with each target category, and the target category of the auxiliary target area is taken as an auxiliary target category.

Optionally, determining the auxiliary target area according to the target category of each target area and the candidate abnormal detection result matched with each target category includes: if the number of the target areas is one, the target areas are taken as auxiliary target areas; if the number of the target areas is at least two, determining an auxiliary target area according to the confidence coefficient of the target category of each target area and the candidate abnormal detection result of the matching of each target category.

Specifically, if the number of the target areas is at least two, a certain algorithm is adopted to determine the auxiliary target area according to the confidence coefficient of the target category of each target area and the candidate abnormal detection result matched with each target category. It can be understood that, by adopting the above technical scheme, if the number of the target areas is one, the target area is taken as an auxiliary target area; if the number of the target areas is at least two, determining an auxiliary target area according to the confidence coefficient of the target category of each target area and the candidate abnormal detection result matched with each target category, and further determining the target abnormal detection result of the product to be detected according to the candidate abnormal detection result matched with the auxiliary target category.

Optionally, determining the auxiliary target area according to the confidence coefficient of the target category of each target area and the candidate abnormal detection result matched with each target category includes: taking the target area with the lowest confidence coefficient of the target category as a first target area in the target areas with the abnormal candidate abnormal detection results of the target category matching, and taking the target area with the highest confidence coefficient of the target category as a second target area in the target areas with the normal candidate abnormal detection results of the target category matching; determining a confidence difference between the confidence of the target class of the first target region and the confidence of the target class of the second target region; and determining an auxiliary target area from the first target area and the second target area according to the confidence difference.

Specifically, a certain algorithm is adopted, and an auxiliary target area is determined from the first target area and the second target area according to the confidence difference value. It can be appreciated that by adopting the above technical scheme, the first target area with the lowest confidence coefficient of the target category and the second target area with the highest confidence coefficient of the target category are determined, and the auxiliary target area is determined from the first and second abnormal areas according to the confidence coefficient difference between the first and second target areas, and then the target abnormal detection result of the product to be detected is determined according to the candidate abnormal detection result matched with the auxiliary target category, thereby improving the accuracy of detecting the abnormality of the product.

Optionally, determining the auxiliary target area from the first target area and the second target area according to the confidence difference value includes: if the confidence coefficient difference value is greater than or equal to a preset threshold value, determining the first target area as an auxiliary target area; and if the confidence difference is smaller than the preset threshold, determining the second target area as an auxiliary target area. It should be noted that, the preset threshold may be set by a technician according to actual requirements or practical experience.

It can be appreciated that by adopting the above technical scheme, the auxiliary target area is determined according to the magnitude relation between the confidence coefficient difference value and the preset threshold value, if the confidence coefficient difference value is greater than or equal to the preset threshold value, the first target area is determined as the auxiliary target area, and the first target area with the abnormal candidate abnormal detection result is determined as the auxiliary target area, so that the target abnormal detection result of the product to be detected is determined according to the candidate abnormal detection result matched with the auxiliary target category, and the accuracy of detecting the abnormality of the product is improved.

S205, determining a target abnormality detection result of the product to be detected according to the candidate abnormality detection result matched with the auxiliary target category.

Specifically, if the candidate abnormal detection result of the auxiliary abnormal category matching is normal, the candidate abnormal detection result is used as an abnormal detection result of the product to be detected; if the candidate abnormality detection result of the auxiliary abnormality category matching is abnormal, determining an abnormality detection result of the product to be detected according to the area position of the auxiliary abnormality area.

In an alternative embodiment, determining the abnormal detection result of the product to be detected according to the area position of the auxiliary abnormal area includes: checking whether the region position of the auxiliary abnormal region is in a preset region; if the area position of the auxiliary abnormal area is in the preset area, determining that the abnormal detection result of the product to be detected is normal; if the region position of the auxiliary abnormal region is not in the preset region, determining that the abnormal detection result of the product to be detected is abnormal. It should be noted that the preset area may be set by a technician according to actual requirements or practical experience. In one embodiment, the predetermined area is a blank flow channel area of the product.

In a specific embodiment, the prior art is adopted to perform anomaly detection on the hundred thousand electric core appearance images, the over-killing Rate of the anomaly detection exceeds 6%, and the anomaly detection method of the product of the embodiment of the invention is adopted to perform anomaly detection on the hundred thousand electric core appearance images, wherein the over-killing Rate (over-kill Rate) of the anomaly detection is 0.0095%; in another embodiment, the prior art is adopted to perform anomaly detection on the appearance image of the hundred thousand welding products, the over-killing rate of the anomaly detection exceeds 8%, and the anomaly detection method of the product in the embodiment of the invention is adopted to perform anomaly detection on the appearance image of the hundred thousand welding products, and the over-killing rate of the anomaly detection is 0. Therefore, the over-killing rate of the product in the technical scheme of the embodiment of the invention approaches to 0, so that the over-killing rate of the abnormality detection can be obviously reduced, and the accuracy of the abnormality detection is improved.

According to the embodiment of the invention, an auxiliary target area is determined according to the target category of each target area and the candidate abnormal detection result matched with each target category, and the target category of the auxiliary target area is taken as an auxiliary target category; and determining a target abnormality detection result of the product to be detected according to the candidate abnormality detection result of the auxiliary target category matching. According to the technical scheme, the auxiliary target area is determined from each target area according to the target category of each target area and the candidate abnormality detection result matched with each target category, the target category of the auxiliary target area is used as the auxiliary target category, and the target abnormality detection result of the product to be detected is determined according to the candidate abnormality detection result matched with the auxiliary target category, so that the accuracy of abnormality detection is improved.

Example III

Fig. 3 is a schematic structural diagram of an abnormality detection device for a product according to a third embodiment of the present invention. The embodiment of the invention is applicable to the condition of detecting the abnormality of the product, the device can execute the abnormality detection method of the product, the abnormality detection device of the product can be realized in the form of hardware and/or software, and the device can be configured in electronic equipment, such as a server.

Referring to the abnormality detection apparatus of the product shown in fig. 3, it includes an image acquisition module 301, a target category determination module 302, a candidate result determination module 303, and a target result determination module 304, wherein,

an image acquisition module 301, configured to acquire an appearance image of a product to be detected;

the target class determining module 302 is configured to perform target detection on the appearance image by using a target detection model, so as to obtain at least one target area of the appearance image and a target class of each target area;

a candidate result determining module 303, configured to query, for each target area, a candidate abnormal detection result that matches a target category of the target area;

the target result determining module 304 is configured to determine a target abnormality detection result of the product to be detected according to each target area, a target class of each target area, and a candidate abnormality detection result matched with each target class.

According to the embodiment of the invention, the appearance image of the product to be detected is obtained through the image obtaining module; the target category determining module is used for carrying out target detection on the appearance image by adopting a target detection model to obtain at least one target area of the appearance image and target categories of the target areas; querying candidate abnormal detection results matched with the target category of the target area aiming at each target area through a candidate result determining module; and determining a target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category by a target result determination module. Compared with the technical scheme that the detection result with the highest confidence is used as the final detection result of the product in the prior art, the technical scheme provided by the embodiment of the invention can determine the appearance of the product to be detected, a plurality of target areas and the target categories of the target areas, comprehensively determine the target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category, and improve the abnormality detection accuracy.

Optionally, the target result determining module 304 includes:

an auxiliary area determining unit, configured to determine an auxiliary target area according to the target category of each target area and the candidate anomaly detection result matched with each target category, and take the target category of the auxiliary target area as an auxiliary target category;

and the target result determining unit is used for determining a target abnormality detection result of the product to be detected according to the candidate abnormality detection result matched with the auxiliary target category.

Optionally, the auxiliary area determining unit includes:

a first region determining subunit, configured to take the target region as an auxiliary target region if the number of target regions is one;

and the second region determining subunit is used for determining the auxiliary target region according to the confidence coefficient of the target category of each target region and the candidate abnormal detection result matched with each target category if the number of the target regions is at least two.

Optionally, the second area determining subunit is specifically configured to:

taking the target area with the lowest confidence coefficient of the target category as a first target area in the target areas with the abnormal candidate abnormal detection results of the target category matching, and taking the target area with the highest confidence coefficient of the target category as a second target area in the target areas with the normal candidate abnormal detection results of the target category matching;

Determining a confidence difference between the confidence of the target class of the first target region and the confidence of the target class of the second target region;

and determining an auxiliary target area from the first target area and the second target area according to the confidence difference.

Optionally, the second area determining subunit is specifically configured to:

if the confidence coefficient difference value is greater than or equal to a preset threshold value, determining the first target area as an auxiliary target area;

and if the confidence difference is smaller than the preset threshold, determining the second target area as an auxiliary target area.

Optionally, the abnormality detection device for a product includes:

the marking data acquisition module is used for acquiring at least one sample abnormal image and marking data of the sample abnormal image; the labeling data comprises a labeling target area and a labeling target category of the labeling target area;

the image rotating module is used for rotating the sample abnormal image to obtain a rotating sample image, and taking the labeling data of the sample abnormal image as the labeling data of the rotating sample image;

the model training module is used for taking the sample abnormal image and the rotation sample image as candidate abnormal images, and training a preset deep learning model according to each candidate abnormal image and the labeling data of each candidate abnormal image to obtain a target detection model.

Optionally, the abnormality detection device for a product includes:

the detection data acquisition module is used for carrying out anomaly detection on each candidate abnormal image by adopting a target detection model, determining detection data of each candidate abnormal image and checking whether the candidate abnormal image with different detection data and labeling data exists or not;

the error image determining module is used for taking the candidate abnormal image with the same detection data as the labeling data as a correct candidate image and taking the candidate abnormal image with the different detection data as an error candidate image if the verification result is that the candidate abnormal image with the different detection data as the labeling data exists;

the information generation module is used for generating prompt information of the error candidate image so as to prompt a technician to correct the labeling data of the error candidate image and input the corrected labeling data of the error candidate image;

the model updating module is used for receiving correction marking data of the error abnormal image, updating the candidate abnormal image into a correct candidate image and an error candidate image, training a preset deep learning model according to the marking data of the candidate abnormal image and the candidate abnormal image to obtain a corrected abnormal detection model, and updating the target detection model into the corrected abnormal detection model;

And the return module is used for returning to execute the steps of adopting the target detection model to perform anomaly detection on each candidate abnormal image, determining detection data of each candidate abnormal image and checking whether the candidate abnormal image with different detection data and different labeling data exists or not until the candidate abnormal image with different detection data and different labeling data does not exist.

The abnormality detection device for the product provided by the embodiment of the invention can execute the abnormality detection method for the product provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of executing the abnormality detection method for the product.

Example IV

Fig. 4 shows a schematic diagram of an electronic device 400 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.

As shown in fig. 4, the electronic device 400 includes at least one processor 401, and a memory communicatively connected to the at least one processor 401, such as a Read Only Memory (ROM) 402, a Random Access Memory (RAM) 403, etc., in which the memory stores a computer program executable by the at least one processor, and the processor 401 may perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM) 402 or the computer program loaded from the storage unit 408 into the Random Access Memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the electronic device 400 may also be stored. The processor 401, the ROM 402, and the RAM 403 are connected to each other by a bus 404. An input/output (I/O) interface 405 is also connected to bus 404.

Various components in electronic device 400 are connected to I/O interface 405, including: an input unit 406 such as a keyboard, a mouse, etc.; an output unit 407 such as various types of displays, speakers, and the like; a storage unit 408, such as a magnetic disk, optical disk, etc.; and a communication unit 409 such as a network card, modem, wireless communication transceiver, etc. The communication unit 409 allows the electronic device 400 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.

Processor 401 may be a variety of general purpose and/or special purpose processing components with processing and computing capabilities. Some examples of processor 401 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running deep learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 401 performs the various methods and processes described above, such as an anomaly detection method for a product.

In some embodiments, the anomaly detection method of the product may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 408. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 400 via the ROM 402 and/or the communication unit 409. When the computer program is loaded into RAM 403 and executed by processor 401, one or more steps of the anomaly detection method for the article of manufacture described above may be performed. Alternatively, in other embodiments, processor 401 may be configured to perform the anomaly detection method of the product in any other suitable manner (e.g., by means of firmware).

Various implementations of the systems and techniques described here above can be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), complex Programmable Logic Devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.

A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of an anomaly detection apparatus of a general purpose computer, special purpose computer, or other programmable article of manufacture such that the computer programs, when executed by the processor, cause the functions/operations specified in the flowchart and/or block diagram to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.

The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.

The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical host and VPS (Virtual Private Server ) service are overcome.

It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.

The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.

Claims (10)

1. A method for detecting anomalies in a product, the method comprising:

obtaining an appearance image of a product to be detected;

performing target detection on the appearance image by adopting a target detection model to obtain at least one target area of the appearance image and a target category of each target area;

querying a candidate abnormal detection result matched with a target category of each target area aiming at each target area;

And determining a target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category.

2. The method according to claim 1, wherein the determining the target abnormality detection result of the product to be detected based on each of the target areas, the target category of each of the target areas, and the candidate abnormality detection result of each of the target categories matching includes:

determining an auxiliary target area according to the target category of each target area and the candidate abnormal detection result matched with each target category, and taking the target category of the auxiliary target area as an auxiliary target category;

and determining the target abnormality detection result of the product to be detected according to the candidate abnormality detection result of the auxiliary target category matching.

3. The method of claim 2, wherein the determining the auxiliary target area based on the target class of each of the target areas and the candidate anomaly detection result of the target class match comprises:

if the number of the target areas is one, the target areas are used as auxiliary target areas;

If the number of the target areas is at least two, determining an auxiliary target area according to the confidence coefficient of the target category of each target area and the candidate abnormal detection result of the matching of each target category.

4. The method of claim 3, wherein the determining the auxiliary target region based on the confidence of the target class of each of the target regions and the candidate anomaly detection result of each of the target class matches comprises:

taking the target area with the lowest confidence coefficient of the target category as a first target area in the target areas with the abnormal candidate abnormal detection results of the target category matching, and taking the target area with the highest confidence coefficient of the target category as a second target area in the target areas with the normal candidate abnormal detection results of the target category matching;

determining a confidence difference between the confidence of the target class of the first target region and the confidence of the target class of the second target region;

and determining an auxiliary target area from the first target area and the second target area according to the confidence difference value.

5. The method of claim 4, wherein determining an auxiliary target region from the first target region and the second target region based on the confidence difference value comprises:

If the confidence coefficient difference value is larger than or equal to a preset threshold value, determining the first target area as an auxiliary target area;

and if the confidence coefficient difference value is smaller than a preset threshold value, determining the second target area as an auxiliary target area.

6. The method of claim 1, further comprising, prior to acquiring the appearance image of the product to be inspected:

acquiring at least one sample abnormal image and labeling data of the sample abnormal image; the labeling data comprises a labeling target area and a labeling target category of the labeling target area;

rotating the sample abnormal image to obtain a rotating sample image, and taking the labeling data of the sample abnormal image as the labeling data of the rotating sample image;

and taking the sample abnormal image and the rotation sample image as candidate abnormal images, and training a preset deep learning model according to the labeling data of each candidate abnormal image and each candidate abnormal image to obtain a target detection model.

7. The method of claim 6, further comprising, after obtaining the target detection model:

performing anomaly detection on each candidate anomaly image by adopting the target detection model, determining detection data of each candidate anomaly image, and checking whether the candidate anomaly images with different detection data and labeling data exist or not;

If the checking result is that the candidate abnormal images with different detection data and labeling data exist, the candidate abnormal images with the same detection data and labeling data are used as correct candidate images, and the candidate abnormal images with different detection data and labeling data are used as error candidate images;

generating prompt information of the error candidate image so as to prompt a technician to correct the labeling data of the error candidate image and input the corrected labeling data of the error candidate image;

receiving correction labeling data of an error abnormal image, updating the candidate abnormal image into a correct candidate image and an error candidate image, training a preset deep learning model according to the labeling data of the candidate abnormal image and the candidate abnormal image to obtain a corrected abnormal detection model, and updating a target detection model into the corrected abnormal detection model;

and returning to execute the step of adopting the target detection model to perform anomaly detection on each candidate abnormal image, determining detection data of each candidate abnormal image, and checking whether the candidate abnormal image with different detection data and different labeling data exists or not until the candidate abnormal image with different detection data and different labeling data does not exist.

8. An abnormality detection device for a product, the device comprising:

the image acquisition module is used for acquiring an appearance image of the product to be detected;

the target category determining module is used for carrying out target detection on the appearance image by adopting a target detection model to obtain at least one target area of the appearance image and a target category of each target area;

the candidate result determining module is used for querying candidate abnormal detection results matched with the target category of each target area aiming at each target area;

the target result determining module is used for determining the target abnormality detection result of the product to be detected according to each target area, the target category of each target area and the candidate abnormality detection result matched with each target category.

9. An electronic device, the electronic device comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the anomaly detection method of the product of any one of claims 1-7.

10. A computer readable storage medium storing computer instructions for causing a processor to perform the anomaly detection method for the product of any one of claims 1-7.