CN116342585A - Product defect detection method, device, equipment and storage medium - Google Patents
Product defect detection method, device, equipment and storage medium Download PDFInfo
- Publication number
- CN116342585A CN116342585A CN202310551901.9A CN202310551901A CN116342585A CN 116342585 A CN116342585 A CN 116342585A CN 202310551901 A CN202310551901 A CN 202310551901A CN 116342585 A CN116342585 A CN 116342585A
- Authority
- CN
- China
- Prior art keywords
- image
- processed
- target product
- product
- defect detection
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000007547 defect Effects 0.000 title claims abstract description 88
- 238000001514 detection method Methods 0.000 title claims abstract description 67
- 238000012937 correction Methods 0.000 claims abstract description 37
- PXFBZOLANLWPMH-UHFFFAOYSA-N 16-Epiaffinine Natural products C1C(C2=CC=CC=C2N2)=C2C(=O)CC2C(=CC)CN(C)C1C2CO PXFBZOLANLWPMH-UHFFFAOYSA-N 0.000 claims description 39
- 230000009466 transformation Effects 0.000 claims description 28
- 238000000034 method Methods 0.000 claims description 27
- 238000004590 computer program Methods 0.000 claims description 16
- 238000012545 processing Methods 0.000 claims description 12
- 238000000605 extraction Methods 0.000 claims description 11
- 238000003702 image correction Methods 0.000 claims description 6
- 230000002950 deficient Effects 0.000 claims description 5
- 239000004973 liquid crystal related substance Substances 0.000 claims description 5
- 230000011218 segmentation Effects 0.000 claims description 4
- 230000001131 transforming effect Effects 0.000 claims 2
- 238000004891 communication Methods 0.000 description 8
- 238000010586 diagram Methods 0.000 description 8
- 238000013507 mapping Methods 0.000 description 5
- 230000008569 process Effects 0.000 description 4
- 230000003287 optical effect Effects 0.000 description 3
- 230000006870 function Effects 0.000 description 2
- 230000003993 interaction Effects 0.000 description 2
- 238000004519 manufacturing process Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012549 training Methods 0.000 description 2
- 230000002159 abnormal effect Effects 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 230000001413 cellular effect Effects 0.000 description 1
- 230000006835 compression Effects 0.000 description 1
- 238000007906 compression Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000011521 glass Substances 0.000 description 1
- 238000007689 inspection Methods 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 239000013307 optical fiber Substances 0.000 description 1
- 238000005070 sampling Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 230000001953 sensory effect Effects 0.000 description 1
- 238000006467 substitution reaction Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
- G06T7/001—Industrial image inspection using an image reference approach
-
- G06T3/02—
-
- G06T5/80—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The invention discloses a product defect detection method, a device, equipment and a storage medium, comprising the following steps: acquiring a reference image containing a target product and an image to be processed, wherein the reference image comprises the target product in a normal form; according to the image characteristic point pairs determined from the reference image and the image to be processed, carrying out distortion correction on the image to be processed to obtain a corrected image; and comparing the corrected image with the reference image, and detecting defects of the target product according to the comparison result. According to the technical scheme, the problem of product defect detection errors caused by image acquisition distortion is effectively avoided, the detection precision of product defect detection is guaranteed, and the accuracy of product defect detection is improved.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to a method, an apparatus, a device, and a storage medium for detecting product defects.
Background
Line scan cameras are widely used for defect detection of product surfaces on production line platforms. The line scanning camera only scans one line at a time, so the line scanning camera is very suitable for products moving in a pipeline.
At present, in the traditional product defect detection method, a qualified product image is prepared as a template image, and during detection, a system can compare an image obtained in real time on a production line with the template image to detect whether a defect exists. When the line scanning camera images, the speed of the conveyor belt needs to be ensured to be stable, but the conveyor belt adopts the assembly line conveyor belt to convey products, the problem that the speed of the conveyor belt shakes or the products slide often exists, and uneven stretching or compression deformation is further caused when the line scanning camera images. When the deformed image is compared with the template image, the corresponding position is easily shifted and misplaced, so that erroneous judgment is caused, and the defect detection error of the product is caused.
Disclosure of Invention
The invention provides a product defect detection method, device, equipment and storage medium, which effectively avoid the problem of product defect detection errors caused by image acquisition distortion, ensure the detection precision of product defect detection and improve the accuracy of product defect detection.
In a first aspect, an embodiment of the present disclosure provides a method for detecting a product defect, including:
acquiring a reference image containing a target product and an image to be processed, wherein the reference image comprises the target product in a normal form;
according to the image characteristic point pairs determined from the reference image and the image to be processed, carrying out distortion correction on the image to be processed to obtain a corrected image;
and comparing the corrected image with the reference image, and detecting defects of the target product according to the comparison result.
In a second aspect, an embodiment of the present disclosure provides a product defect detection apparatus, including:
the image acquisition module is used for acquiring a reference image containing a target product and an image to be processed, wherein the reference image comprises the target product in a normal form;
the image correction module is used for carrying out distortion correction on the image to be processed according to the image characteristic point pairs determined from the reference image and the image to be processed to obtain a corrected image;
and the defect detection module is used for comparing the corrected image with the reference image and detecting the defect of the target product according to the comparison result.
In a third aspect, an embodiment of the present disclosure provides an electronic device, including:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform the product defect detection method provided by the embodiment of the first aspect described above.
In a fourth aspect, embodiments of the present disclosure provide a computer readable storage medium storing computer instructions for causing a processor to execute the product defect detection method provided in the first aspect.
According to the product defect detection method, device, equipment and storage medium, a reference image containing a target product and an image to be processed are obtained, wherein the reference image comprises the target product in a normal form; according to the image characteristic point pairs determined from the reference image and the image to be processed, carrying out distortion correction on the image to be processed to obtain a corrected image; and comparing the corrected image with the reference image, and detecting defects of the target product according to the comparison result. According to the technical scheme, the image to be processed of the target product is subjected to distortion correction, the correction image is compared with the reference image to detect the defect of the target product, the problem of product defect detection errors caused by image acquisition distortion is effectively avoided, the detection precision of product defect detection is guaranteed, and the accuracy of product defect detection is improved.
It should be understood that the description in this section is not intended to identify key or critical features of the embodiments of the invention or to delineate the scope of the invention. Other features of the present invention will become apparent from the description that follows.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings required for the description of the embodiments will be briefly described below, and it is apparent that the drawings in the following description are only some embodiments of the present invention, and other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flowchart of a method for detecting product defects according to an embodiment of the present invention;
fig. 2 is an exemplary display diagram of image feature point pairs involved in a product defect detection method according to a first embodiment of the present invention;
FIG. 3 is a flowchart of a method for detecting defects of a product according to a second embodiment of the present invention;
fig. 4 is an exemplary illustration of a reference sectional view involved in a product defect detection method according to a second embodiment of the present invention;
fig. 5 is an exemplary illustration of a sectional view to be processed involved in a product defect detection method according to a second embodiment of the present invention;
fig. 6 is a schematic structural diagram of a product defect detecting device according to a third embodiment of the present invention;
fig. 7 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present invention.
Detailed Description
In order that those skilled in the art will better understand the present invention, a technical solution in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in which it is apparent that the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the present invention without making any inventive effort, shall fall within the scope of the present invention.
It should be noted that the terms "first," "reference," and "object" in the description of the present invention and the claims and the above drawings are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the data so used may be interchanged where appropriate such that the embodiments of the invention described herein may be implemented in sequences other than those illustrated or otherwise described herein. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.
Example 1
Fig. 1 is a flowchart of a product defect detection method according to an embodiment of the present invention, where the method may be implemented by a product defect detection device, and the device may be implemented in hardware and/or software.
As shown in fig. 1, the method includes:
s101, acquiring a reference image containing a target product and an image to be processed, wherein the reference image comprises the target product in a normal form.
In this embodiment, the target product may be understood as a part product that is transported through the pipeline, scanned by the line scan camera. The reference image can be understood as an undistorted image acquired when the target product is a qualified product, and the reference image comprises the target product in a normal form. The image to be processed can be understood as an image of a target product acquired by the line scanning camera in real time, and the target product in the image to be processed can be in a normal form or distorted, and the specific product form is determined according to the actually acquired image, so that the embodiment is not limited.
Specifically, a reference image which is transmitted by a line scanning camera and contains a target product in a normal form and an image to be processed which contains the target product are obtained. The line scanning industrial camera is a special industrial detection camera with the advantages of high speed and high precision, can collect pictures of objects to be detected continuously in real time, and transmits collected images to electronic equipment through an image collecting card to perform image processing and product defect detection.
S102, carrying out distortion correction on the image to be processed according to the image characteristic point pairs determined from the reference image and the image to be processed, and obtaining a corrected image.
In this embodiment, fig. 2 is an exemplary illustration of an image feature point pair involved in a product defect detection method according to the first embodiment of the present invention, and as shown in fig. 2, the image feature point pair may be understood as a feature point pair formed by a feature point of a target product included in a reference image and a feature point of a target product included in an image to be processed, and exemplary, feature point 1 in the reference image and feature point 1 in the image to be processed are one image feature point pair. The feature points can be understood as feature positions of corners, end points, edges and the like of the target product. The corrected image may be understood as a normal-shaped product image formed by subjecting an image to be processed to distortion correction.
Specifically, feature points of a target product are determined from a reference image, feature points of the target product are determined from an image to be processed, and feature points obtained from the reference image and feature points obtained from the image to be processed are combined into feature point pairs according to a position corresponding relation. And correcting distortion of the image to be processed according to the image characteristic point pairs determined from the reference image and the image to be processed, and correcting the image to be processed, which is deformed in the image acquisition process, into a product image (corrected image) in a normal form.
S103, comparing the corrected image with the reference image, and detecting defects of the target product according to the comparison result.
In this embodiment, the comparison result is a result of comparing the correction image with the reference image, including two results of matching and not matching the two images, and it can be understood that if the target product does not exist in an abnormal condition, the correction image and the reference image should be completely matched.
Specifically, the corrected image in the normal form is compared with the reference image, whether the corrected image can be completely matched is determined, and the defect detection is performed on the target product according to the comparison result of the matching degree of the corrected image and the reference image, so that whether the target product has defects is determined.
In this embodiment, a reference image including a target product in a normal form is obtained by acquiring a reference image including the target product and an image to be processed; according to the image characteristic point pairs determined from the reference image and the image to be processed, carrying out distortion correction on the image to be processed to obtain a corrected image; and comparing the corrected image with the reference image, and detecting defects of the target product according to the comparison result. According to the technical scheme, the image to be processed of the target product is subjected to distortion correction, the correction image is compared with the reference image to detect the defect of the target product, the problem of product defect detection errors caused by image acquisition distortion is effectively avoided, the detection precision of product defect detection is guaranteed, and the accuracy of product defect detection is improved.
As a first alternative embodiment of the embodiments, on the basis of the above embodiments, the first alternative embodiment further optimizes and increases:
and determining position information of the incompletely matched features, and determining the defect position in the target product according to the position information.
In this embodiment, after comparing the corrected image with the reference image, and detecting the defect of the target product according to the comparison result, if the detection result is that the features are not completely matched, it may be determined that the image to be processed of the target product is not completely matched with the reference image, which indicates that the defect exists in the target product. When the defect of the target product is determined, determining the position information of the characteristic points of the target product which are not completely matched in the correction image and the reference image, and determining the defect position in the target product and marking according to the position information so as to facilitate the inspection or correction of workers.
Example two
Fig. 3 is a flowchart of a product defect detection method according to a second embodiment of the present invention, where any of the foregoing embodiments is further optimized, and the method may be performed by a product defect detection device, and the device may be implemented in hardware and/or software.
As shown in fig. 3, the method includes:
s201, acquiring a reference image containing a target product and an image to be processed, wherein the reference image comprises the target product in a normal form.
S202, triangulating the reference image and the image to be processed respectively according to the image feature point pairs to obtain corresponding reference sectional images and corresponding sectional images to be processed.
In this embodiment, fig. 4 is an exemplary illustration of a reference split map related to a product defect detection method according to the second embodiment of the present invention, and as shown in fig. 4, the reference split map may be understood as an image generated by triangulating a reference image according to feature points of a target product. Fig. 5 is an exemplary illustration of a to-be-processed split map related to a product defect detection method according to a second embodiment of the present invention, where, as shown in fig. 5, the to-be-processed split map may be understood as an image generated after triangulating an image to be processed according to feature points of a target product.
Specifically, triangulation is performed on the reference image and the image to be processed according to the image feature point pairs of the target product in the reference image and the image to be processed, and a corresponding triangulation set is generated, wherein each triangulation region after triangulation is the smallest triangle region formed by combining three feature points in the image. And determining a reference split map of the reference image shown in fig. 4 and a to-be-processed split map of the to-be-processed image shown in fig. 5 according to a triangle set generated by triangulating the reference image and the to-be-processed image.
S203, carrying out affine transformation on the image to be processed according to the reference split map and the split map to be processed, and determining an image formed after affine transformation as a correction image.
In this embodiment, affine transformation is performed on each triangulation region in the image to be processed one by one according to the reference subdivision map shown in fig. 4 and each triangulation region in the image to be processed shown in fig. 5, so as to form an image with normal morphology of the target product after transformation, and the image is determined as a correction image.
S204, comparing the characteristics of the target product identified from the corrected image with those of the target product identified from the reference image.
In this embodiment, the feature points of the target product after correction and the feature information thereof are identified from the correction image, the feature points of the target product identified from the correction image are compared with the feature points of the target product identified from the reference image, and the comparison result is determined. The comparison result comprises complete characteristic matching and incomplete characteristic matching.
S205, if the comparison result is that the features are completely matched, determining that the target product in the corrected image is a normal product.
In this embodiment, if the comparison result is that the corrected image of the normal form formed after the distortion correction processing is performed on the image to be processed is completely matched with the feature of the target product in the reference image, it may be determined that the target product in the corrected image is a normal product, and further it is determined that the physical product of the target product is a normal product.
S206, if the comparison result is that the characteristics are not completely matched, determining that the target product in the corrected image is a defective product.
In this embodiment, if the comparison result is that the characteristic of the target product in the reference image is not completely matched with the corrected image in the normal form formed after the distortion correction processing is performed on the image to be processed, it may be determined that the target product in the corrected image is a defective product, and it may be further determined that the physical product of the target product has a defect.
In this embodiment, a reference image including a target product in a normal form is obtained by acquiring a reference image including the target product and an image to be processed; according to the image characteristic point pairs, triangulation is carried out on the reference image and the image to be processed respectively, and a corresponding reference segmentation map and a corresponding segmentation map to be processed are obtained; affine transformation is carried out on the image to be processed according to the reference split map and the split map to be processed, and an image formed after affine transformation is determined to be a correction image; comparing the characteristics of the target product identified from the corrected image with those of the target product identified from the reference image; if the comparison result is that the features are completely matched, determining that the target product in the corrected image is a normal product; and if the comparison result is that the characteristics are not completely matched, determining that the target product in the corrected image is a defective product. According to the technical scheme, the image to be processed is triangulated through the feature points, the correction image of the image to be processed is determined based on the image after the triangulated, and defect detection is carried out on the target product according to the correction image. By adopting the technical scheme, the problem of product defect detection errors caused by image acquisition distortion is effectively avoided, the detection precision of product defect detection is ensured, and the accuracy of product defect detection is improved.
As a first optional embodiment of the embodiments, on the basis of the above embodiments, the first optional embodiment further optimizes and adds step S203, and according to the reference split map and the to-be-processed split map, performs affine transformation on the to-be-processed image, and determines an image formed after the affine transformation as a corrected image, including:
a1 Obtaining a distorted triangular region contained in the to-be-processed sectional view, and determining a reference triangular region of the relative distorted triangular region from the reference sectional view.
In the present embodiment, the distorted triangle area can be understood as a triangle split area in the to-be-processed split map as shown in fig. 5. The reference triangle area may be understood as a triangle split area in the reference split map as shown in fig. 4.
Specifically, a triangular region formed by three characteristic points in a sectional graph to be processed is obtained, and is determined to be a distorted triangular region; and acquiring a triangular region formed by three characteristic points in the reference sectional graph, and determining the triangular region as a reference triangular region. A distorted triangular region and a reference triangular region formed according to the same feature point position (feature point pair) are determined as a triangular pair, for example, a reference triangular region formed by feature points 1, 4 and 5 in a reference sectional view and a distorted triangular region formed by feature points 1, 4 and 5 in a to-be-processed sectional view are determined as a triangular pair.
b1 For each distortion triangle region, carrying out affine transformation to the corresponding vertex in the reference triangle region according to the vertex coordinates of each vertex in the distortion triangle region, and obtaining affine vertex coordinates of each vertex after affine transformation of the distortion triangle region.
In the present embodiment, the distorted triangle region and the reference triangle region determined from the feature point pairs of three vertices of the triangle are taken, and the mapping matrix is calculated based on the vertex coordinates of six feature points of the three feature point pairs
The mapping parameters a, b, c, d, e, f are obtained, and when the mapping parameters are calculated, (x ', y') is the vertex coordinates of the reference triangle area in the reference split map, and (x, y) is the vertex coordinates of the distorted triangle area in the split map to be processed. After the mapping parameters are determined, affine transformation from the distortion triangle area to the corresponding vertexes in the reference triangle area is carried out according to the vertex coordinates of the vertexes in the distortion triangle area and the mapping matrix, and affine vertex coordinates of the vertexes after affine transformation of the distortion triangle area are obtained. In this case, (x ', y') is the transformed affine vertex coordinates, and (x, y) is the vertex coordinates of each vertex in the distorted triangle region. It is understood that the coordinates referred to in this embodiment are all pixel coordinates.
c1 Copying each distorted triangle area on the newly built blank image layer according to each affine vertex coordinate, and determining the image formed after copying as a correction image.
In this embodiment, a blank layer with the same size as the reference image needs to be prepared in advance, and after the determination of the affine vertex coordinates in step b 1) is completed, a triangle area is built on the newly built blank layer according to the three affine vertex coordinates, so as to realize the copy of the distorted triangle area on the newly built blank layer after the completion of affine transformation. After the copying of the distorted triangular areas is completed, traversing the to-be-processed split diagram, determining whether the copying of each distorted triangular area in the to-be-processed split diagram on a newly built blank layer is completed, and if not, returning to the re-executing step b 1) until all the distorted triangular areas are converted; if yes, the image formed after copying is determined to be a corrected image.
In this embodiment, affine transformation is performed on each distorted triangle area, instead of directly performing affine transformation on the whole image to be processed, so that local distortion correction on the linear camera sampling image can be effectively realized.
As a second optional embodiment of the embodiments, on the basis of the above embodiments, the second optional embodiment further optimally adds a step of determining an image feature point pair from the reference image and the image to be processed, including:
a2 Inputting the reference image and the image to be processed into the trained feature point extraction model respectively.
In the present embodiment, the feature point extraction model may be understood as a model for extracting feature points from an input image and outputting feature points corresponding to the image.
Specifically, training a neural network based on deep learning in advance to obtain a complete training feature point extraction model, and respectively inputting a reference image and an image to be processed into the trained feature point extraction model.
b2 Obtaining a reference characteristic point set output by the characteristic point extraction model relative to the reference image and a to-be-processed characteristic point set output relative to the to-be-processed image.
In the present embodiment, when the model input is a reference image, a reference feature point set of the relative reference image output by the feature point extraction model, for example, a reference feature point set {1,2 …,14} of 14 feature points shown in fig. 2 is obtained; when the model input is an image to be processed, a set of feature points to be processed of the image to be processed, such as a set of feature points {1,2 …, 13,14} of 14 feature points shown in fig. 2, output from the feature point extraction model is obtained.
c2 And (3) matching the characteristic points in the reference characteristic point set with the characteristic point set to be processed to obtain matched image characteristic point pairs.
In this embodiment, each feature point in the reference feature point set is paired with each feature point in the feature point set to be processed, and a matched image feature point pair is obtained. Illustratively, as shown in fig. 2, the feature point 1 of the reference image and the feature point 1 of the image to be processed are matched feature point pairs.
Example III
Fig. 6 is a schematic structural diagram of a product defect detecting device according to a third embodiment of the present invention. As shown in fig. 6, the apparatus includes:
an image acquisition module 31, configured to acquire a reference image including a target product and an image to be processed, where the reference image includes the target product in a normal form;
an image correction module 32, configured to perform distortion correction on the image to be processed according to the image feature point pairs determined from the reference image and the image to be processed, so as to obtain a corrected image;
and the defect detection module 33 is configured to compare the corrected image with the reference image, and detect a defect of the target product according to the comparison result.
According to the product defect detection device adopted by the technical scheme, the target product is subjected to defect detection by comparing the corrected image with the reference image through distortion correction of the image to be processed of the target product, so that the problem of product defect detection errors caused by image acquisition distortion is effectively avoided, the detection precision of product defect detection is ensured, and the accuracy of product defect detection is improved.
Optionally, the image correction module 32 includes:
the subdivision processing unit is used for performing triangulation on the reference image and the image to be processed according to the image characteristic point pairs to obtain a corresponding reference subdivision graph and a corresponding subdivision graph to be processed;
and the affine transformation unit is used for carrying out affine transformation on the image to be processed according to the reference split map and the split map to be processed, and determining an image formed after affine transformation as the correction image.
Optionally, the affine transformation unit is specifically configured to:
acquiring a distorted triangular region contained in the to-be-processed sectional graph, and determining a reference triangular region corresponding to the distorted triangular region from the reference sectional graph;
for each distorted triangular region, carrying out affine transformation to the corresponding vertex in the reference triangular region according to the vertex coordinates of each vertex in the distorted triangular region, and obtaining affine vertex coordinates of each vertex after affine transformation of the distorted triangular region;
and copying each distorted triangle area on the newly built blank image layer according to each affine vertex coordinate, and determining the image formed after copying as the correction image.
Further, the image correction module 32 further includes: the characteristic point pair determining unit is used for determining an image characteristic point pair from the reference image and the image to be processed, and the characteristic point pair determining unit is specifically used for:
respectively inputting the reference image and the image to be processed into a trained feature point extraction model;
obtaining a reference feature point set output by the feature point extraction model relative to the reference image and a to-be-processed feature point set output relative to the to-be-processed image;
and matching the characteristic points in the reference characteristic point set with the characteristic point set to be processed to obtain matched image characteristic point pairs.
Optionally, the defect detection module 33 is specifically configured to:
comparing the characteristics of the target product identified from the corrected image with those of the target product identified from the reference image;
if the comparison result is that the characteristics are completely matched, determining that the target product in the corrected image is a normal product;
and if the comparison result is that the characteristics are not completely matched, determining that the target product in the corrected image is a defective product.
Optionally, the device further comprises a defect position determining unit, which is used for determining position information of the incomplete matching of the features, and determining the defect position in the target product according to the position information.
The product defect detection device provided by the embodiment of the invention can execute the product defect detection method provided by any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.
Example IV
Fig. 7 shows a schematic diagram of an electronic device 40 that may be used to implement an embodiment of the invention. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Electronic equipment may also represent various forms of mobile devices, such as personal digital processing, cellular telephones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be exemplary only, and are not meant to limit implementations of the inventions described and/or claimed herein.
As shown in fig. 7, the electronic device 40 includes at least one processor 41, and a memory communicatively connected to the at least one processor 41, such as a Read Only Memory (ROM) 42, a Random Access Memory (RAM) 43, etc., in which the memory stores a computer program executable by the at least one processor, and the processor 41 may perform various suitable actions and processes according to the computer program stored in the Read Only Memory (ROM) 42 or the computer program loaded from the storage unit 48 into the Random Access Memory (RAM) 43. In the RAM 43, various programs and data required for the operation of the electronic device 40 may also be stored. The processor 41, the ROM 42 and the RAM 43 are connected to each other via a bus 44. An input/output (I/O) interface 45 is also connected to bus 44.
Various components in electronic device 40 are connected to I/O interface 45, including: an input unit 46 such as a keyboard, a mouse, etc.; an output unit 47 such as various types of displays, speakers, and the like; a storage unit 48 such as a magnetic disk, an optical disk, or the like; and a communication unit 49 such as a network card, modem, wireless communication transceiver, etc. The communication unit 49 allows the electronic device 40 to exchange information/data with other devices via a computer network, such as the internet, and/or various telecommunication networks.
The processor 41 may be various general and/or special purpose processing components with processing and computing capabilities. Some examples of processor 41 include, but are not limited to, a Central Processing Unit (CPU), a Graphics Processing Unit (GPU), various specialized Artificial Intelligence (AI) computing chips, various processors running machine learning model algorithms, digital Signal Processors (DSPs), and any suitable processor, controller, microcontroller, etc. The processor 41 performs the various methods and processes described above, such as the product defect detection method.
In some embodiments, the product defect detection method may be implemented as a computer program tangibly embodied on a computer-readable storage medium, such as the storage unit 48. In some embodiments, part or all of the computer program may be loaded and/or installed onto the electronic device 40 via the ROM 42 and/or the communication unit 49. When the computer program is loaded into RAM 43 and executed by processor 41, one or more steps of the product defect detection method described above may be performed. Alternatively, in other embodiments, processor 41 may be configured to perform the product defect detection method by any other suitable means (e.g., by means of firmware).
Various implementations of the systems and techniques described here above may be implemented in digital electronic circuitry, integrated circuit systems, field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems On Chip (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include: implemented in one or more computer programs, the one or more computer programs may be executed and/or interpreted on a programmable system including at least one programmable processor, which may be a special purpose or general-purpose programmable processor, that may receive data and instructions from, and transmit data and instructions to, a storage system, at least one input device, and at least one output device.
A computer program for carrying out methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer programs, when executed by the processor, cause the functions/acts specified in the flowchart and/or block diagram block or blocks to be implemented. The computer program may execute entirely on the machine, partly on the machine, as a stand-alone software package, partly on the machine and partly on a remote machine or entirely on the remote machine or server.
In the context of the present invention, a computer-readable storage medium may be a tangible medium that can contain, or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer readable storage medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, the computer readable storage medium may be a machine readable signal medium. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.
To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to a user; and a keyboard and a pointing device (e.g., a mouse or a trackball) through which a user can provide input to the electronic device. Other kinds of devices may also be used to provide for interaction with a user; for example, feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form, including acoustic input, speech input, or tactile input.
The systems and techniques described here can be implemented in a computing system that includes a background component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front-end component (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such background, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local Area Networks (LANs), wide Area Networks (WANs), blockchain networks, and the internet.
The computing system may include clients and servers. The client and server are typically remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also called a cloud computing server or a cloud host, and is a host product in a cloud computing service system, so that the defects of high management difficulty and weak service expansibility in the traditional physical hosts and VPS service are overcome.
It should be appreciated that various forms of the flows shown above may be used to reorder, add, or delete steps. For example, the steps described in the present invention may be performed in parallel, sequentially, or in a different order, so long as the desired results of the technical solution of the present invention are achieved, and the present invention is not limited herein.
The above embodiments do not limit the scope of the present invention. It will be apparent to those skilled in the art that various modifications, combinations, sub-combinations and alternatives are possible, depending on design requirements and other factors. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of the present invention.
Claims (10)
1. A method for detecting product defects, comprising:
acquiring a reference image containing a target product and an image to be processed, wherein the reference image comprises the target product in a normal form;
according to the image characteristic point pairs determined from the reference image and the image to be processed, carrying out distortion correction on the image to be processed to obtain a corrected image;
and comparing the corrected image with the reference image, and detecting defects of the target product according to the comparison result.
2. The method according to claim 1, wherein the performing distortion correction on the image to be processed according to the image feature point pair determined from the reference image and the image to be processed to obtain a corrected image includes:
according to the image characteristic point pairs, triangulation is carried out on the reference image and the image to be processed respectively, and a corresponding reference segmentation map and a corresponding segmentation map to be processed are obtained;
and carrying out affine transformation on the image to be processed according to the reference sectional graph and the sectional graph to be processed, and determining an image formed after affine transformation as the correction image.
3. The method according to claim 2, wherein the affine transforming the image to be processed based on the reference split map and the split map to be processed, determining an image formed after affine transforming as the corrected image, comprises:
acquiring a distorted triangular region contained in the to-be-processed sectional graph, and determining a reference triangular region corresponding to the distorted triangular region from the reference sectional graph;
for each distorted triangular region, carrying out affine transformation to the corresponding vertex in the reference triangular region according to the vertex coordinates of each vertex in the distorted triangular region, and obtaining affine vertex coordinates of each vertex after affine transformation of the distorted triangular region;
and copying each distorted triangle area on the newly built blank image layer according to each affine vertex coordinate, and determining the image formed after copying as the correction image.
4. A method according to any one of claims 1-3, wherein the step of determining pairs of image feature points from the reference image and the image to be processed comprises:
respectively inputting the reference image and the image to be processed into a trained feature point extraction model;
obtaining a reference feature point set output by the feature point extraction model relative to the reference image and a to-be-processed feature point set output relative to the to-be-processed image;
and matching the characteristic points in the reference characteristic point set with the characteristic point set to be processed to obtain matched image characteristic point pairs.
5. The method according to claim 1, wherein comparing the corrected image with the reference image, and performing defect detection on the target product according to the comparison result, comprises:
comparing the characteristics of the target product identified from the corrected image with those of the target product identified from the reference image;
if the comparison result is that the characteristics are completely matched, determining that the target product in the corrected image is a normal product;
and if the comparison result is that the characteristics are not completely matched, determining that the target product in the corrected image is a defective product.
6. The method as recited in claim 1, further comprising: and determining position information with incompletely matched characteristics, and determining the defect position in the target product according to the position information.
7. A product defect detection apparatus, comprising:
the image acquisition module is used for acquiring a reference image containing a target product and an image to be processed, wherein the reference image comprises the target product in a normal form;
the image correction module is used for carrying out distortion correction on the image to be processed according to the image characteristic point pairs determined from the reference image and the image to be processed to obtain a corrected image;
and the defect detection module is used for comparing the corrected image with the reference image and detecting the defect of the target product according to the comparison result.
8. The apparatus of claim 7, wherein the image correction module comprises:
the subdivision processing unit is used for performing triangulation on the reference image and the image to be processed according to the image characteristic point pairs to obtain a corresponding reference subdivision graph and a corresponding subdivision graph to be processed;
and the affine transformation unit is used for carrying out affine transformation on the image to be processed according to the reference split map and the split map to be processed, and determining an image formed after affine transformation as the correction image.
9. An electronic device, comprising:
at least one processor; and
a memory communicatively coupled to the at least one processor; wherein, the liquid crystal display device comprises a liquid crystal display device,
the memory stores a computer program executable by the at least one processor to enable the at least one processor to perform a product defect detection method according to any one of claims 1-6.
10. A computer readable storage medium, characterized in that the computer readable storage medium stores computer instructions for causing a processor to implement the product defect detection method of any of claims 1-6 when executed.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310551901.9A CN116342585A (en) | 2023-05-16 | 2023-05-16 | Product defect detection method, device, equipment and storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310551901.9A CN116342585A (en) | 2023-05-16 | 2023-05-16 | Product defect detection method, device, equipment and storage medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN116342585A true CN116342585A (en) | 2023-06-27 |
Family
ID=86893213
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310551901.9A Pending CN116342585A (en) | 2023-05-16 | 2023-05-16 | Product defect detection method, device, equipment and storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116342585A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117119115A (en) * | 2023-10-23 | 2023-11-24 | 杭州百子尖科技股份有限公司 | Calibration method and device based on machine vision, electronic equipment and storage medium |
-
2023
- 2023-05-16 CN CN202310551901.9A patent/CN116342585A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117119115A (en) * | 2023-10-23 | 2023-11-24 | 杭州百子尖科技股份有限公司 | Calibration method and device based on machine vision, electronic equipment and storage medium |
| CN117119115B (en) * | 2023-10-23 | 2024-02-06 | 杭州百子尖科技股份有限公司 | Calibration method and device based on machine vision, electronic equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11120254B2 (en) | Methods and apparatuses for determining hand three-dimensional data | |
| CN111833303A (en) | Product detection method and device, electronic equipment and storage medium | |
| WO2019001164A1 (en) | Optical filter concentricity measurement method and terminal device | |
| US10074551B2 (en) | Position detection apparatus, position detection method, information processing program, and storage medium | |
| WO2024002187A1 (en) | Defect detection method, defect detection device, and storage medium | |
| CN116342585A (en) | Product defect detection method, device, equipment and storage medium | |
| CN113362314A (en) | Medical image recognition method, recognition model training method and device | |
| CN112652020A (en) | Visual SLAM method based on AdaLAM algorithm | |
| CN115471476A (en) | Method, device, equipment and medium for detecting component defects | |
| CN116559177A (en) | Defect detection method, device, equipment and storage medium | |
| CN115311469A (en) | Image labeling method, training method, image processing method and electronic equipment | |
| CN115457152A (en) | External parameter calibration method and device, electronic equipment and storage medium | |
| CN113705564B (en) | Pointer type instrument identification reading method | |
| CN115311624B (en) | Slope displacement monitoring method and device, electronic equipment and storage medium | |
| CN114734444B (en) | Target positioning method and device, electronic equipment and storage medium | |
| CN115546143A (en) | Method and device for positioning center point of wafer, storage medium and electronic equipment | |
| CN110874837B (en) | Defect automatic detection method based on local feature distribution | |
| CN116258714B (en) | Defect identification method and device, electronic equipment and storage medium | |
| CN114581890B (en) | Method and device for determining lane line, electronic equipment and storage medium | |
| CN116958145B (en) | Image processing method and device, visual detection system and electronic equipment | |
| CN117689660B (en) | Vacuum cup temperature quality inspection method based on machine vision | |
| CN116883488B (en) | Method, device, equipment and medium for determining center position of circular pipe | |
| CN116182807B (en) | Gesture information determining method, device, electronic equipment, system and medium | |
| CN117739993B (en) | Robot positioning method and device, robot and storage medium | |
| Jinzhuo et al. | Research on Two-step Pointer Meter Recognition Method Based on Yolov7 |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |