CN116977250A - Defect detection method and device for industrial parts and computer equipment - Google Patents

Defect detection method and device for industrial parts and computer equipment Download PDF

Info

Publication number
CN116977250A
CN116977250A CN202211539122.9A CN202211539122A CN116977250A CN 116977250 A CN116977250 A CN 116977250A CN 202211539122 A CN202211539122 A CN 202211539122A CN 116977250 A CN116977250 A CN 116977250A
Authority
CN
China
Prior art keywords
image
contour
industrial
information
sample
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202211539122.9A
Other languages
Chinese (zh)
Inventor
彭瑾龙
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Tencent Technology Shenzhen Co Ltd
Original Assignee
Tencent Technology Shenzhen Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Tencent Technology Shenzhen Co Ltd filed Critical Tencent Technology Shenzhen Co Ltd
Priority to CN202211539122.9A priority Critical patent/CN116977250A/en
Publication of CN116977250A publication Critical patent/CN116977250A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0004—Industrial image inspection
    • G06T7/0008—Industrial image inspection checking presence/absence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/11—Region-based segmentation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/10—Segmentation; Edge detection
    • G06T7/13—Edge detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G06T7/344—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30108—Industrial image inspection
    • G06T2207/30164—Workpiece; Machine component
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30—Computing systems specially adapted for manufacturing

Abstract

The embodiment of the application discloses a defect detection method, a device and computer equipment for industrial parts; the embodiment of the application can acquire the part image of the industrial part to be detected and the structural feature cluster library of the industrial part sample, wherein the industrial part to be detected comprises at least two part structures; performing region detection processing on a part image of the industrial part to be detected to obtain region position information of a part structure in the part image; based on the regional position information of the part structure, performing image segmentation processing on the part image to obtain a structural image corresponding to the part structure of the industrial part to be detected; and matching the feature information corresponding to the structural image of the part structure with part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected, so that the accuracy of structural defect detection of the industrial part can be improved.

Description

Defect detection method and device for industrial parts and computer equipment
Technical Field
The present application relates to the field of computer technologies, and in particular, to a method and an apparatus for detecting defects of an industrial part, and a computer device.
Background
The defect detection of the industrial parts refers to quality detection of the industrial parts in the production and manufacturing process, and the industrial parts generally need to be subjected to defect detection after being produced, so that the quality of the industrial parts is improved. Defect detection of industrial parts can be classified into appearance detection and structural detection. Common appearance tests may include industrial parts crush, scratch or mar, and the like. Structural inspection is mainly used for judging whether a specific part of an industrial part has structural errors or not. For example, when the industrial part is a screw, structural inspection can determine if the screw has missing threads, incomplete threads, and no screw hole problems. Through practice of the prior art, the inventor of the present application finds that the prior art is used for detecting structural defects of industrial parts, and has the problem of low accuracy.
Disclosure of Invention
The embodiment of the application provides a defect detection method, device and computer equipment for industrial parts, which can improve the accuracy of structural defect detection for the industrial parts.
The embodiment of the application provides a defect detection method for industrial parts, which comprises the following steps:
acquiring a part image of an industrial part to be detected and a structural feature cluster library of an industrial part sample, wherein the industrial part to be detected comprises at least two part structures, and the structural feature cluster library comprises part structural feature clusters corresponding to at least two part structural samples when the industrial part sample is in a normal state;
Performing region detection processing on the part image of the industrial part to be detected to obtain region position information of a part structure in the part image;
based on the regional position information of the part structure, performing image segmentation processing on the part image to obtain a structure image corresponding to the part structure of the industrial part to be detected;
and carrying out matching processing on feature information corresponding to the structural image of the part structure and part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected.
Correspondingly, the embodiment of the application also provides a defect detection device of the industrial part, which comprises the following components:
the device comprises an acquisition unit, a detection unit and a detection unit, wherein the acquisition unit is used for acquiring a part image of an industrial part to be detected and a structural feature cluster library of an industrial part sample, the industrial part to be detected comprises at least two part structures, and the structural feature cluster library comprises part structural feature clusters corresponding to at least two part structural samples when the industrial part sample is in a normal state;
the area detection unit is used for carrying out area detection processing on the part image of the industrial part to be detected to obtain area position information of a part structure in the part image;
The image segmentation unit is used for carrying out image segmentation processing on the part image based on the region position information of the part structure to obtain a structure image corresponding to the part structure of the industrial part to be detected;
and the matching unit is used for carrying out matching processing on the feature information corresponding to the structural image of the part structure and the part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected.
In an embodiment, the area detecting unit may include:
an image acquisition subunit, configured to acquire a template image for the industrial part to be detected;
the contrast calculation subunit is used for carrying out contrast calculation processing on the template image and the part image to obtain a transformation matrix between the template image and the part image;
the registration subunit is used for carrying out registration processing on the part image by utilizing the transformation matrix to obtain a registered part image;
and the mapping subunit is used for mapping the region identification information of the template image to the registered part image to obtain the region position information of the part structure.
In an embodiment, the comparison computation subunit may include:
The first contour error recognition module is used for carrying out contour error recognition on the template image and the part image to obtain first contour error information;
the transformation module is used for transforming the part image by utilizing a preset initial transformation matrix to obtain a transformed part image;
the second contour error recognition module is used for carrying out contour error recognition on the template image and the transformed part image to obtain second contour error information;
the comparison calculation module is used for carrying out comparison calculation processing on the first contour error information and the second contour error information to obtain error comparison information;
and the adjustment module is used for adjusting the preset initial transformation matrix based on the error comparison information to obtain the transformation matrix.
In an embodiment, the first contour error identification module may include:
the contour recognition sub-module is used for carrying out contour recognition on the template image and the part image to obtain a contour point set of the template image and a contour point set of the part image, wherein the contour point set of the template image comprises the position information of at least one template contour point, and the contour point set of the part image comprises the position information of at least one part contour point;
The matching combination sub-module is used for carrying out matching combination processing on the at least one template contour point and the at least one part contour point based on the position information of the template contour point and the position information of the part contour point to obtain at least one contour point matching pair;
the calculation sub-module is used for calculating the contour point error information of each contour point matching pair respectively;
a fusion sub-module, configured to fuse the contour point error information of each contour point matching pair to obtain the first contour error information
In an embodiment, the profile recognition sub-module may include:
the edge detection assembly is used for carrying out edge detection on the template image and the part image to obtain a plurality of template contour images corresponding to the template image and a plurality of part contour images corresponding to the part image;
the probability prediction component is used for carrying out probability prediction processing on the template contour images and the part contour images to obtain probability information of each template contour image and probability information of each part contour image;
a determining component for determining a target template contour image at the plurality of template contour images based on probability information of the template contour image, and determining a target part contour image at the plurality of part contour images based on probability information of the part contour image;
And the positioning component is used for positioning the target template contour image and the target part contour image to obtain a contour point set of the template image and a contour point set of the part image.
In an embodiment, the adjusting module may include:
the comparison sub-module is used for comparing the error comparison information with a preset comparison threshold value;
the screening sub-module is used for screening out a target contour point matching pair from the at least one contour point matching pair according to contour point error information of each contour point matching pair when the error comparison threshold is larger than the preset comparison threshold, wherein the target contour point matching pair comprises a target part contour point and a target template contour point;
the removing submodule is used for removing the target template contour points in the contour point set of the template image to obtain an updated contour point set of the template image, and removing the target part contour points in the contour point set of the part image to obtain an updated contour point set of the part image;
and the adjustment sub-module is used for adjusting the preset initial transformation matrix based on the updated contour point set of the template image and the updated contour point set of the part image to obtain the transformation matrix.
In an embodiment, the adjusting sub-module may include:
the acquisition component is used for acquiring transformation reference contour points in the updated contour point set of the template image and the updated contour point set of the part image respectively;
the parameter estimation component is used for carrying out parameter estimation processing on the preset initial transformation matrix by utilizing the transformation reference contour points to obtain the matrix estimation parameters;
the evaluation component is used for performing evaluation processing on the matrix estimation parameters to obtain an evaluation result of the matrix estimation parameters;
and the replacing component is used for replacing the matrix parameters of the preset initial transformation matrix with the matrix estimation parameters according to the evaluation result to obtain the transformation matrix.
In an embodiment, the matching unit may include:
the statistics subunit is used for carrying out statistics processing on sample feature information in each part structural feature cluster of the industrial part sample to obtain a cluster center corresponding to each part structural feature cluster;
the similarity calculation subunit is used for calculating the similarity of the feature information corresponding to the structural image of the part structure and the clustering center corresponding to each part structural feature cluster to obtain at least one piece of similarity information;
And the judging subunit is used for judging the at least one similarity information by utilizing a preset similarity threshold value to obtain a defect detection result of the industrial part to be detected.
In an embodiment, the determining subunit may include:
the generation module is used for generating and outputting a defect detection result of the industrial part to be detected passing detection when the similarity information in the at least one piece of similarity information accords with the preset similarity threshold value;
the structure determining module is used for determining the defective part structure of the industrial part to be detected when each piece of similarity information does not accord with the preset similarity threshold value;
and the output module is used for generating and outputting the part structure name and the region position information of the defective part structure.
In an embodiment, the defect detecting apparatus may further include:
the sample acquisition unit is used for acquiring sample images of a plurality of industrial part samples, wherein the sample images are images when the industrial part samples are in a normal state, and the industrial part samples comprise at least two part structure samples;
the sample area detection unit is used for carrying out area detection processing on sample images of the plurality of industrial part samples to obtain area position information of part structure samples in each sample image;
The sample image segmentation unit is used for carrying out image segmentation processing on the sample images based on the region position information of the part structure samples aiming at each sample image to obtain a plurality of structure sample images;
and the clustering unit is used for carrying out clustering processing on the structural feature information of the structural sample images of the same part structural sample to obtain at least one part structural feature cluster of the part structural sample.
In an embodiment, the clustering unit may include:
a central screening subunit, configured to screen at least one initial clustering center from structural feature information of a structural sample image of the same part structural sample;
the calculating subunit is used for calculating the structural feature information of the structural sample image and the sample similarity information between each initial clustering center;
and the classifying sub-unit is used for classifying the structural feature information according to the sample similarity information of the structural feature information to obtain at least one part structural feature cluster of the part structural sample.
Embodiments of the present application also provide a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The computer instructions are read from a computer-readable storage medium by a processor of a computer device, and executed by the processor, cause the computer device to perform the methods provided in the various alternatives of the above aspect.
Correspondingly, the embodiment of the application also provides a storage medium, wherein the storage medium stores instructions which are executed by a processor to realize the defect detection method of the industrial part provided by any one of the embodiments of the application.
The embodiment of the application can acquire the part image of the industrial part to be detected and the structural feature cluster library of the industrial part sample, wherein the industrial part to be detected comprises at least two part structures, and the structural feature cluster library comprises part structural feature clusters corresponding to at least two part structural samples when the industrial part sample is in a normal state; performing region detection processing on a part image of the industrial part to be detected to obtain region position information of a part structure in the part image; based on the regional position information of the part structure, performing image segmentation processing on the part image to obtain a structural image corresponding to the part structure of the industrial part to be detected; and matching the feature information corresponding to the structural image of the part structure with part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected, so that the accuracy of structural defect detection of the industrial part can be improved.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings that are needed in the description of the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present application, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a schematic view of a method for detecting defects of an industrial part according to an embodiment of the present application;
FIG. 2 is a flow chart of a method for detecting defects of an industrial part according to an embodiment of the present application;
FIG. 3 is a schematic view of another exemplary embodiment of a method for detecting defects in an industrial part;
FIG. 4 is a schematic view of another exemplary embodiment of a method for detecting defects in an industrial part;
FIG. 5 is a schematic view of another exemplary embodiment of a method for detecting defects in an industrial part;
FIG. 6 is a schematic view of another scenario of a defect detection method for industrial parts according to an embodiment of the present application;
FIG. 7 is a schematic view of another scenario of a defect detection method for industrial parts according to an embodiment of the present application;
FIG. 8 is a schematic view of another exemplary embodiment of a method for detecting defects in an industrial part;
FIG. 9 is a schematic view of another scenario of a defect detection method for industrial parts according to an embodiment of the present application;
FIG. 10 is a schematic flow chart of a method for detecting defects of an industrial part according to an embodiment of the present application;
FIG. 11 is a schematic flow chart of another exemplary method for detecting defects of an industrial part according to an embodiment of the present application;
FIG. 12 is a schematic flow chart of another exemplary method for detecting defects of an industrial part according to an embodiment of the present application;
FIG. 13 is a schematic view of a defect detection apparatus for industrial parts according to an embodiment of the present application;
fig. 14 is a schematic structural diagram of a computer device according to an embodiment of the present application.
Detailed Description
The following description of the embodiments of the present application will be made clearly and fully with reference to the accompanying drawings, in which embodiments of the application are shown, however, in which embodiments are shown, by way of illustration, only, and not in any way all embodiments of the application. All other embodiments, which can be made by those skilled in the art based on the embodiments of the application without making any inventive effort, are intended to fall within the scope of the application.
The embodiment of the application provides a defect detection method of an industrial part, which can be executed by a defect detection device of the industrial part, and the defect detection device of the industrial part can be integrated in computer equipment. The computer device may include at least one of a terminal, a server, and the like. That is, the defect detection method of the industrial part according to the embodiment of the present application may be executed by a terminal, a server, or both a terminal and a server capable of communicating with each other.
The terminals may include, but are not limited to, smart phones, tablet computers, notebook computers, personal computers (Personal Computer, PCs), smart appliances, wearable electronic devices, VR/AR devices, vehicle terminals, smart voice interaction devices, and the like.
The server may be an interworking server or a background server among a plurality of heterogeneous systems, may be an independent physical server, may be a server cluster or a distributed system formed by a plurality of physical servers, and may be a cloud server for providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, basic cloud computing services such as big data and an artificial intelligence platform, and the like.
It should be noted that the embodiments of the present application may be applied to various scenarios, including, but not limited to, cloud technology, artificial intelligence, intelligent transportation, driving assistance, and the like.
In an embodiment, as shown in fig. 1, the device for detecting defects of an industrial part may be integrated on a computer device such as a terminal or a server, so as to implement the method for detecting defects of an industrial part according to the embodiment of the present application. Specifically, the server 11 may acquire, through the terminal 10, a part image of an industrial part to be detected and a structural feature cluster library of industrial part samples, where the industrial part to be detected includes at least two part structures, and the structural feature cluster library includes part structural feature clusters corresponding to at least two part structural samples when the industrial part samples are in a normal state; performing region detection processing on a part image of the industrial part to be detected to obtain region position information of a part structure in the part image; based on the regional position information of the part structure, performing image segmentation processing on the part image to obtain a structural image corresponding to the part structure of the industrial part to be detected; and carrying out matching processing on the feature information corresponding to the structural image of the part structure and the part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected.
The following detailed description is given, respectively, of the embodiments, and the description sequence of the following embodiments is not to be taken as a limitation of the preferred sequence of the embodiments.
The embodiment of the application will be described from the perspective of a defect detection device for an industrial part, which may be integrated in a computer device, and the computer device may be a server, a terminal, or other devices.
As shown in fig. 2, a method for detecting defects of an industrial part is provided, and the specific process includes:
101. obtaining a part image of an industrial part to be detected and a structural feature cluster library of an industrial part sample, wherein the industrial part to be detected comprises at least two part structures, and the structural feature cluster library comprises part structural feature clusters corresponding to at least two part structural samples when the industrial part sample is in a normal state.
The industrial part to be inspected may refer to an industrial part for which defect inspection is required. For example, the industrial parts to be inspected may be screws, diodes and chips, etc. which require defect inspection.
The part image may include, among other things, an image that captures the appearance of the industrial part. For example, as shown in FIG. 3, a schematic representation of a part image may be provided.
The industrial part sample can be a reference sample used for judging whether the industrial part to be detected has defects or not. In one embodiment, the embodiment of the application can realize structural defect detection on the industrial part, namely the embodiment of the application can detect the structural defect of the industrial part. For example, when the industrial part is a screw, the embodiment of the application can detect whether the screw has the problems of screw thread missing, screw thread insufficiency and screw hole free. However, there are fewer industrial part samples with structural defects, and thus, in embodiments of the present application, the industrial part to be inspected is inspected for defects with the aid of normal industrial part samples. Thus, the industrial part sample in the embodiment of the present application is a sample in a normal state. For example, an industrial part sample in an embodiment of the present application is one that is free of structural defects.
In one embodiment, the present application performs defect detection on an industrial part to be detected by means of a structural feature cluster library of industrial part samples. The structural feature cluster library comprises part structural feature clusters corresponding to at least two part structural samples of the industrial part samples. For example, if the industrial part sample is composed of 3 part structures, the part structure feature clusters corresponding to the 3 part structures are included in the structure feature cluster library. Wherein one part structure corresponds to at least one part structure feature cluster.
For example, as shown in fig. 4, an industrial part sample is schematically illustrated, wherein the industrial part sample includes 4 part structures, namely a chip body 001, a first pin 002, a second pin 003, and a third pin 004. The structural feature cluster library may include part structural feature clusters corresponding to the 4 part structures, where each part structure may correspond to at least one part structural feature cluster. For example, the chip body may correspond to at least one part structure feature cluster, and the first pin may also correspond to at least one part structure cluster.
In one embodiment, the present application may collect a sample image of an industrial part sample. A structural feature cluster library can then be constructed from the sample images of the industrial part. Specifically, the embodiment of the application can further include:
acquiring sample images of a plurality of industrial part samples, wherein the sample images are images when the industrial part samples are in a normal state, and the industrial part samples comprise at least two part structure samples;
carrying out region detection processing on sample images of a plurality of industrial part samples to obtain region position information of part structure samples in each sample image;
For each sample image, carrying out image segmentation processing on the sample image based on the region position information of the part structure sample to obtain a plurality of structure sample images;
and clustering structural feature information of the structural sample images of the same part structural sample to obtain at least one part structural feature cluster of the part structural sample.
The sample image may include, among other things, an image that captures the appearance of an industrial part sample.
In an embodiment, sample images of a plurality of industrial part samples may be acquired, wherein the sample images are images of the industrial part samples in a normal state, the industrial part samples including at least two part structure samples. For example, images of a plurality of industrial part samples may be taken, resulting in a plurality of sample images.
Wherein the industrial part sample and the industrial part to be detected are the same industrial part. For example, the industrial part to be inspected is a screw, and the industrial part sample is also a screw. For another example, the industrial part to be inspected is a chip, and the industrial part sample is also a chip. For example, sample images of 1000 industrial part samples may be taken, resulting in 1000 sample images. For another example, sample images of 5000 industrial part samples may be taken, resulting in 5000 sample images.
In one embodiment, the sample images of the plurality of industrial part samples may be subjected to a region detection process to obtain region position information of the part structure sample in each sample image. For example, there are 1000 sample images, and the area detection processing may be performed on the 1000 sample images to obtain the area position information of the part structure sample in the 1000 sample images.
For example, assume that the industrial part sample is a screw that includes two part structure samples, a nut and a thread, respectively. The region detection processing is carried out on the sample image, so that the region position information of the screw cap and the region position information of the screw thread in the sample image can be obtained.
In one embodiment, step 102 may refer to "performing region detection processing on sample images of a plurality of industrial part samples to obtain region position information of part structure samples in each sample image". For example, a template image may be acquired, and then a comparison calculation may be performed between the template image and the sample image to obtain a transformation matrix between the template image and the sample image. Then, the sample image may be registered by using the transformation matrix, to obtain a registered sample image. Then, the region identification information of the template image can be mapped to the configured sample image to obtain the region position information of the part structure sample in the sample image.
In an embodiment, for each sample image, image segmentation processing may be performed on the sample image based on the region position information of the part structure sample, so as to obtain a plurality of structure sample images. The step of "performing image segmentation processing on the sample image based on the region position information of the part structure sample to obtain a plurality of structure sample images" may refer to step 103.
For example, assume that the industrial part sample is a screw that includes two part structure samples, a nut and a thread, respectively. By performing image segmentation processing on the sample images based on the region position information of the part structure samples, 1000 structure sample images of the nut and 1000 structure sample images of the screw thread can be obtained.
In an embodiment, the clustering processing is performed on the structural feature information of the structural sample images of the same part structural sample to obtain at least one part structural feature cluster of the part structural sample.
Specifically, the step of clustering structural feature information of a structural sample image of the same part structural sample to obtain at least one part structural feature cluster of the part structural sample may include:
screening out at least one initial clustering center from structural feature information of a structural sample image of the same part structural sample;
Calculating the structural feature information of the structural sample image and sample similarity information between each initial clustering center;
and classifying the structural feature information according to the sample similarity information of the structural feature information to obtain at least one part structural feature cluster of the part structural sample.
In an embodiment, feature extraction may be performed on the structural sample image to obtain structural feature information of the structural sample image. For example, feature extraction may be performed on 1000 structural sample images of the nut, so as to obtain feature information corresponding to the 1000 structural sample images of the nut. For another example, feature extraction may be performed on 1000 structural sample images of the thread, so as to obtain feature information corresponding to the 1000 structural sample images of the thread.
Then, at least one initial clustering center can be screened out from structural feature information corresponding to the structural sample image aiming at the same part structural sample. For example, for a thread, at least one initial cluster center may be screened out of the feature information of 1000 structural sample images of the thread. For example, 100 pieces of feature information can be randomly selected from the 1000 pieces of feature information as an initial cluster center.
Then, the structural feature information of the structural sample image and the sample similarity information between each initial cluster center may be calculated. For example, the structural feature information of the structural sample image and the sample similarity information between each initial cluster center may be calculated from the euclidean distance or the cosine distance.
And then, classifying the structural feature information according to the sample similarity information to obtain at least one part structural feature cluster of the part structural sample. For example, if the similarity between the structural feature information and a certain initial cluster center is the highest, the structural feature information and the initial cluster center are classified into one type.
By the method, at least one structural feature cluster of the part structural sample can be obtained. For example, for a nut, 100 clusters of structural features may be obtained. For threads, 100 clusters of structural features can also be obtained.
102. And carrying out region detection processing on the part image of the industrial part to be detected to obtain region position information of the part structure in the part image.
The area detection of the part image may refer to detecting a position corresponding to each part structure in the part image.
The region position information may be used to describe the position of the part structure in the part image.
In an embodiment, after the part image of the industrial part to be detected is subjected to the region detection processing, a detection frame corresponding to each part structure can be obtained. The area position information may be position information of the detection frame. For example, coordinate axes may be set up on the part image, and then position information of the detection frame may be generated based on the coordinate axes on the part image.
In one embodiment, the placement position and placement angle of the industrial part to be inspected may be different due to different part images. For example, as shown in fig. 5, the placement positions and placement angles of the industrial parts to be inspected in the part image 005 and the part image 006 are not the same.
Because the placement positions and placement angles of the industrial parts to be detected in different part images may be different, in order to improve the accuracy of region detection and thus improve the accuracy of defect detection of the industrial parts to be detected, the part images can be registered by using the model images, and the registered part images are obtained. The template image may then be used to determine region location information for the part structure in the registered part image. Specifically, the step of performing region detection processing on a part image of an industrial part to be detected to obtain region position information of a part structure in the part image may include:
Acquiring a template image aiming at an industrial part to be detected;
performing contrast calculation processing on the template image and the part image to obtain a transformation matrix between the template image and the part image;
registering the part images by utilizing the transformation matrix to obtain registered part images;
and mapping the region identification information of the template image to the registered part image to obtain the region position information of the part structure.
The template image can be used as a reference standard for registering the part image and positioning the part image.
The registering processing of the part image can mean that the angle of the part image is adjusted, so that the placement angle and the position of the industrial part to be detected in the part image are the same as those of the template image. For example, as shown in fig. 6, 007 in fig. 6 may be a template image, 008 may be a part image, and 009 may be a registered part image.
Wherein the transformation matrix can be used to adjust the angle and position of the part image to be the same as the template image.
In one embodiment, a template image for an industrial part to be inspected may be acquired. For example, a template image of the industrial part to be inspected may be generated in advance. The template image may then be stored in a database. When the part image of the industrial part to be detected is subjected to the region detection processing, a template image for the industrial part to be detected can be acquired from the database.
In one embodiment, the template image and the part image may be subjected to a contrast calculation process to obtain a transformation matrix between the template image and the part image. The comparison calculation processing is performed on the template image and the part image, so that outline errors of industrial parts in the part image and the part image can be identified. Then, a transformation matrix that can transform the part image into a template image can be calculated from the contour error of the industrial part.
Specifically, the step of performing a contrast calculation process on the template image and the part image to obtain a transformation matrix between the template image and the part image may include:
performing contour error recognition on the template image and the part image to obtain first contour error information;
transforming the part image by using a preset initial transformation matrix to obtain a transformed part image;
performing contour error recognition on the template image and the transformed part image to obtain second contour error information;
comparing and calculating the first contour error information and the second contour error information to obtain error comparison information;
and based on the error comparison information, adjusting the preset initial transformation matrix to obtain a transformation matrix.
In one embodiment, the transformation matrix may be calculated using a loop optimization approach. A preset initial transformation matrix may thus be initialized in advance. Then, the transformation matrix is obtained by continuously optimizing the preset initial transformation matrix. For example, the predetermined initial transformation matrix may be a randomly generated matrix. For another example, the preset initial transformation matrix may be an identity matrix.
In one embodiment, the contour error recognition may be performed on the template image and the part image to obtain the first contour error information. The outline error recognition is carried out on the template image and the part image, and the errors of the industrial parts in the template image and the part image in the placement angle and the placement position can be judged through the outlines of the industrial parts in the template image and the part image. Therefore, contour recognition can be performed on the template image and the contour image, and a contour point set of the template image are obtained. Then, first contour error information may be calculated from the contour point set of the template image and the contour point set of the template image.
Specifically, the step of performing contour error recognition on the template image and the part image to obtain first contour error information may include:
Carrying out contour recognition on the template image and the part image to obtain a contour point set of the template image and a contour point set of the part image, wherein the contour point set of the template image comprises the position information of at least one template contour point, and the contour point set of the part image comprises the position information of at least one part contour point;
based on the position information of the template contour points and the position information of the part contour points, carrying out matching combination processing on at least one template contour point and at least one part contour point to obtain at least one contour point matching pair;
calculating contour point error information of each contour point matching pair respectively;
and carrying out fusion processing on the contour point error information of each contour point matching pair to obtain first contour error information.
In one embodiment, contour recognition may be performed on the template image and the part image to obtain a set of contour points for the template image and a set of contour points for the part image. Wherein the set of contour points of the template image includes positional information of at least one contour point of the template, and the set of contour points of the part image includes positional information of at least one contour point of the part.
The contour recognition is carried out on the template image and the part image by various methods to obtain a contour point set of the template image and a contour point set of the part image.
For example, the contour recognition can be performed on the template image and the part image by using an artificial intelligence algorithm to obtain a contour point set of the template image and a contour point set of the part image.
Among these, artificial intelligence (Artificial Intelligence, AI) is the theory, method, technique and application system that uses a digital computer or a digital computer-controlled machine to simulate, extend and extend human intelligence, sense the environment, acquire knowledge and use knowledge to obtain optimal results. In other words, artificial intelligence is an integrated technology of computer science that attempts to understand the essence of intelligence and to produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence, i.e. research on design principles and implementation methods of various intelligent machines, enables the machines to have functions of sensing, reasoning and decision.
The artificial intelligence technology is a comprehensive subject, and relates to the technology with wide fields, namely the technology with a hardware level and the technology with a software level. Artificial intelligence infrastructure technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technologies, operation/interaction systems, mechatronics, and the like. The artificial intelligence software technology mainly comprises a computer vision technology, a voice processing technology, a natural language processing technology, machine learning/deep learning and other directions.
Machine Learning (ML) is a multi-domain interdisciplinary, involving multiple disciplines such as probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, etc. It is specially studied how a computer simulates or implements learning behavior of a human to acquire new knowledge or skills, and reorganizes existing knowledge structures to continuously improve own performance. Machine learning is the core of artificial intelligence, a fundamental approach to letting computers have intelligence, which is applied throughout various areas of artificial intelligence. Machine learning and deep learning typically include techniques such as artificial neural networks, belief networks, reinforcement learning, transfer learning, induction learning, teaching learning, and the like. Reinforcement learning is one area of machine learning, among others, emphasizing how to act on an environmental basis to achieve the greatest expected benefit. Deep reinforcement learning is a technique of combining deep learning with reinforcement learning and solving the reinforcement learning problem.
For example, the contour recognition of the template image and the part image may be performed using a convolutional neural network (Convolutional Neural Networks, CNN), a deconvolution neural network (De-Convolutional Networks, DN), a deep neural network (Deep Neural Networks, DNN), a deep convolutional inverse graph network (Deep Convolutional Inverse Graphics Networks, DCIGN), or a Region-based convolutional network (Region-based Convolutional Networks, RCNN), to obtain a contour point set of the template image and a contour point set of the part image.
For another example, the contour recognition may be performed on the template image and the part image by using a pixel method to obtain a contour point set of the template image and a contour point set of the part image. For example, pixel information of the template image may be extracted. In general, the pixel information of the template contour point will be different from the pixel information of other points, so the template contour point can be determined by comparing the pixel points of the template image.
In an embodiment, the step of performing contour recognition on the template image and the part image to obtain a contour point set of the template image and a contour point set of the part image may include:
edge detection is carried out on the template image and the part image, so that a plurality of template contour images corresponding to the template image and a plurality of part contour images corresponding to the part image are obtained;
carrying out probability prediction processing on the template contour images and the part contour images to obtain probability information of each template contour image and probability information of each part contour image;
determining a target template contour image in the plurality of template contour images according to probability information of the template contour image, and determining a target part contour image in the plurality of part contour images according to probability information of the part contour image;
And positioning the target template contour image and the target part contour image to obtain a contour point set of the template image and a contour point set of the part image.
In an embodiment, edge detection may be performed on the template image and the part image to obtain a plurality of template contour images corresponding to the template image and a plurality of part contour images corresponding to the part image.
For example, edge detection may be performed on the template image and the part image based on an artificial intelligent model such as RCNN, fast-RCNN, or fast-RCNN, to obtain a plurality of template contour images corresponding to the template image and a plurality of part contour images corresponding to the part image, respectively. The template contour image may be a rectangular image, and the template contour image includes a result of contour prediction of the template image by the model. For example, as shown in fig. 7, it may be a schematic view of a template profile image, and 010, 011, and 012 may all be template profile images. As can be seen from fig. 7, when the artificial intelligence model performs edge detection on the template image, a plurality of prediction results can be generated, that is, a plurality of template contour images of the template image can be obtained. Similarly, the part contour image may be a rectangular image, where the part contour image includes the result of contour detection of the template image by the model.
In an embodiment, probability prediction processing may be performed on a plurality of template contour images and a plurality of part contour images, to obtain probability information of each template contour image and probability information of each part contour image. Wherein, probability prediction of the template contour image may refer to calculating a probability that the contour prediction of each template contour image is accurate. Similarly, performing probability prediction on the part contour image may refer to calculating the probability that the contour prediction of each part contour image is accurate. For example, probability prediction processing may be performed on the plurality of template contour images and the plurality of part contour images using non-maximal suppression (Non Maximum Suppression, NMS) to obtain probability information for each template contour image and probability information for each part contour image.
In one embodiment, a target template contour image may be determined at a plurality of template contour images based on probability information of the template contour image, and a target part contour image may be determined at a plurality of part contour images based on probability information of the part contour image. For example, a template contour image having the largest probability information may be screened as the target template contour image. For another example, the part contour image with the largest probability information may be screened as the target part contour image.
In an embodiment, since the image edges of the target template contour image and the target part contour image are basically the edges of the industrial part, the target template contour image and the target part contour image can be subjected to positioning processing to obtain a contour point set of the template image and a contour point set of the part image. For example, edge location may be performed on the target template contour image and the target part contour image to obtain a contour point set of the template image and a contour point set of the part image. For example, pixel points at the edge of the outline image of the target template can be sampled to obtain the outline points of the template. Then, a coordinate system may be established based on the template image, and then, position information of the template contour points is determined according to the coordinate system. Next, the template contour points and the position information corresponding to the template contour points may be added to the contour point set of the template image. Similarly, the contour point set of the part image can be obtained by the same method.
In an embodiment, matching and combining processing may be performed on at least one template contour point and at least one part contour point based on the position information of the template contour point and the position information of the part contour point, so as to obtain at least one contour point matching pair.
The matching and combining processing of at least one template contour point and at least one part contour point may refer to associating the corresponding template contour point and part contour point to obtain a contour point matching pair.
For example, the contour point set of the template image includes 72 template contour points, and the contour point set of the part image includes 72 part contour points. Then, the 72 template contour points and the 72 part contour points can be in one-to-one correspondence to obtain at least one contour point matching pair. For example, the template contour point is a contour point of the first pin of the chip, and the part contour point is also a contour point of the first pin of the chip, and the template contour point and the part contour point may constitute a contour point matching pair. For another example, if the template contour point is a contour point of a first pin of the chip and the part contour point is a contour point of a second pin of the chip, the template contour point and the part contour point do not form a contour point matching pair.
In one embodiment, there are a number of ways to match and combine the contour points of the mold and the contour points of the part to obtain at least one contour point matching pair.
For example, a matching relationship between the template contour points and the part contour points can be established by utilizing nearest neighbor search, so that a contour point matching pair is obtained. For another example, a distance matching algorithm may be used to establish a matching relationship between the template contour points and the part contour points, resulting in a contour point matching pair.
In an embodiment, after obtaining at least one contour point matching pair, contour point error information of each contour point matching pair may be calculated separately. There are various methods for calculating the contour point error information of the contour point matching pair. For example, the contour point matching pair includes a template contour point and a part contour point. Then, the euclidean distance or cosine distance between the position information of the mold contour point and the position information of the part contour point can be calculated, and contour point error information of the contour point matching pair can be obtained. For another example, the position information of the mold contour point and the position information of the part contour point may be subjected to a difference operation and then the absolute value may be obtained to obtain the contour point error information of the contour point matching pair.
In an embodiment, the contour point error information of each contour point matching pair may be fused to obtain the first contour error information. For example, the first contour error information may be obtained by adding the contour point error information of each contour point matching pair and dividing the sum by the number of contour point matching pairs.
In one embodiment, the contour point error information for each contour point matching pair may be calculated according to the following formula. Then, carrying out fusion processing on the contour point error information of each contour point matching pair to obtain first contour error information:
Where F may represent the first contour error information and K may represent the number of template contour points. P is p k Coordinate information of a kth part contour point of the part image may be represented.Coordinate information of a kth template contour point of the template image may be represented.
In an embodiment, the transformation matrix may be calculated in a loop-optimized manner. A preset initial transformation matrix may thus be initialized in advance. Then, the transformation matrix is obtained by continuously optimizing the preset initial transformation matrix. Therefore, the part image can be subjected to change processing by using the preset initial change matrix, and the part image after the change is obtained. For example, the initial transformation matrix and the part image may be multiplied to obtain a transformed part image.
Then, contour error recognition can be performed on the template image and the transformed part image to obtain second contour error information. And then, comparing and calculating the first contour error information and the second contour error information to obtain error comparison information. And finally, based on the error comparison information, carrying out adjustment processing on the preset initial transformation matrix to obtain the transformation matrix.
In an embodiment, the step of "performing contour error recognition on the template image and the transformed part image to obtain the second contour error information" the step of "performing contour error recognition on the template image and the part image to obtain the first contour error information" may be referred to, and the description thereof will not be repeated here.
In an embodiment, the first contour error information and the second contour error information may be subjected to a comparison calculation process to obtain error comparison information. For example, the first contour error information and the second contour error information may be subtracted to obtain the absolute value, thereby obtaining the error comparison information.
In an embodiment, the adjustment process may be performed on the preset initial transformation matrix based on the error comparison information, to obtain the transformation matrix.
Specifically, the step of "adjusting the preset initial transformation matrix based on the error comparison information to obtain the transformation matrix" may include:
comparing the error comparison information with a preset comparison threshold value;
when the error comparison threshold is larger than a preset comparison threshold, screening out a target contour point matching pair from at least one contour point matching pair according to contour point error information of each contour point matching pair, wherein the target contour point matching pair comprises a target part contour point and a target template contour point;
removing target template contour points in the contour point set of the template image to obtain an updated contour point set of the template image, and removing target part contour points in the contour point set of the part image to obtain an updated contour point set of the part image;
And adjusting the preset initial transformation matrix based on the updated contour point set of the template image and the updated contour point set of the part image to obtain the transformation matrix.
In one embodiment, the error contrast information may be compared to a preset contrast threshold. When the error contrast information is smaller than or equal to the error contrast threshold value, the transformation matrix is indicated to reach the requirement, and the transformation matrix can be not adjusted any more. And when the error contrast information is larger than the preset contrast threshold value, the transformation matrix still needs to be adjusted.
In an embodiment, when the error comparison threshold is greater than the preset comparison threshold, in order to reduce interference and improve accuracy, the target contour point matching pair may be selected from at least one contour point matching pair according to contour point error information of each contour point matching pair, where the target contour point matching pair includes a target part contour point and a target template contour point. For example, the contour point matching pair with the largest contour error information may be screened out as the target contour point matching pair.
Then, the target template contour points in the contour point set of the template image can be removed to obtain an updated contour point set of the template image, and the target part contour points in the contour point set of the part image can be removed to obtain an updated contour point set of the part image.
Then, the preset initial transformation matrix can be adjusted based on the updated contour point set of the template image and the updated contour point set of the part image, so as to obtain the transformation matrix.
There are various ways in which the transformation matrix can be adjusted. The transformation matrix may be adjusted, for example, by the RANSAC algorithm.
For another example, the step of "adjusting the preset initial transformation matrix based on the updated contour point set of the template image and the updated contour point set of the part image to obtain the transformation matrix" may include:
respectively acquiring transformation reference contour points in an updated contour point set of the template image and an updated contour point set of the part image;
performing parameter estimation processing on a preset initial transformation matrix by using transformation reference contour points to obtain matrix estimation parameters;
evaluating the matrix estimation parameters to obtain an evaluation result of the matrix estimation parameters;
and replacing matrix parameters of a preset initial transformation matrix with matrix estimation parameters according to the evaluation result to obtain the transformation matrix.
In an embodiment, the transformed reference contour points may be acquired in an updated contour point set of the template image and an updated contour point set of the part image, respectively. For example, the same number of transformed reference contour points may be acquired in the updated contour point set of the template image and the updated contour point set of the part image, respectively. For example, if 4 transformation reference contour points are collected in the contour point set after updating of the template image, 4 transformation reference contour points may be collected in the contour point set after updating of the part image. In addition, the transformation reference contour points can be acquired in the updated contour point set of the template image and the updated contour point set of the part image according to the dimension of the transformation matrix. For example, assuming that the dimension of the transformation matrix is 3*3, a minimum of 8 transformation reference contour points need to be acquired. Wherein the updated set of contour points may be sampled in a number of ways. For example, the updated set of contour points may be randomly sampled or equally sampled, etc.
In an embodiment, the transformation reference contour points may be used to perform parameter estimation processing on the preset initial transformation matrix to obtain matrix estimation parameters. For example, the interior points may be screened out from the transformed reference point contour points, and then modeling processing may be performed on the screened interior points based on a preset initial transformation matrix to obtain model parameters. The model parameters resulting from the modeling may be matrix estimation parameters.
In one embodiment, to further improve accuracy, the matrix estimation parameters may be evaluated. For example, other transformation reference contour points which are not screened into interior points can be carried into the initial transformation matrix for calculation, so that whether the matrix estimation parameters calculated by the interior points are the current optimal solution is judged. If the matrix estimation parameters calculated by the inner points are the current optimal solution, the matrix parameters of the preset initial transformation matrix can be replaced by the matrix estimation parameters according to the evaluation result, and the transformation matrix is obtained.
In an embodiment, after the transformation matrix is calculated, the registration processing may be performed on the part image by using the transformation matrix, so as to obtain a registered part image. For example, the transformation matrix and the part image may be multiplied to obtain a registered part image. For example, as shown in fig. 6, a transformation matrix is applied to the part image. By carrying out registration processing on the part images, the positions and angles of the industrial parts in the registered part images are aligned with those of the industrial parts in the template images.
In an embodiment, after the configured part image is obtained, the region identification information of the template image may be mapped to the registered part image to obtain the region position information of the part structure in the part image. For example, the reference region position information of the part structure may be calibrated in advance in the template image, and then the reference region position information of the template image may be used as the region position information of the part structure. For example, as shown in fig. 8, the reference region position information of the template image 013 may be used as the region position information of the part image 014. From the reference area position information of the template image, it is possible to know where the positions of the nuts and threads, respectively, are in the part image.
103. And carrying out image segmentation processing on the part image based on the region position information of the part structure to obtain a structural image corresponding to the part structure of the industrial part to be detected.
In an embodiment, after the area position information of the part structure in the part image is obtained, the part image may be subjected to image segmentation processing according to the area position information of the part structure, so as to obtain a structural image corresponding to the part structure of the industrial part to be detected.
For example, as shown in fig. 8, it is assumed that the industrial part to be inspected is a screw. After the area position information of the screw thread and the screw cap in the part image is obtained, the part image can be subjected to image segmentation processing to obtain a structural image of the screw thread and a structural image of the screw cap.
104. And carrying out matching processing on the feature information corresponding to the structural image of the part structure and the part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected.
In an embodiment, after obtaining the structural image corresponding to the part structure of the industrial part to be detected, feature extraction may be performed on the structural image of each part structure to obtain feature information of the structural image. For example, the feature extraction may be performed on the structural image of the part structure by using an artificial intelligent network such as CNN or DNN, to obtain feature information of the structural image.
In an embodiment, feature information corresponding to a structural image of a part structure and part structural feature clusters of an industrial part sample in a structural feature cluster library can be matched to obtain a defect detection result of the industrial part to be detected.
For example, for a part structure nut, the feature information corresponding to the structural image of the nut and the part structure feature clusters of the nut sample in the structure feature cluster library can be matched, so as to judge whether the nut has structural defects.
For another example, for the part structure screw thread, the feature information corresponding to the structure image of the screw thread and the part structure feature cluster of the screw thread sample in the structure feature cluster library can be matched, so as to judge whether the screw thread has structural defects.
In an embodiment, the step of performing matching processing on feature information corresponding to a structural image of a part structure and part structural feature clusters of an industrial part sample in a structural feature cluster library to obtain a defect detection result of the industrial part to be detected may include:
carrying out statistical processing on sample feature information in each part structural feature cluster of the industrial part sample to obtain a cluster center corresponding to each part structural feature cluster;
performing similarity calculation on feature information corresponding to the structural image of the part structure and a clustering center corresponding to each part structural feature cluster to obtain at least one piece of similarity information;
and judging at least one piece of similarity information by using a preset similarity threshold value to obtain a defect detection result of the industrial part to be detected.
In an embodiment, sample feature information in each part structural feature cluster of the industrial part sample may be statistically processed to obtain a cluster center corresponding to each part structural feature cluster.
For example, if the industrial part sample is a screw, the structural feature cluster library includes a plurality of part structural feature clusters of the screw thread and a plurality of part structural feature clusters of the screw cap. Then, the clustering center corresponding to each part structural feature cluster of the thread can be obtained, and a plurality of clustering centers of the thread are obtained. And similarly, the clustering centers corresponding to the structural feature clusters of each part of the nut can be obtained, and a plurality of clustering centers of the nut are obtained.
The clustering center corresponding to each part structural feature cluster can be obtained in various modes.
For example, feature information in the part structural feature clusters may be averaged to obtain a cluster center of the part structural feature clusters. For another example, the median value of the feature information in the part structural feature cluster may be used as the cluster center, and so on. For example, the cluster center corresponding to the part structural feature cluster may be calculated according to the following formula:
wherein, center k May refer to a cluster center corresponding to a kth part structural feature cluster of the part structure. C (C) k May refer to the number of feature information in the kth part structural feature cluster. X is x i Feature information for a kth part structural feature cluster may be represented.
In an embodiment, similarity calculation may be performed on feature information corresponding to a structural image of the part structure and a cluster center corresponding to each part structural feature cluster, to obtain at least one piece of similarity information.
For example, a part structure sample nut corresponds to 100 cluster centers. And then, carrying out similarity calculation on the characteristic information of the part structure nuts of the part to be detected and the 100 clustering centers to obtain the characteristic information of the nuts and the similarity information between each clustering center.
Wherein, how many ways can calculate the characteristic information of nut and the similarity information between each cluster center. For example, the feature information of the nuts and the similarity information between each cluster center may be calculated from cosine distances or euclidean distances.
Similarly, for example, part structure threads correspond to 100 cluster centers. Then, similarity calculation can be performed on the feature information of the part structure threads of the part to be detected and the 100 clustering centers, so that the feature information of the threads and the similarity information between each clustering center are obtained.
And then, judging and processing at least one piece of similarity information according to a preset similarity threshold value to obtain a defect detection result of the industrial part to be detected.
Specifically, the step of performing a discrimination process on at least one similarity information by using a preset similarity threshold to obtain a defect detection result of the industrial part to be detected may include:
when the similarity information in the at least one piece of similarity information accords with a preset similarity threshold value, generating and outputting a defect detection result of the industrial part to be detected through detection;
when each piece of similarity information does not accord with a preset similarity threshold value, determining a defective part structure of the industrial part to be detected;
Part structure names and region position information of defective part structures are generated and output.
In one embodiment, the industrial part sample in a normal state is utilized to detect the defects of the industrial part to be detected. Therefore, when the similarity information in the at least one piece of similarity information accords with the preset similarity threshold value, the characteristic information of the part structure sample and the characteristic information of the part structure are similar, and the part structure is in a normal state.
For example, if at least one similarity information is greater than or equal to 0.5, it is assumed that the preset similarity threshold is 0.5, which indicates that the part structure is normal. For example, for a part structure nut, if there is similarity information between the feature information of the nut and the cluster center that is greater than or equal to 0.5, it is indicated that the nut is normal.
In an embodiment, when the similarity information in the at least one piece of similarity information accords with the preset similarity threshold value, a defect detection result of the industrial part to be detected passing detection can be generated and output.
In one embodiment, when each of the similarity information does not meet a predetermined similarity threshold, a defective part structure of the industrial part to be inspected may be determined. For example, the feature information of the part structure and the similarity of the clustering center do not meet the preset similarity threshold, and the part structure may be considered as not in a normal state, that is, the part structure is a defective part structure. Accordingly, a defective part structure of the industrial part to be inspected can be determined, and part structure name and region position information of the defective part structure can be generated and outputted.
For example, if the feature information of the part structure nuts and the similarity between each cluster center do not meet the preset similarity threshold, it is indicated that the nuts are structures with structural defects. Then, the detection result of the structural defect of the nut can be output.
By the embodiment of the application, the structural defect detection of the industrial part to be detected can be realized. For example, as shown in fig. 9, the embodiment of the present application can identify, based on the part structural feature cluster of the industrial part sample in the normal state, the structural defect that the industrial part 1 to be detected has the first pin damaged, and also identify the structural defect that the industrial part sample 2 to be detected has the second pin missing.
In the embodiment of the application, a part image of an industrial part to be detected and a structural feature cluster library of an industrial part sample can be taken, wherein the industrial part to be detected comprises at least two part structures, and the structural feature cluster library comprises part structural feature clusters corresponding to at least two part structural samples when the industrial part sample is in a normal state; performing region detection processing on a part image of the industrial part to be detected to obtain region position information of a part structure in the part image; based on the regional position information of the part structure, performing image segmentation processing on the part image to obtain a structural image corresponding to the part structure of the industrial part to be detected; and carrying out matching processing on the feature information corresponding to the structural image of the part structure and the part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected. The embodiment of the application can realize structural defect detection of industrial parts. In the embodiment of the application, the structural defect detection can be carried out on the industrial part to be detected according to the structural feature cluster library by constructing the structural feature cluster library of the industrial part. By means of the method for constructing the structural feature cluster library of the industrial parts, the method provided by the embodiment of the application is high in generalization and accuracy. Moreover, by means of the method for constructing the structural feature cluster library of the industrial parts, compared with a traditional artificial intelligence algorithm, a large amount of manual parameter adjustment work is not needed, and the detection process is simplified.
According to the method described in the above embodiments, examples are described in further detail below.
The method of the embodiment of the application will be described by taking the example of integrating the defect detection method of the industrial part on a server.
In one embodiment, as shown in fig. 10, a method for detecting defects of an industrial part comprises the following steps:
201. the server acquires sample images of a plurality of industrial part samples, wherein the sample images are images when the industrial part samples are in a normal state, and the industrial part samples comprise at least two part structure samples.
In one embodiment, a structural anomaly sample of an industrial part is difficult to obtain, and therefore it is difficult to train a classification model to make anomaly determinations. However, the normal industrial part samples of the industrial part are insufficient, and thus, part structural feature clusters can be constructed by a large number of normal industrial part samples.
Thus, the server may obtain sample images of a plurality of industrial part samples, wherein the sample images are images of the industrial part samples in a normal state, and the industrial part samples include at least two part structure samples. For example, the server may obtain sample images of a plurality of screws that are not defective. For example, the server may obtain 1000 sample images of a screw without a defect.
202. And the server performs area detection processing on sample images of the plurality of industrial part samples to obtain area position information of the part structure samples in each sample image.
For example, as shown in fig. 11, the server may perform a region detection process on sample images of a plurality of industrial part samples using a template map, to obtain region position information of the part structure samples in each sample image. For example, a template image may be acquired, and then a comparison calculation may be performed between the template image and the sample image to obtain a transformation matrix between the template image and the sample image. Then, the sample image may be registered by using the transformation matrix, to obtain a registered sample image. Then, the region identification information of the template image can be mapped to the configured sample image to obtain the region position information of the part structure sample in the sample image.
For example, the server may perform region detection processing on a sample image of 1000 screws without defects to obtain region position information of nuts and threads in the sample image
203. The server performs image segmentation processing on the sample images based on the region position information of the part structure samples for each sample image to obtain a plurality of structure sample images.
For example, by dividing the sample image, 1000 structural sample images of nuts and 1000 structural sample images of threads can be obtained.
204. And the server performs clustering processing on the structural feature information of the structural sample images of the same part structural sample to obtain at least one part structural feature cluster of the part structural sample.
For example, the server may perform feature extraction on the structure sample images of 1000 nuts to obtain structure feature information. For example, feature extraction may be performed on the structural sample image using the trained depth model to obtain structural feature information.
For another example, the server may perform feature extraction on the 1000 threaded structure sample images to obtain structure feature information.
Then, the structural feature information of the screw cap can be clustered to obtain at least one part structural feature cluster of the screw cap. And similarly, the structural feature information of the threads can be clustered to obtain at least one part structural feature cluster of the threads.
Because the number of normal part samples is often relatively large, 2 problems can be generated by directly storing the features of all normal samples into the image block feature library, 1 is that the memory occupied by the feature library is excessively large, and 2 is that the test sample needs to be compared with the features of all normal samples when abnormal detection is caused, so that the time consumption is very high. Therefore, the structural feature information of the structural sample image of the same part structural sample can be clustered, and the specific method can be as follows:
For the structural feature information of the structural sample image of each part structural sample, randomly selecting 100 features as initial clustering centers.
For example, 100 pieces of feature information can be randomly selected from the 1000 pieces of feature information as an initial cluster center.
Then, the structural feature information of the structural sample image and the sample similarity information between each initial cluster center may be calculated. For example, the structural feature information of the structural sample image and the sample similarity information between each initial cluster center may be calculated from the euclidean distance or the cosine distance.
And then, classifying the structural feature information according to the sample similarity information to obtain at least one part structural feature cluster of the part structural sample. For example, if the similarity between the structural feature information and a certain initial cluster center is the highest, the structural feature information and the initial cluster center are classified into one type.
By the method, at least one structural feature cluster of the part structural sample can be obtained. For example, for a nut, 100 clusters of structural features may be obtained. For threads, 100 clusters of structural features can also be obtained.
205. The server acquires part images of the industrial part to be detected, wherein the industrial part to be detected comprises at least two part structures.
For example, as shown in fig. 11, the server acquires a part image of an industrial part to be inspected.
206. The server performs region detection processing on the part image of the industrial part to be detected to obtain region position information of the part structure in the part image.
In one embodiment, the server may collect a set of template images by taking a golden sample (a sample of the absolute normal component that is detected) and photographing the sample at all points of the image that need to be detected, and the collected set of images is the template images.
Then, as shown in fig. 11, the server may perform registration processing on the part image using the template image, to obtain a registered part image. The purpose of image registration is to correct the input part picture so that the position of the part picture can be completely aligned with the template image of the corresponding point position, thereby being convenient for accurately extracting the structural image of the part structure.
As shown in fig. 11, the difference between the positions and angles of the part image and the template image is large, the transformation matrix T between the template image and the part image can be calculated by the image registration module, and the registered part image can be obtained by applying the transformation matrix T. The part image after registration is aligned with the template image in visual position and angle, and then the structural image in the part image can be intercepted according to the position information of the pre-calibrated reference area in the template image.
In one embodiment, the transformation matrix may be calculated by a minimum error iterative method. The method comprises the steps of calculating contour error information, defining parameters to be estimated, and optimizing the parameters by using an iterative algorithm based on current estimation to gradually reduce the contour error information. As shown in FIG. 12, the method of the application firstly carries out edge detection on the part image and the model image to obtain the corresponding contour image. Then, NMS operation can be performed on the contour image partitions in parallel to obtain a contour point set. The average error of the contour point set between the part image and the template image is contour error information.
After the contour error information is set, as shown in fig. 12, a specific iterative optimization mode is that an initial transformation matrix (initial unit matrix) is firstly applied to a contour point set, a matching relation between the contour point set before a part image and a template image is established by nearest neighbor search, a contour point matching pair with overlarge error is eliminated, a transformation matrix is estimated by RANSAC (random sample area) and then the transformation matrix is recycled to the first step until the contour error information is obtained.
207. And the server performs image segmentation processing on the part image based on the region position information of the part structure to obtain a structure image corresponding to the part structure of the industrial part to be detected.
208. And the server performs matching processing on the feature information corresponding to the structural image of the part structure and the part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected.
For example, the cluster center corresponding to the part structural feature cluster may be calculated according to the following formula:
wherein, center k May refer to a cluster center corresponding to a kth part structural feature cluster of the part structure. C (C) k May refer to the number of feature information in the kth part structural feature cluster. X is x i Feature information for a kth part structural feature cluster may be represented.
For example, a part structure sample nut corresponds to 100 cluster centers. And then, carrying out similarity calculation on the characteristic information of the part structure nuts of the part to be detected and the 100 clustering centers to obtain the characteristic information of the nuts and the similarity information between each clustering center.
For example, the feature information and the similarity information between each cluster center may be calculated from cosine similarity.
And then, judging and processing at least one piece of similarity information by utilizing a preset similarity threshold value to obtain a defect detection result of the industrial part to be detected.
For example, if at least one similarity information is greater than or equal to 0.5, it is assumed that the preset similarity threshold is 0.5, which indicates that the part structure is normal. For example, for a part structure nut, if there is similarity information between the feature information of the nut and the cluster center that is greater than or equal to 0.5, it is indicated that the nut is normal.
If each similarity information is smaller than 0.5, the part structure is abnormal.
In the embodiment of the application, a server acquires sample images of a plurality of industrial part samples, wherein the sample images are images of the industrial part samples in a normal state, and the industrial part samples comprise at least two part structure samples; the server performs area detection processing on sample images of a plurality of industrial part samples to obtain area position information of part structure samples in each sample image; the server performs image segmentation processing on the sample images based on the region position information of the part structure samples aiming at each sample image to obtain a plurality of structure sample images; the server performs clustering processing on the structural feature information of the structural sample images of the same part structural sample to obtain at least one part structural feature cluster of the part structural sample; the method comprises the steps that a server obtains part images of industrial parts to be detected, wherein the industrial parts to be detected comprise at least two part structures; the server performs area detection processing on the part image of the industrial part to be detected to obtain area position information of a part structure in the part image; the server performs image segmentation processing on the part image based on the region position information of the part structure to obtain a structure image corresponding to the part structure of the industrial part to be detected; the server performs matching processing on feature information corresponding to the structural image of the part structure and part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected, and the accuracy of detecting the defects of the industrial part can be improved.
In order to better implement the method for detecting defects of industrial parts provided by the embodiment of the application, in one embodiment, a device for detecting defects of industrial parts is also provided, and the device for detecting defects of industrial parts can be integrated in computer equipment. Wherein the meaning of the terms is the same as in the above-mentioned defect detection method of industrial parts, specific implementation details can be referred to the description in the method embodiments.
In one embodiment, there is provided a defect detecting apparatus for an industrial part, which may be integrated in a computer device, as shown in fig. 13, the defect detecting apparatus for an industrial part including: an acquisition unit 301, a region detection unit 302, an image segmentation unit 303, and a matching unit 304 are specifically as follows:
an obtaining unit 301, configured to obtain a part image of an industrial part to be detected and a structural feature cluster library of industrial part samples, where the industrial part to be detected includes at least two part structures, and the structural feature cluster library includes part structural feature clusters corresponding to at least two part structural samples when the industrial part samples are in a normal state;
the area detection unit 302 is configured to perform area detection processing on the part image of the industrial part to be detected, so as to obtain area position information of a part structure in the part image;
An image segmentation unit 303, configured to perform image segmentation processing on the part image based on the region position information of the part structure, so as to obtain a structural image corresponding to the part structure of the industrial part to be detected;
and the matching unit 304 is configured to perform matching processing on feature information corresponding to the structural image of the part structure and part structural feature clusters of the industrial part sample in the structural feature cluster library, so as to obtain a defect detection result of the industrial part to be detected.
In an embodiment, the area detecting unit 302 may include:
an image acquisition subunit, configured to acquire a template image for the industrial part to be detected;
the contrast calculation subunit is used for carrying out contrast calculation processing on the template image and the part image to obtain a transformation matrix between the template image and the part image;
the registration subunit is used for carrying out registration processing on the part image by utilizing the transformation matrix to obtain a registered part image;
and the mapping subunit is used for mapping the region identification information of the template image to the registered part image to obtain the region position information of the part structure.
In an embodiment, the comparison computation subunit may include:
the first contour error recognition module is used for carrying out contour error recognition on the template image and the part image to obtain first contour error information;
the transformation module is used for transforming the part image by utilizing a preset initial transformation matrix to obtain a transformed part image;
the second contour error recognition module is used for carrying out contour error recognition on the template image and the transformed part image to obtain second contour error information;
the comparison calculation module is used for carrying out comparison calculation processing on the first contour error information and the second contour error information to obtain error comparison information;
and the adjustment module is used for adjusting the preset initial transformation matrix based on the error comparison information to obtain the transformation matrix.
In an embodiment, the first contour error identification module may include:
the contour recognition sub-module is used for carrying out contour recognition on the template image and the part image to obtain a contour point set of the template image and a contour point set of the part image, wherein the contour point set of the template image comprises the position information of at least one template contour point, and the contour point set of the part image comprises the position information of at least one part contour point;
The matching combination sub-module is used for carrying out matching combination processing on the at least one template contour point and the at least one part contour point based on the position information of the template contour point and the position information of the part contour point to obtain at least one contour point matching pair;
the calculation sub-module is used for calculating the contour point error information of each contour point matching pair respectively;
and the fusion sub-module is used for carrying out fusion processing on the contour point error information of each contour point matching pair to obtain the first contour error information.
In an embodiment, the profile recognition sub-module may include:
the edge detection assembly is used for carrying out edge detection on the template image and the part image to obtain a plurality of template contour images corresponding to the template image and a plurality of part contour images corresponding to the part image;
the probability prediction component is used for carrying out probability prediction processing on the template contour images and the part contour images to obtain probability information of each template contour image and probability information of each part contour image;
a determining component for determining a target template contour image at the plurality of template contour images based on probability information of the template contour image, and determining a target part contour image at the plurality of part contour images based on probability information of the part contour image;
And the positioning component is used for positioning the target template contour image and the target part contour image to obtain a contour point set of the template image and a contour point set of the part image.
In an embodiment, the adjusting module may include:
the comparison sub-module is used for comparing the error comparison information with a preset comparison threshold value;
the screening sub-module is used for screening out a target contour point matching pair from the at least one contour point matching pair according to contour point error information of each contour point matching pair when the error comparison threshold is larger than the preset comparison threshold, wherein the target contour point matching pair comprises a target part contour point and a target template contour point;
the removing submodule is used for removing the target template contour points in the contour point set of the template image to obtain an updated contour point set of the template image, and removing the target part contour points in the contour point set of the part image to obtain an updated contour point set of the part image;
and the adjustment sub-module is used for adjusting the preset initial transformation matrix based on the updated contour point set of the template image and the updated contour point set of the part image to obtain the transformation matrix.
In an embodiment, the adjusting sub-module may include:
the acquisition component is used for acquiring transformation reference contour points in the updated contour point set of the template image and the updated contour point set of the part image respectively;
the parameter estimation component is used for carrying out parameter estimation processing on the preset initial transformation matrix by utilizing the transformation reference contour points to obtain the matrix estimation parameters;
the evaluation component is used for performing evaluation processing on the matrix estimation parameters to obtain an evaluation result of the matrix estimation parameters;
and the replacing component is used for replacing the matrix parameters of the preset initial transformation matrix with the matrix estimation parameters according to the evaluation result to obtain the transformation matrix.
In an embodiment, the matching unit 304 may include:
the statistics subunit is used for carrying out statistics processing on sample feature information in each part structural feature cluster of the industrial part sample to obtain a cluster center corresponding to each part structural feature cluster;
the similarity calculation subunit is used for calculating the similarity of the feature information corresponding to the structural image of the part structure and the clustering center corresponding to each part structural feature cluster to obtain at least one piece of similarity information;
And the judging subunit is used for judging the at least one similarity information by utilizing a preset similarity threshold value to obtain a defect detection result of the industrial part to be detected.
In an embodiment, the determining subunit may include:
the generation module is used for generating and outputting a defect detection result of the industrial part to be detected passing detection when the similarity information in the at least one piece of similarity information accords with the preset similarity threshold value;
the structure determining module is used for determining the defective part structure of the industrial part to be detected when each piece of similarity information does not accord with the preset similarity threshold value;
and the output module is used for generating and outputting the part structure name and the region position information of the defective part structure.
In an embodiment, the defect detecting apparatus may further include:
the sample acquisition unit is used for acquiring sample images of a plurality of industrial part samples, wherein the sample images are images when the industrial part samples are in a normal state, and the industrial part samples comprise at least two part structure samples;
the sample area detection unit is used for carrying out area detection processing on sample images of the plurality of industrial part samples to obtain area position information of part structure samples in each sample image;
The sample image segmentation unit is used for carrying out image segmentation processing on the sample images based on the region position information of the part structure samples aiming at each sample image to obtain a plurality of structure sample images;
and the clustering unit is used for carrying out clustering processing on the structural feature information of the structural sample images of the same part structural sample to obtain at least one part structural feature cluster of the part structural sample.
In an embodiment, the clustering unit may include:
a central screening subunit, configured to screen at least one initial clustering center from structural feature information of a structural sample image of the same part structural sample;
the calculating subunit is used for calculating the structural feature information of the structural sample image and the sample similarity information between each initial clustering center;
and the classifying sub-unit is used for classifying the structural feature information according to the sample similarity information of the structural feature information to obtain at least one part structural feature cluster of the part structural sample.
In the implementation, each unit may be implemented as an independent entity, or may be implemented as the same entity or several entities in any combination, and the implementation of each unit may be referred to the foregoing method embodiment, which is not described herein again.
The defect detection device for the industrial parts can improve the accuracy of structural defect detection on the industrial parts.
The embodiment of the application also provides computer equipment, which can comprise a terminal or a server, for example, the computer equipment can be used as a defect detection terminal of an industrial part, and the terminal can be a mobile phone, a tablet computer and the like; for another example, the computer device may be a server, such as a defect detection server for an industrial part, or the like. As shown in fig. 14, a schematic structural diagram of a terminal according to an embodiment of the present application is shown, specifically:
the computer device may include one or more processors 401 of a processing core, memory 402 of one or more computer readable storage media, a power supply 403, and an input unit 404, among other components. Those skilled in the art will appreciate that the computer device structure shown in FIG. 14 is not limiting of the computer device and may include more or fewer components than shown, or may be combined with certain components, or a different arrangement of components. Wherein:
the processor 401 is a control center of the computer device, connects various parts of the entire computer device using various interfaces and lines, performs various functions of the computer device and processes data by running or executing software programs and/or modules stored in the memory 402, and calling data stored in the memory 402. Optionally, processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user page, an application program, etc., and the modem processor mainly processes wireless communication. It will be appreciated that the modem processor described above may not be integrated into the processor 401.
The memory 402 may be used to store software programs and modules, and the processor 401 executes various functional applications and data processing by executing the software programs and modules stored in the memory 402. The memory 402 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program (such as a sound playing function, an image playing function, etc.) required for at least one function, and the like; the storage data area may store data created according to the use of the computer device, etc. In addition, memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.
The computer device further comprises a power supply 403 for supplying power to the various components, preferably the power supply 403 may be logically connected to the processor 401 by a power management system, so that functions of charge, discharge, and power consumption management may be performed by the power management system. The power supply 403 may also include one or more of any of a direct current or alternating current power supply, a recharging system, a power failure detection circuit, a power converter or inverter, a power status indicator, and the like.
The computer device may also include an input unit 404, which input unit 404 may be used to receive input numeric or character information and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.
Although not shown, the computer device may further include a display unit or the like, which is not described herein. In particular, in this embodiment, the processor 401 in the computer device loads executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 executes the application programs stored in the memory 402, so as to implement various functions as follows:
acquiring a part image of an industrial part to be detected and a structural feature cluster library of an industrial part sample, wherein the industrial part to be detected comprises at least two part structures, and the structural feature cluster library comprises part structural feature clusters corresponding to at least two part structural samples when the industrial part sample is in a normal state;
performing region detection processing on the part image of the industrial part to be detected to obtain region position information of a part structure in the part image;
Based on the regional position information of the part structure, performing image segmentation processing on the part image to obtain a structure image corresponding to the part structure of the industrial part to be detected;
and carrying out matching processing on feature information corresponding to the structural image of the part structure and part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
According to one aspect of the present application, there is provided a computer program product or computer program comprising computer instructions stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the methods provided in the various alternative implementations of the above embodiments.
It will be appreciated by those of ordinary skill in the art that all or part of the steps of the various methods of the above embodiments may be performed by a computer program, or by computer program control related hardware, which may be stored in a computer readable storage medium and loaded and executed by a processor.
To this end, an embodiment of the present application also provides a storage medium in which a computer program is stored, the computer program being capable of being loaded by a processor to perform the steps of any of the method for detecting defects of industrial parts provided by the embodiment of the present application. For example, the computer program may perform the steps of:
acquiring a part image of an industrial part to be detected and a structural feature cluster library of an industrial part sample, wherein the industrial part to be detected comprises at least two part structures, and the structural feature cluster library comprises part structural feature clusters corresponding to at least two part structural samples when the industrial part sample is in a normal state;
performing region detection processing on the part image of the industrial part to be detected to obtain region position information of a part structure in the part image;
based on the regional position information of the part structure, performing image segmentation processing on the part image to obtain a structure image corresponding to the part structure of the industrial part to be detected;
and carrying out matching processing on feature information corresponding to the structural image of the part structure and part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected.
The specific implementation of each operation above may be referred to the previous embodiments, and will not be described herein.
The steps in the defect detection method for any industrial part provided by the embodiment of the present application can be executed by the computer program stored in the storage medium, so that the beneficial effects of the defect detection method for any industrial part provided by the embodiment of the present application can be achieved, which are detailed in the previous embodiments and are not described herein.
The foregoing has outlined rather broadly the principles and embodiments of the present application in order that the detailed description of the method and apparatus for detecting defects in an industrial part and the computer device provided herein may be implemented in any particular manner; meanwhile, as those skilled in the art will have variations in the specific embodiments and application scope in light of the ideas of the present application, the present description should not be construed as limiting the present application.

Claims (15)

1. A method for detecting defects in an industrial part, comprising:
acquiring a part image of an industrial part to be detected and a structural feature cluster library of an industrial part sample, wherein the industrial part to be detected comprises at least two part structures, and the structural feature cluster library comprises part structural feature clusters corresponding to at least two part structural samples when the industrial part sample is in a normal state;
Performing region detection processing on the part image of the industrial part to be detected to obtain region position information of a part structure in the part image;
based on the regional position information of the part structure, performing image segmentation processing on the part image to obtain a structure image corresponding to the part structure of the industrial part to be detected;
and carrying out matching processing on feature information corresponding to the structural image of the part structure and part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected.
2. The method according to claim 1, wherein the performing the region detection processing on the part image of the industrial part to be detected to obtain the region position information of the part structure in the part image includes:
acquiring a template image aiming at the industrial part to be detected;
performing contrast calculation processing on the template image and the part image to obtain a transformation matrix between the template image and the part image;
registering the part images by utilizing the transformation matrix to obtain registered part images;
and mapping the region identification information of the template image to the registered part image to obtain the region position information of the part structure.
3. The method of claim 2, wherein the performing a contrast calculation process on the template image and the part image to obtain a transformation matrix between the template image and the part image includes:
performing contour error recognition on the template image and the part image to obtain first contour error information;
transforming the part image by using a preset initial transformation matrix to obtain a transformed part image;
performing contour error recognition on the template image and the transformed part image to obtain second contour error information;
comparing and calculating the first contour error information and the second contour error information to obtain error comparison information;
and based on the error comparison information, adjusting the preset initial transformation matrix to obtain the transformation matrix.
4. A method according to claim 3, wherein said performing contour error recognition on said template image and said part image to obtain said first contour error information comprises:
performing contour recognition on the template image and the part image to obtain a contour point set of the template image and a contour point set of the part image, wherein the contour point set of the template image comprises position information of at least one template contour point, and the contour point set of the part image comprises position information of at least one part contour point;
Based on the position information of the template contour points and the position information of the part contour points, carrying out matching combination processing on the at least one template contour point and the at least one part contour point to obtain at least one contour point matching pair;
calculating contour point error information of each contour point matching pair respectively;
and carrying out fusion processing on the contour point error information of each contour point matching pair to obtain the first contour error information.
5. The method of claim 4, wherein performing contour recognition on the template image and the part image to obtain a contour point set of the template image and a contour point set of the part image comprises:
performing edge detection on the template image and the part image to obtain a plurality of template contour images corresponding to the template image and a plurality of part contour images corresponding to the part image;
carrying out probability prediction processing on the template contour images and the part contour images to obtain probability information of each template contour image and probability information of each part contour image;
determining a target template contour image in the plurality of template contour images according to probability information of the template contour image, and determining a target part contour image in the plurality of part contour images according to probability information of the part contour image;
And carrying out positioning processing on the target template contour image and the target part contour image to obtain a contour point set of the template image and a contour point set of the part image.
6. The method of claim 4, wherein the adjusting the preset initial transformation matrix based on the error comparison information to obtain the transformation matrix comprises:
comparing the error comparison information with a preset comparison threshold;
when the error comparison threshold is larger than the preset comparison threshold, screening out a target contour point matching pair from the at least one contour point matching pair according to contour point error information of each contour point matching pair, wherein the target contour point matching pair comprises a target part contour point and a target template contour point;
removing the target template contour points in the contour point set of the template image to obtain an updated contour point set of the template image, and removing the target part contour points in the contour point set of the part image to obtain an updated contour point set of the part image;
and adjusting the preset initial transformation matrix based on the updated contour point set of the template image and the updated contour point set of the part image to obtain the transformation matrix.
7. The method of claim 6, wherein the adjusting the preset initial transformation matrix based on the updated set of contour points of the template image and the updated set of contour points of the part image to obtain the transformation matrix comprises:
respectively acquiring transformation reference contour points in the updated contour point set of the template image and the updated contour point set of the part image;
performing parameter estimation processing on the preset initial transformation matrix by using the transformation reference contour points to obtain matrix estimation parameters;
performing evaluation processing on the matrix estimation parameters to obtain an evaluation result of the matrix estimation parameters;
and replacing matrix parameters of the preset initial transformation matrix with the matrix estimation parameters according to the evaluation result to obtain the transformation matrix.
8. The method according to claim 1, wherein the matching the feature information corresponding to the structural image of the part structure with the part structural feature clusters of the industrial part samples in the structural feature cluster library to obtain the defect detection result of the industrial part to be detected includes:
carrying out statistical processing on sample feature information in each part structural feature cluster of the industrial part sample to obtain a cluster center corresponding to each part structural feature cluster;
Performing similarity calculation on feature information corresponding to the structural image of the part structure and a clustering center corresponding to each part structural feature cluster to obtain at least one piece of similarity information;
and judging the at least one piece of similarity information by using a preset similarity threshold value to obtain a defect detection result of the industrial part to be detected.
9. The method of claim 8, wherein the performing the discriminating processing on the at least one similarity information by using a preset similarity threshold value to obtain a defect detection result of the industrial part to be detected includes:
when the similarity information in the at least one piece of similarity information accords with the preset similarity threshold value, generating and outputting a defect detection result of the industrial part to be detected through detection;
when each piece of similarity information does not accord with the preset similarity threshold value, determining a defective part structure of the industrial part to be detected;
and generating and outputting the part structure name and the region position information of the defective part structure.
10. The method according to claim 1, wherein the method further comprises:
acquiring sample images of a plurality of industrial part samples, wherein the sample images are images when the industrial part samples are in a normal state, and the industrial part samples comprise at least two part structure samples;
Carrying out region detection processing on sample images of the plurality of industrial part samples to obtain region position information of part structure samples in each sample image;
for each sample image, carrying out image segmentation processing on the sample image based on the region position information of the part structure sample to obtain a plurality of structure sample images;
and clustering structural feature information of the structural sample images of the same part structural sample to obtain at least one part structural feature cluster of the part structural sample.
11. The method according to claim 10, wherein the clustering the structural feature information of the structural sample images of the same part structural sample to obtain at least one part structural feature cluster of the part structural sample includes:
screening out at least one initial clustering center from the structural feature information of the structural sample image of the structural sample of the same part;
calculating the structural feature information of the structural sample image and sample similarity information between each initial clustering center;
and classifying the structural feature information according to the sample similarity information of the structural feature information to obtain at least one part structural feature cluster of the part structural sample.
12. A defect detection device for an industrial part, comprising:
the device comprises an acquisition unit, a detection unit and a detection unit, wherein the acquisition unit is used for acquiring a part image of an industrial part to be detected and a structural feature cluster library of an industrial part sample, the industrial part to be detected comprises at least two part structures, and the structural feature cluster library comprises part structural feature clusters corresponding to at least two part structural samples when the industrial part sample is in a normal state;
the area detection unit is used for carrying out area detection processing on the part image of the industrial part to be detected to obtain area position information of a part structure in the part image;
the image segmentation unit is used for carrying out image segmentation processing on the part image based on the region position information of the part structure to obtain a structure image corresponding to the part structure of the industrial part to be detected;
and the matching unit is used for carrying out matching processing on the feature information corresponding to the structural image of the part structure and the part structural feature clusters of the industrial part sample in the structural feature cluster library to obtain a defect detection result of the industrial part to be detected.
13. A computer device comprising a memory and a processor; the memory stores an application program, and the processor is configured to execute the application program in the memory to perform the operations in the defect detection method of an industrial part according to any one of claims 1 to 11.
14. A computer readable storage medium storing a plurality of instructions adapted to be loaded by a processor to perform the steps in the method of defect detection of an industrial part according to any one of claims 1 to 11.
15. A computer program product comprising computer programs or instructions which, when executed by a processor, implement the steps in the method of defect detection of industrial parts according to any one of claims 1 to 11.
CN202211539122.9A 2022-12-01 2022-12-01 Defect detection method and device for industrial parts and computer equipment Pending CN116977250A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202211539122.9A CN116977250A (en) 2022-12-01 2022-12-01 Defect detection method and device for industrial parts and computer equipment

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202211539122.9A CN116977250A (en) 2022-12-01 2022-12-01 Defect detection method and device for industrial parts and computer equipment

Publications (1)

Publication Number Publication Date
CN116977250A true CN116977250A (en) 2023-10-31

Family

ID=88473708

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202211539122.9A Pending CN116977250A (en) 2022-12-01 2022-12-01 Defect detection method and device for industrial parts and computer equipment

Country Status (1)

Country Link
CN (1) CN116977250A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117830319A (en) * 2024-03-06 2024-04-05 陕西星辰电子技术有限责任公司 Power adapter product detection method based on image processing

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117830319A (en) * 2024-03-06 2024-04-05 陕西星辰电子技术有限责任公司 Power adapter product detection method based on image processing

Similar Documents

Publication Publication Date Title
Li et al. Automatic pavement crack detection by multi-scale image fusion
US11645744B2 (en) Inspection device and inspection method
Wu et al. Automatic fabric defect detection using a wide-and-light network
CN109711228B (en) Image processing method and device for realizing image recognition and electronic equipment
CN112016401A (en) Cross-modal-based pedestrian re-identification method and device
US9025863B2 (en) Depth camera system with machine learning for recognition of patches within a structured light pattern
CN111126482A (en) Remote sensing image automatic classification method based on multi-classifier cascade model
CN109711443A (en) Floor plan recognition methods, device, equipment and storage medium neural network based
CN110717554A (en) Image recognition method, electronic device, and storage medium
CN116977250A (en) Defect detection method and device for industrial parts and computer equipment
Zhang et al. Multi-scale defect detection of printed circuit board based on feature pyramid network
CN103913150B (en) Intelligent electric energy meter electronic devices and components consistency detecting method
CN110135428B (en) Image segmentation processing method and device
CN111738319A (en) Clustering result evaluation method and device based on large-scale samples
CN114998432A (en) YOLOv 5-based circuit board detection point positioning method
CN106709490B (en) Character recognition method and device
CN114581654A (en) Mutual inductor based state monitoring method and device
KR101789979B1 (en) Method for calculating hausdorff distance based on gradient orientation information
CN113793370A (en) Three-dimensional point cloud registration method and device, electronic equipment and readable medium
CN111222558A (en) Image processing method and storage medium
CN113139540B (en) Backboard detection method and equipment
CN113344994B (en) Image registration method, device, electronic equipment and storage medium
CN114355234A (en) Intelligent quality detection method and system for power module
CN115619698A (en) Method and device for detecting defects of circuit board and model training method
CN108073934A (en) Nearly multiimage detection method and device

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination