CN117291918B - Automobile stamping part defect detection method based on three-dimensional point cloud - Google Patents
Automobile stamping part defect detection method based on three-dimensional point cloud Download PDFInfo
- Publication number
- CN117291918B CN117291918B CN202311581899.6A CN202311581899A CN117291918B CN 117291918 B CN117291918 B CN 117291918B CN 202311581899 A CN202311581899 A CN 202311581899A CN 117291918 B CN117291918 B CN 117291918B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- defect
- conveyor belt
- defects
- mechanical arm
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 230000007547 defect Effects 0.000 title claims abstract description 127
- 238000001514 detection method Methods 0.000 title claims abstract description 59
- 238000000034 method Methods 0.000 claims abstract description 48
- 239000013598 vector Substances 0.000 claims description 29
- 230000008569 process Effects 0.000 claims description 11
- 230000005477 standard model Effects 0.000 claims description 9
- 230000002146 bilateral effect Effects 0.000 claims description 8
- 230000011218 segmentation Effects 0.000 claims description 8
- 230000002950 deficient Effects 0.000 claims description 5
- 238000013135 deep learning Methods 0.000 abstract description 3
- 238000013507 mapping Methods 0.000 abstract description 3
- 238000010586 diagram Methods 0.000 description 7
- 238000004519 manufacturing process Methods 0.000 description 7
- 238000005259 measurement Methods 0.000 description 6
- 238000012545 processing Methods 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 3
- 239000011159 matrix material Substances 0.000 description 3
- 230000009466 transformation Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 230000008901 benefit Effects 0.000 description 2
- 238000005336 cracking Methods 0.000 description 2
- 238000000605 extraction Methods 0.000 description 2
- 238000005070 sampling Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013528 artificial neural network Methods 0.000 description 1
- 230000001174 ascending effect Effects 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 238000013461 design Methods 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 230000006872 improvement Effects 0.000 description 1
- 238000007373 indentation Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 238000012544 monitoring process Methods 0.000 description 1
- 238000011176 pooling Methods 0.000 description 1
- 238000003908 quality control method Methods 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000011895 specific detection Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/0093—Programme-controlled manipulators co-operating with conveyor means
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/89—Investigating the presence of flaws or contamination in moving material, e.g. running paper or textiles
- G01N21/892—Investigating the presence of flaws or contamination in moving material, e.g. running paper or textiles characterised by the flaw, defect or object feature examined
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/89—Investigating the presence of flaws or contamination in moving material, e.g. running paper or textiles
- G01N21/892—Investigating the presence of flaws or contamination in moving material, e.g. running paper or textiles characterised by the flaw, defect or object feature examined
- G01N21/8921—Streaks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V10/00—Arrangements for image or video recognition or understanding
- G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
- G06V10/762—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using clustering, e.g. of similar faces in social networks
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01N—INVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
- G01N21/00—Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
- G01N21/84—Systems specially adapted for particular applications
- G01N21/88—Investigating the presence of flaws or contamination
- G01N21/89—Investigating the presence of flaws or contamination in moving material, e.g. running paper or textiles
- G01N21/892—Investigating the presence of flaws or contamination in moving material, e.g. running paper or textiles characterised by the flaw, defect or object feature examined
- G01N2021/8924—Dents; Relief flaws
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/30—Computing systems specially adapted for manufacturing
Abstract
The invention relates to the technical field of defect detection of automobile stamping parts, and discloses a three-dimensional point cloud-based automobile stamping part defect detection method, which comprises the steps of preliminarily judging whether a defect exists by adopting an image registration method based on SURF through an image acquisition system, carrying out defect positioning and category judgment on a defect part by using a target detection method based on deep learning, optimally mapping acquired image information onto a standard image, enabling points of two images corresponding to the same position in space to correspond one by one, and realizing rapid detection of dominant defects such as scratches, pits, cracks and the like of the stamping parts; and acquiring point cloud data by using a structured light sensor, and accurately detecting hidden defects such as necking, hidden cracks, folds and the like by using an algorithm combining point cloud characteristics and model matching. The defect detection method for the automobile stamping part based on the three-dimensional point cloud can accurately detect the dominant defect and the recessive defect of the automobile stamping part.
Description
Technical Field
The invention relates to the technical field of defect detection of automobile stamping parts, in particular to a three-dimensional point cloud-based defect detection method for automobile stamping parts.
Background
Automobile stamping parts are one of important production parts in the field of automobile manufacturing, and measurement and detection of automobile stamping parts are quality requirements of various automobile factories and spare part factories at present. The manufacturing defect types of the stamping parts are complicated and various due to the complicated and various manufacturing processes of the stamping parts. The traditional automobile stamping part detection method based on manpower has the problems of low efficiency, high error rate and the like, and has the defects of low detection efficiency and hidden danger of workpiece quality.
Compared with the traditional defect detection method for the artificial stamping part, the method based on visual image processing has become a mainstream method for detecting defects in the industrial field due to high efficiency and strong robustness. However, the pure vision method cannot reliably detect the slight recessive defects such as necking, indentation and the like. In recent years, the three-dimensional laser scanning technology is rapidly developed, and a new method is provided for detecting stamping part defects which are difficult to process by visual methods.
Disclosure of Invention
The invention aims to provide a three-dimensional point cloud-based automobile stamping part defect detection method, which solves the problem that the conventional automobile stamping part detection method cannot accurately detect hidden defects.
In order to achieve the above purpose, the invention provides a three-dimensional point cloud-based automobile stamping part defect detection method, which uses an automobile stamping part defect detection device, wherein the automobile stamping part defect detection device comprises a conveyor belt, a bracket and a mechanical arm group are sequentially arranged in the conveying direction of the conveyor belt, an image acquisition system is arranged at the top end of the bracket, the mechanical arm group comprises a sensor mechanical arm and a grabbing mechanical arm, the sensor mechanical arm is arranged at one side of the conveyor belt, the grabbing mechanical arm is arranged at the other side of the conveyor belt, a point cloud acquisition structure is arranged on the sensor mechanical arm, and the point cloud acquisition structure is arranged behind the conveying direction of the conveyor belt of the image acquisition system;
the defect detection method for the automobile stamping part comprises the following steps:
firstly, detecting dominant defects of stamping parts to be detected through an image acquisition system at the top end of a bracket, if the stamping parts to be detected are qualified products, performing the next process, and if the stamping parts to be detected are defective parts, sorting the stamping parts by a grabbing mechanical arm;
secondly, for stamping parts without dominant defects, carrying out invisible defect detection on one part of stamping parts reaching a point cloud acquisition structure of a sensor mechanical arm along with a conveyor belt, grabbing the other part of stamping parts to another conveyor belt through a grabbing mechanical arm, and carrying out invisible defect detection through the point cloud acquisition structure on the other conveyor belt;
step three, for stamping parts without hidden defects on the conveyor belt, entering the next working procedure along with the conveyor belt, and sorting the stamping parts with hidden defects on the conveyor belt by using a grabbing mechanical arm; and sorting out stamping parts with hidden defects on the other conveyor belt by using a grabbing mechanical arm, grabbing stamping parts without hidden defects on the other conveyor belt by using the grabbing mechanical arm until the stamping parts with hidden defects enter the next working procedure.
Preferably, the image acquisition system is a system combining a high-definition camera with a matched light source.
Preferably, for the dominant defect, a high-definition camera and a matched light source in an image acquisition system are used, an SURF-based image registration method is adopted to map acquired image information onto a standard image, so that points of two images corresponding to the same position in space are in one-to-one correspondence, whether the defect exists or not is judged through defect-free information of the mapped images, and the detection of the dominant defect of the stamping part is completed.
Preferably, the point cloud acquisition structure is a structural light sensor, for the hidden defect, the point cloud data is acquired through the structural light sensor, the point cloud denoising is carried out on the acquired point cloud data, the normal vector and curvature of the point cloud are calculated, and the clustering segmentation algorithm based on the point cloud region growth is used for judging the flat region and the characteristic region of the stamping part.
Preferably, for the flat area, a bilateral weight algorithm is adopted to find the difference between the defect position and a standard area, in the standard area, the vector direction from any point to the adjacent point is perpendicular to the normal vector direction, in the defect area, the vector direction from the target point to the adjacent point is not perpendicular to the normal vector direction, and whether the flat area has an implicit defect is judged.
Preferably, for the feature region, calculating the distance between the point cloud of the part to be detected and the standard model by a method of matching the point cloud of the stamping part with the standard model, and judging whether the feature region has an implicit defect by setting a distance threshold.
Therefore, the defect detection method for the automobile stamping part based on the three-dimensional point cloud has the following beneficial effects:
(1) The invention adopts an image acquisition system, adopts an SURF-based image registration method to preliminarily judge whether defects exist, optimally maps the acquired image information to a standard image, enables the two images to correspond to points at the same position in space one by one, and realizes the rapid detection of dominant defects such as scratches, pits, cracks and the like of stamping parts;
(2) The method comprises the steps of collecting point cloud data of a stamping part by adopting a point cloud collecting system, denoising the point cloud, calculating normal vector and curvature of the point cloud, and judging a flat area and a characteristic area of the stamping part by using a clustering segmentation algorithm based on the growth of a point cloud area; for the flat area, searching the difference between the defect position and the standard area by adopting a bilateral weight algorithm, and further judging whether the flat area has hidden defects or not; for the characteristic region, accurate detection of hidden defects such as necking, hidden cracking, wrinkling and the like is realized through an algorithm combining point cloud characteristics and model matching;
(3) According to the invention, the defect detection system of the automobile stamping part can detect the dominant defect and the recessive defect of the stamping part, can be applied to production to realize on-line detection, can realize rapid scanning on a large-area stamping part, improves the detection efficiency, and shortens the detection time.
The technical scheme of the invention is further described in detail through the drawings and the embodiments.
Drawings
FIG. 1 is a schematic diagram of a defect detection device for an automotive stamping part according to an embodiment of the invention;
FIG. 2 is a flowchart illustrating a hidden defect detection process according to an embodiment of the present invention;
FIG. 3 is a schematic diagram illustrating a calculation of a defect position by a bilateral weight algorithm according to an embodiment of the present invention;
FIG. 4 is a schematic diagram of point cloud data and a standard model of an automotive stamping part according to an embodiment of the invention;
FIG. 5 is a schematic diagram of registration of point cloud data of an automotive stamping part with a standard model according to an embodiment of the invention;
FIG. 6 is a schematic diagram of calculating a distance between a point cloud of a stamping part and a standard model according to an embodiment of the present invention;
FIG. 7 is a flowchart of an image registration method based on the SURF method according to an embodiment of the invention;
FIG. 8 is a schematic diagram of a network architecture of a point of sale network according to an embodiment of the invention;
fig. 9 is a schematic diagram of an image capturing system according to an embodiment of the present invention.
Reference numerals
1. A conveyor belt; 2. a bracket; 3. an image acquisition system; 4. a sensor robotic arm; 5. grabbing a mechanical arm; 6. a point cloud acquisition structure; 7. stamping.
Detailed Description
The technical scheme of the invention is further described below through the attached drawings and the embodiments.
Unless defined otherwise, technical or scientific terms used herein should be given the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs. The terms "first," "second," and the like, as used herein, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "disposed," "mounted," "connected," and "connected" are to be construed broadly, and may be, for example, fixedly connected, detachably connected, or integrally connected; can be mechanically or electrically connected; can be directly connected or indirectly connected through an intermediate medium, and can be communication between two elements. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed.
Examples
As shown in FIG. 1, the defect detection device for the automobile stamping part comprises a conveyor belt 1, wherein a bracket 2 and a mechanical arm group are sequentially arranged in the conveying direction of the conveyor belt 1. The top of support 2 is provided with image acquisition system 3, and the robotic arm group includes sensor robotic arm 4 and snatchs robotic arm 5, and sensor robotic arm 4 sets up in one side of conveyer belt 1, snatchs robotic arm 5 setting at the opposite side of conveyer belt 1. The sensor mechanical arm 4 is provided with a point cloud acquisition structure 6. The point cloud acquisition structure 6 is arranged behind the conveying direction of the conveyor belt 1 of the image acquisition system 3. As shown in fig. 9, the image acquisition system 3 is a system combining a high-definition camera with a matched light source, and the point cloud acquisition structure 6 is a structured light sensor.
The defect detection device for the automobile stamping part detects easily-observed defects (hereinafter referred to as dominant defects) and only weak thickness-changing defects (hereinafter referred to as recessive defects) on the surface of the stamping part 7 respectively.
As shown in fig. 2, the method for detecting the defects of the automobile stamping part based on the three-dimensional point cloud comprises the following steps:
firstly, the stamping part 7 to be detected firstly carries out dominant defect detection through the image acquisition system 3 at the top end of the bracket 2, if the stamping part is detected to be a qualified product, the next process is carried out, and if the stamping part is detected to be a defective product, the stamping part is sorted out through the grabbing mechanical arm 5.
For the dominant defects, a high-definition camera and a matched light source in the image acquisition system 3 are used, an SURF-based image registration method is adopted to map acquired image information onto a standard image, points of two images corresponding to the same position in space are in one-to-one correspondence, whether defects exist or not is judged through the defect-free information of the mapped images, and the detection of the dominant defects of the stamping parts 7 is completed.
Secondly, for stamping parts 7 without dominant defects, one part of stamping parts reaches a point cloud acquisition structure 6 of a sensor mechanical arm 4 along with the conveyor belt 1 to carry out invisible defect detection, and the other part of stamping parts is grabbed to the other conveyor belt 1 through a grabbing mechanical arm 5 to carry out invisible defect detection through the point cloud acquisition structure 6 on the other conveyor belt 1.
Step three, for stamping parts 7 without hidden defects on the conveyor belt 1, entering the next working procedure along with the conveyor belt 1, and sorting out the stamping parts 7 with hidden defects on the conveyor belt 1 by using a grabbing mechanical arm 5; the stamping parts 7 with hidden defects on the other conveyor belt 1 are sorted out by using the grabbing mechanical arm 5, and the stamping parts 7 without hidden defects on the other conveyor belt 1 are grabbed to the original conveyor belt 1 by using the grabbing mechanical arm 5 to enter the next working procedure.
And for the hidden defects, point cloud data are acquired through a structural light sensor, point cloud denoising is carried out on the acquired point cloud data, normal vectors and curvatures of the point cloud are calculated, and a clustering segmentation algorithm based on the point cloud region growth is used for judging the flat region and the characteristic region of the stamping part 7.
As shown in fig. 3, for a flat area, a bilateral weight algorithm is used to find the difference between the defect position and a standard area, in the standard area, the vector direction from any point to its adjacent point is perpendicular to the normal vector direction, in the defect area, the vector direction from the target point to its adjacent point is not perpendicular to the normal vector direction, and whether the flat area has an implicit defect is judged.
And for the characteristic region, calculating the distance between the point cloud of the part to be detected and the standard model by a method for matching the point cloud of the stamping part 7 with the standard model, and judging whether the characteristic region has an implicit defect by setting a distance threshold.
The specific detection device and the specific method adopted in the embodiment are as follows:
(1) data acquisition and processing:
image data acquisition:
image acquisition is completed by adopting an MER2-1220-32U3M/C model camera, and the parameters are as follows:
；
and (3) acquiring point cloud data:
as shown in fig. 4, 5 and 6, in order to meet the present detection requirement, a line structured light sensor of model LJ-G200 manufactured by Keyence corporation is selected in this embodiment. LJ strips can realize the simultaneous measurement of a plurality of points, and the image sensor adopted by the LJ strips ensures the consistency of the measurement process, and in addition, the sampling speed of the Quattro connection system is highest in the sensors of the same type, so that the high-speed sampling is ensured, and meanwhile, the high measurement precision is ensured. The working principle of the sensor is a direct laser triangulation method, the sensor emits laser beams through a laser emitter, the laser beams strike the surface of a measured object, the laser beams are received by a receiver through diffuse reflection, obtained data are transmitted to a computer, after data processing, point cloud data in the width direction and the height direction of the measured object can be obtained, the point cloud data in the width direction of the surface of the measured object are measured by an X axis, and the point cloud data in the height direction are measured by a Z axis. The device has the advantages that the measurement result is basically not influenced by external factors (light intensity, surface color of the measured object and the like), the measurement result is stable, the high-precision response is high, and the like. The parameter data of the LJ-G200 are shown in the table, and the structured light sensor meets the requirement of precision required by the experiment.
Parameters of LJ-G200
；
(2) Image registration based on the SURF method:
in this embodiment, in order to effectively detect the defect of the surface of the stamping part 7, an image registration method based on SURF (Speeded Up Robust Features) is adopted, as shown in fig. 7. First, feature points of the surface images of the two stamping parts 7 are extracted by using the SURF algorithm, and feature descriptors of the feature points are calculated. In order to quickly and accurately match the feature points, a K-D tree nearest neighbor search method is adopted, and the method has high efficiency and can effectively find the matching relation between the corresponding feature points. Subsequently, in order to eliminate mismatching points due to noise or other interference factors, a RANSAC (Random Sample Consensus) algorithm is applied, which can robustly estimate the transformation matrix between the two surface images of the stamping 7. The transformation matrix is used for accurately registering the two images so as to align the two images in corresponding areas, thereby realizing reliable detection and positioning of the surface defects of the stamping part 7, and detecting whether the image of the stamping part 7 has dominant defects or not by calculating image deviation. By the method, high-quality detection of surface defects of the stamping part 7 can be realized, and powerful support is provided for quality control and production process monitoring.
(3) Clustering segmentation algorithm based on point cloud region growth:
the embodiment adopts a clustering segmentation algorithm based on the point cloud region growth so as to effectively subdivide the acquired point cloud data into a flat region and a characteristic region. The algorithm flow is as follows: first, the point cloud data calculates the algorithm vector and curvature, and sorts the point cloud data in ascending order of curvature. Then, the point with the lowest curvature is used as the initial seed point. Next, neighboring points around the initial seed point are compared to the seed point cloud. In the process, a normal angle threshold is set for screening the neighborhood points of the seed points. Only if the angle between the normal of the neighborhood point and the normal of the current seed point is smaller than the threshold value, the neighborhood point is included in the current region.
A curvature threshold is then set to determine if the curvature of the neighborhood points is small enough to indicate that they lie under similar degrees of curvature. For each neighborhood point, checking if its curvature is less than a curvature threshold, if so, adding the neighborhood point to the sequence of seed points and deleting the current seed point to continue growing new seed points. Only if the neighborhood point meets the normal angle and curvature threshold will it be considered available for the seed point.
The above process will be repeated until the sequence of seed points is emptied. At this point, the growth process for a region is completed, which is added to the cluster array to represent a flat region or feature region. The above steps are then repeated for the remaining points until all points have been traversed. Through the algorithm, the point cloud data can be effectively divided into different areas, the flat areas and the characteristic areas are distinguished, and powerful support is provided for point cloud analysis and processing.
(4) Planar region defect extraction is achieved based on a bilateral weight algorithm:
the key to the bilateral weight integration algorithm is to find the difference between the defective area and the standard area where the vector direction from any point to its neighbors is perpendicular to the normal vector direction. In the defect area, the vector direction from the target point to its neighboring point is not perpendicular to the normal vector direction. The vector difference between each point in the point cloud and the entire neighborhood in the normal direction is calculated by the above analysis using the following formula.
（1）
Is a point near the sphere of the sphere,nis the normal vector of the pie and,N(p)is a set of neighbor points that are selected,mis the number of neighbor points of the pie, because the number of different spherical community points is different, the neighbors of each sample point need to be normalized to avoid the neighborhood of a different number of points due to calculation errors, +.>Is->And->Euclidean distance between->Is->Distance in normal direction. Implicit defect detection of flat area point cloud can be realized through bilateral weight algorithm。
(5) Point cloud registration realizes feature region defect extraction:
in the detection of the hidden defects on the surface of the characteristic area of the stamping part 7, the point cloud registration method has better expressive force. The present embodiment employs a two-stage registration strategy, first using a four-Point method (4-Point Congruent Sets) for coarse registration, followed by a Point-to-face closest Point iterative algorithm (Point-to-Plane Iterative Closest Point) for further accurate registration. The four-point method is based on the concept of conformal set, and realizes initial point cloud registration by randomly selecting reference points and finding four closest points. And the point-to-plane ICP algorithm optimizes rigid transformation by using a least square method through nearest neighbor search and projection distance calculation of normal vectors on the basis of considering normal information in the point cloud, so that accurate registration of the point cloud is realized. This strategy shows a significant improvement in accuracy with a change in curvature of the facing surface, providing a reliable method for detecting surface hidden defects in the feature area of the stamping 7.
The following is a core formula of the point-to-face nearest point iterative algorithm:
（2）
（3）
wherein the method comprises the steps ofFor original point cloud->As the cloud of target points,R,Tin order to transform the matrix,dfitting a plane for the distance of the original point cloud to the target point cloud,/->Is the object ofThe point cloud corresponds to the normal vector of the point neighborhood.
(6) Deep learning-based defect classification:
in this embodiment, classification of point cloud defects is achieved through a point network, as shown in fig. 8, and the design of the network mainly includes two key parts: pointNet Encoder and PointNet Decoder. The PointNet Encoder is responsible for taking as input the coordinates of each point in the point cloud, mapping it into a fixed length high-dimensional vector representation. In this process, multi-layer perceptron (MLP) and pooling operations are skillfully combined to capture the local and global features of each point. In this way, each point in the point cloud is assigned a feature vector with rich semantic information. These local features are then integrated into a global feature vector, which can be regarded as an abstract representation of the entire point cloud. The PointNet Decoder section then uses this global feature vector to perform specific tasks such as classification or segmentation. At this stage, an additional neural network layer is introduced for mapping global features to specific class labels or segmentation boundaries. This process is end-to-end, meaning that the entire network can be trained by a back-propagation algorithm, enabling the network to automatically learn the optimal feature representation to facilitate performance of subsequent tasks.
Therefore, the defect detection method for the automobile stamping part based on the three-dimensional point cloud is used for detecting the easily observed defects (scratches, pits and the like) and the defects (necking, hidden cracks and the like) with only weak thickness change on the surface of the stamping part, so that the automatic online detection of production is realized. For the defects (display defects) which are easy to observe on the surface, a high-definition camera and a matched light source are adopted, the advantage of large image processing detection range is utilized, an SURF-based image registration method is adopted to preliminarily judge whether defects exist, then a deep learning-based target detection method is used to carry out defect positioning and category judgment on a defective part, collected image information is optimally mapped onto a standard image, so that points of two images corresponding to the same position in space correspond one by one, and the rapid detection of dominant defects such as scratches, pits and cracks of a stamping part is realized. For the defects (hidden defects) with only weak thickness change, a structured light sensor is used for acquiring point cloud data, and the hidden defects such as necking, hidden cracking, wrinkling and the like are accurately detected through an algorithm combining point cloud characteristics and model matching.
Finally, it should be noted that: the above embodiments are only for illustrating the technical solution of the present invention and not for limiting it, and although the present invention has been described in detail with reference to the preferred embodiments, it will be understood by those skilled in the art that: the technical scheme of the invention can be modified or replaced by the same, and the modified technical scheme cannot deviate from the spirit and scope of the technical scheme of the invention.
Claims (5)
1. A defect detection method for an automobile stamping part based on three-dimensional point cloud is characterized by comprising the following steps: the defect detection method for the automobile stamping part uses a defect detection device for the automobile stamping part, the defect detection device for the automobile stamping part comprises a conveyor belt, a support and a mechanical arm set are sequentially arranged in the conveying direction of the conveyor belt, an image acquisition system is arranged at the top end of the support, the mechanical arm set comprises a sensor mechanical arm and a grabbing mechanical arm, the sensor mechanical arm is arranged at one side of the conveyor belt, the grabbing mechanical arm is arranged at the other side of the conveyor belt, a point cloud acquisition structure is arranged on the sensor mechanical arm, and the point cloud acquisition structure is arranged at the rear of the conveying direction of the conveyor belt of the image acquisition system;
the defect detection method for the automobile stamping part comprises the following steps:
firstly, detecting dominant defects of stamping parts to be detected through an image acquisition system at the top end of a bracket, if the stamping parts to be detected are qualified products, performing the next process, and if the stamping parts to be detected are defective parts, sorting the stamping parts by a grabbing mechanical arm;
secondly, for stamping parts without dominant defects, carrying out invisible defect detection on one part of stamping parts reaching a point cloud acquisition structure of a sensor mechanical arm along with a conveyor belt, grabbing the other part of stamping parts to another conveyor belt through a grabbing mechanical arm, and carrying out invisible defect detection through the point cloud acquisition structure on the other conveyor belt;
step three, for stamping parts without hidden defects on the conveyor belt, entering the next working procedure along with the conveyor belt, and sorting out the stamping parts with hidden defects on the conveyor belt by using a grabbing mechanical arm; sorting out stamping parts with hidden defects on the other conveyor belt by using a grabbing mechanical arm, grabbing stamping parts without hidden defects on the other conveyor belt by using the grabbing mechanical arm until the stamping parts with hidden defects on the other conveyor belt enter the next working procedure;
the point cloud acquisition structure is a structure light sensor, point cloud data are acquired through the structure light sensor, point cloud denoising is carried out on the acquired point cloud data, normal vectors and curvatures of the point cloud are calculated, a clustering segmentation algorithm based on the point cloud region growth is used for judging the stamping part flat region and the characteristic region, and whether the stamping part flat region and the characteristic region have invisible defects or not is judged.
2. The three-dimensional point cloud-based automobile stamping part defect detection method is characterized by comprising the following steps of: the image acquisition system is a system combining a high-definition camera with a matched light source.
3. The three-dimensional point cloud-based automobile stamping part defect detection method is characterized by comprising the following steps of: for the dominant defects, a high-definition camera and a matched light source in an image acquisition system are used, an SURF-based image registration method is adopted to map acquired image information onto a standard image, so that points of two images corresponding to the same position in space are in one-to-one correspondence, whether defects exist or not is judged through the defect-free information of the mapped images, and the detection of the dominant defects of stamping parts is completed.
4. The three-dimensional point cloud-based automobile stamping part defect detection method is characterized by comprising the following steps of: and for the flat area, searching the difference between the defect position and the standard area by adopting a bilateral weight algorithm, wherein in the standard area, the vector direction from any point to the adjacent point is perpendicular to the normal vector direction, and in the defect area, the vector direction from the target point to the adjacent point is not perpendicular to the normal vector direction, and judging whether the flat area has an implicit defect.
5. The three-dimensional point cloud-based automobile stamping part defect detection method is characterized by comprising the following steps of: and for the characteristic region, calculating the distance between the point cloud of the part to be detected and the standard model by a method for matching the point cloud of the stamping part with the standard model, and judging whether the characteristic region has an implicit defect by setting a distance threshold.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311581899.6A CN117291918B (en) | 2023-11-24 | 2023-11-24 | Automobile stamping part defect detection method based on three-dimensional point cloud |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202311581899.6A CN117291918B (en) | 2023-11-24 | 2023-11-24 | Automobile stamping part defect detection method based on three-dimensional point cloud |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN117291918A CN117291918A (en) | 2023-12-26 |
| CN117291918B true CN117291918B (en) | 2024-02-06 |
Family
ID=89258984
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202311581899.6A Active CN117291918B (en) | 2023-11-24 | 2023-11-24 | Automobile stamping part defect detection method based on three-dimensional point cloud |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117291918B (en) |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106204618A (en) * | 2016-07-20 | 2016-12-07 | 南京文采科技有限责任公司 | Product surface of package defects detection based on machine vision and sorting technique |
| CN108648168A (en) * | 2018-03-15 | 2018-10-12 | 北京京仪仪器仪表研究总院有限公司 | IC wafer surface defects detection methods |
| CN109523501A (en) * | 2018-04-28 | 2019-03-26 | 江苏理工学院 | One kind being based on dimensionality reduction and the matched battery open defect detection method of point cloud data |
| CN109772724A (en) * | 2019-03-14 | 2019-05-21 | 溧阳市新力机械铸造有限公司 | A kind of flexible detection and analysis system of casting emphasis surface and internal flaw |
| CN112098419A (en) * | 2020-09-11 | 2020-12-18 | 江苏理工学院 | System and method for detecting surface defects of automobile outer covering part |
| TW202100987A (en) * | 2019-06-25 | 2021-01-01 | 林聖傑 | Three-dimensional image surface defect detection system capable of performing an accurate, effective and simultaneous process of various defects including curvature defects, cavity defects and excessive material defects |
| CN113588655A (en) * | 2021-06-30 | 2021-11-02 | 江苏智配新材料科技有限公司 | Detection device for surface defects of MDF fiber lines and working method thereof |
| CN114119464A (en) * | 2021-10-08 | 2022-03-01 | 厦门微亚智能科技有限公司 | Lithium battery cell top cover welding seam appearance detection algorithm based on deep learning |
| CN115496746A (en) * | 2022-10-20 | 2022-12-20 | 复旦大学 | Method and system for detecting surface defects of plate based on fusion of image and point cloud data |
| CN115841477A (en) * | 2022-12-15 | 2023-03-24 | 北京交通大学 | Multi-mode visual fusion ballastless track structure hidden defect detection method and system |
| CN116958146A (en) * | 2023-09-20 | 2023-10-27 | 深圳市信润富联数字科技有限公司 | Acquisition method and device of 3D point cloud and electronic device |
| CN117036641A (en) * | 2023-06-08 | 2023-11-10 | 四川轻化工大学 | Road scene three-dimensional reconstruction and defect detection method based on binocular vision |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| MX2022003020A (en) * | 2019-09-17 | 2022-06-14 | Boston Polarimetrics Inc | Systems and methods for surface modeling using polarization cues. |
-
2023
- 2023-11-24 CN CN202311581899.6A patent/CN117291918B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106204618A (en) * | 2016-07-20 | 2016-12-07 | 南京文采科技有限责任公司 | Product surface of package defects detection based on machine vision and sorting technique |
| CN108648168A (en) * | 2018-03-15 | 2018-10-12 | 北京京仪仪器仪表研究总院有限公司 | IC wafer surface defects detection methods |
| CN109523501A (en) * | 2018-04-28 | 2019-03-26 | 江苏理工学院 | One kind being based on dimensionality reduction and the matched battery open defect detection method of point cloud data |
| CN109772724A (en) * | 2019-03-14 | 2019-05-21 | 溧阳市新力机械铸造有限公司 | A kind of flexible detection and analysis system of casting emphasis surface and internal flaw |
| TW202100987A (en) * | 2019-06-25 | 2021-01-01 | 林聖傑 | Three-dimensional image surface defect detection system capable of performing an accurate, effective and simultaneous process of various defects including curvature defects, cavity defects and excessive material defects |
| CN112098419A (en) * | 2020-09-11 | 2020-12-18 | 江苏理工学院 | System and method for detecting surface defects of automobile outer covering part |
| CN113588655A (en) * | 2021-06-30 | 2021-11-02 | 江苏智配新材料科技有限公司 | Detection device for surface defects of MDF fiber lines and working method thereof |
| CN114119464A (en) * | 2021-10-08 | 2022-03-01 | 厦门微亚智能科技有限公司 | Lithium battery cell top cover welding seam appearance detection algorithm based on deep learning |
| CN115496746A (en) * | 2022-10-20 | 2022-12-20 | 复旦大学 | Method and system for detecting surface defects of plate based on fusion of image and point cloud data |
| CN115841477A (en) * | 2022-12-15 | 2023-03-24 | 北京交通大学 | Multi-mode visual fusion ballastless track structure hidden defect detection method and system |
| CN117036641A (en) * | 2023-06-08 | 2023-11-10 | 四川轻化工大学 | Road scene three-dimensional reconstruction and defect detection method based on binocular vision |
| CN116958146A (en) * | 2023-09-20 | 2023-10-27 | 深圳市信润富联数字科技有限公司 | Acquisition method and device of 3D point cloud and electronic device |
Non-Patent Citations (1)
| Title |
|---|
| 基于机器视觉的高压电缆开线缺陷检测研究;刘城;《中国优秀硕士学位论文全文数据库 (工程科技Ⅱ辑)》(第8期);第1-72页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN117291918A (en) | 2023-12-26 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108555908B (en) | Stacked workpiece posture recognition and pickup method based on RGBD camera | |
| CN105066892B (en) | A kind of BGA element testings and localization method based on straight line clustering | |
| CN110675376A (en) | PCB defect detection method based on template matching | |
| CN107590498A (en) | A kind of self-adapted car instrument detecting method based on Character segmentation level di- grader | |
| CN105957082A (en) | Printing quality on-line monitoring method based on area-array camera | |
| CN110146017B (en) | Industrial robot repeated positioning precision measuring method | |
| CN112037219A (en) | Metal surface defect detection method based on two-stage convolution neural network | |
| CN114581782B (en) | Fine defect detection method based on coarse-to-fine detection strategy | |
| CN113222982A (en) | Wafer surface defect detection method and system based on improved YOLO network | |
| CN111127417B (en) | Printing defect detection method based on SIFT feature matching and SSD algorithm improvement | |
| CN115496746A (en) | Method and system for detecting surface defects of plate based on fusion of image and point cloud data | |
| CN115330734A (en) | Automatic robot repair welding system based on three-dimensional target detection and point cloud defect completion | |
| CN114998308A (en) | Defect detection method and system based on photometric stereo | |
| CN115775236A (en) | Surface tiny defect visual detection method and system based on multi-scale feature fusion | |
| CN108205210B (en) | LCD defect detection system and method based on Fourier mellin and feature matching | |
| KR101782364B1 (en) | Vision inspection method based on learning data | |
| CN113706496B (en) | Aircraft structure crack detection method based on deep learning model | |
| CN117291918B (en) | Automobile stamping part defect detection method based on three-dimensional point cloud | |
| CN112070811A (en) | Image registration method based on continuous domain ant colony algorithm improvement | |
| CN116843618A (en) | Method for detecting shallow apparent appearance defects of metal parts | |
| CN114187269B (en) | Rapid detection method for surface defect edge of small component | |
| CN116309817A (en) | Tray detection and positioning method based on RGB-D camera | |
| CN113591548B (en) | Target ring identification method and system | |
| CN111415384A (en) | Industrial image component accurate positioning system based on deep learning | |
| CN116843615B (en) | Lead frame intelligent total inspection method based on flexible light path |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |