CN113674236A - Airplane surface multi-circular-hole detection method based on feature learning - Google Patents
Airplane surface multi-circular-hole detection method based on feature learning Download PDFInfo
- Publication number
- CN113674236A CN113674236A CN202110934836.9A CN202110934836A CN113674236A CN 113674236 A CN113674236 A CN 113674236A CN 202110934836 A CN202110934836 A CN 202110934836A CN 113674236 A CN113674236 A CN 113674236A
- Authority
- CN
- China
- Prior art keywords
- round hole
- point cloud
- boundary
- round
- holes
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0004—Industrial image inspection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/23—Clustering techniques
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/24—Classification techniques
- G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
- G06F18/2415—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on parametric or probabilistic models, e.g. based on likelihood ratio or false acceptance rate versus a false rejection rate
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/047—Probabilistic or stochastic networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/60—Analysis of geometric attributes
- G06T7/66—Analysis of geometric attributes of image moments or centre of gravity
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30108—Industrial image inspection
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- General Physics & Mathematics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Computing Systems (AREA)
- Biomedical Technology (AREA)
- General Health & Medical Sciences (AREA)
- Computational Linguistics (AREA)
- Biophysics (AREA)
- Mathematical Physics (AREA)
- Software Systems (AREA)
- Molecular Biology (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Health & Medical Sciences (AREA)
- Probability & Statistics with Applications (AREA)
- Quality & Reliability (AREA)
- Geometry (AREA)
- Length Measuring Devices With Unspecified Measuring Means (AREA)
Abstract
The invention discloses a method for detecting multiple circular holes on the surface of an airplane based on feature learning, and relates to the technical field of aviation manufacturing. The method includes inputting point cloud data of round holes into a three-dimensional point cloud feature learning network to extract all round hole boundary feature points, classifying all the extracted round hole boundary feature points onto the round holes, learning fitting weights of normal directions and boundary feature points of corresponding round holes according to classification results and the point cloud data, and calculating parameters of each round hole by a learning weighted least square method based on the normal directions and the fitting weights obtained through learning. The method for detecting the multiple circular holes on the surface of the airplane can improve the accuracy and stability of circular hole detection and can effectively solve the problem of detecting the multiple circular holes on the surface of the airplane.
Description
Technical Field
The invention relates to the technical field of aviation manufacturing, in particular to a method for detecting multiple circular holes on the surface of an airplane based on feature learning.
Background
In the field of aviation manufacturing, the main mode of assembly connection during modern aircraft assembly is rivet connection, and the precision of a rivet hole determines the quality of riveting, so that the manufacturing quality and the service life of an aircraft are influenced. The plane surface circular hole detection technology is an important guarantee for the plane automatic hole making quality. The traditional contact detection is difficult to meet the requirement of automation due to the problems of easy ring hole surface breakage, low efficiency and the like. The non-contact two-dimensional vision method has limitations in automation and precision, and detection of circular hole parameters is incomplete. The complete structure of the surface of the round hole can be obtained through three-dimensional measurement, but due to the fact that the surface of an airplane is rough, reflected, shielded or limited in measurement range, the obtained three-dimensional point cloud data of the round hole often has certain defects, and it is very difficult to directly detect all round hole structures. In order to solve the problems, the invention provides a method for detecting multiple circular holes on the surface of an airplane based on feature learning, which effectively solves the problems of low detection precision, low efficiency, low stability, incomplete detection parameters and the like of the multiple circular holes on the surface of the airplane and reduces the detection difficulty of the multiple circular holes on the surface of the airplane.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a method for detecting multiple circular holes on the surface of an airplane based on feature learning. According to the method for detecting the multiple round holes in the surface of the airplane, the point cloud data of the multiple round holes in the surface of the airplane is obtained, and the round hole parameters are calculated based on a weighted least square method of learning weight, so that the accuracy and stability of round hole detection are improved, the problem of detecting the multiple round holes in the surface of the airplane is effectively solved, the automatic detection of the round holes in the surface of the airplane is realized, and the airplane assembly quality is improved.
In order to achieve the purpose, the invention adopts the following technical scheme: a method for detecting multiple circular holes on the surface of an airplane based on feature learning comprises the following steps:
(1) scanning to obtain point cloud data of all round holes on the surface of the airplane;
(2) inputting point cloud data into a three-dimensional point cloud feature learning network to extract all round hole boundary feature points, classifying all the extracted round hole boundary feature points onto round holes, and learning the normal direction of the corresponding round holes and the fitting weight of the boundary feature points according to the classification result and the point cloud data;
(3) calculating corresponding round hole parameters of the normal direction and the fitting weight of the round hole obtained by learning by adopting a least square method, training a three-dimensional point cloud characteristic learning network by taking the difference between the calculated round hole parameters and the real round hole parameters as a loss function until the loss function is converged, and finishing the training of the three-dimensional point cloud characteristic learning network;
(4) and scanning the point cloud data of the round holes on the surface of the airplane again, inputting the point cloud data into the trained three-dimensional point cloud characteristic learning network, and outputting parameters corresponding to the round holes.
Further, step (1) comprises the following sub-steps:
(1.1) fixing the scanner on a mechanical arm, and adjusting the pose of the scanner by using the mechanical arm to enable the distance from the surface of the circular hole to the scanner to meet the parameter requirement of the scanner;
(1.2) controlling the mechanical arm to move the scanner to enable the round holes on the surface of the airplane to be located in the field of view of the scanner, and then collecting point cloud data of all round holes in the field of view of the scanner;
and (1.3) repeatedly adjusting the visual field range of the scanner through the mechanical arm until point cloud data of all round holes on the surface of the airplane are obtained.
Further, the three-dimensional point cloud feature learning network comprises: the sensor comprises a first graph volume layer, a first multilayer sensor, a first maximum pooling layer, a second graph volume layer, a second multilayer sensor, a second maximum pooling layer, a Transformer, a first full-link layer, a second full-link layer, a third maximum pooling layer and a third full-link layer, wherein the first graph volume layer, the first multilayer sensor and the first maximum pooling layer are sequentially connected, the second graph volume layer, the second multilayer sensor and the second maximum pooling layer are sequentially connected, the first maximum pooling layer and the second maximum pooling layer are both connected with the Transformer, the Transformer is respectively connected with the first full-link layer, the second full-link layer and the third full-link layer, and the second full-link layer and the third maximum pooling layer are connected.
Further, the specific process of extracting the boundary feature points of all round holes to be detected in the step (2) is as follows: the method comprises the steps of inquiring scanned point cloud data by using two spheres with different radiuses to obtain a local neighborhood point set and a global neighborhood point set of each round hole point cloud data, obtaining local features of each round hole point cloud data by passing the local neighborhood point set through a first graph volume layer, a first multilayer sensor and a first maximum pooling layer, obtaining global features of each round hole point cloud data by passing the global neighborhood point set through a second graph volume layer, a second multilayer sensor and a second maximum pooling layer, obtaining fusion features by fusing the local features and the global features of each round hole point cloud data through a Transformer, reducing the fusion features of each round hole point cloud data to two dimensions through a first full connection layer, calculating by using a softmax function to obtain the probability that each round hole data is a round hole boundary common point and a round hole boundary feature point, and if the probability of the round hole boundary feature point is greater than that of the round hole boundary common point, and taking the round hole point cloud data as round hole boundary feature points, and forming a round hole boundary point set by the round hole boundary feature points.
Further, the radius of the global neighborhood point set queried by using a sphere is 2 times of the radius of the maximum circular hole, and the radius of the local neighborhood point set queried by using a sphere is 1/4 times of the radius of the maximum circular hole.
Further, the method for classifying the boundary feature points of the circular holes comprises the following steps: a classification method, a RANSAC method or a clustering method based on the position relation of the circular holes.
Further, the normal learning method of each round hole specifically comprises the following steps: and inputting the fusion features corresponding to the circular hole boundary feature points classified to the circular holes into the second full-connection layer and the third maximum pooling layer, outputting three-dimensional data, and unitizing the output three-dimensional data to obtain the normal direction of the circular holes.
Further, the learning method of the fitting weight of the boundary feature point of each round hole specifically includes: and inputting the fusion features corresponding to the circular hole boundary feature points classified to the circular holes into the third full-connection layer, and obtaining the fitting weight of the boundary feature points through a softmax function.
Further, the calculation method of each round hole parameter specifically comprises the following steps: determining a projection plane according to the normal direction of the round hole and the three-dimensional coordinate mean value of the boundary characteristic point in the three-dimensional space point cloud coordinate system, establishing a two-dimensional coordinate system, projecting the boundary characteristic point of the round hole onto the two-dimensional coordinate system to obtain a two-dimensional boundary characteristic point, fitting the two-dimensional boundary characteristic point into the round hole by using the fitting weight of the round hole boundary characteristic point through a weighted least square method to obtain the parameter of the round hole, wherein the parameter of the round hole comprises a radius and a circle center, and rotating the circle center into the three-dimensional space point cloud coordinate system to obtain the three-dimensional coordinate of the circle center.
Compared with the prior art, the invention has the following beneficial effects: the invention provides a method for detecting multiple circular holes on the surface of an airplane based on circular hole boundary point detection of feature learning and a weighted least square fitting technology based on learning. According to the method, the point cloud data of the round holes on the surface of the airplane are obtained, the round hole boundary points are extracted through a feature learning round hole boundary point detection method, and then the normal direction and the boundary feature point fitting weight of the round holes are learned, so that the problem of multi-round hole detection on the surface of the airplane is solved, the automatic detection of the round holes on the surface of the airplane is realized, meanwhile, the accuracy and the stability of the round hole detection are improved, and the airplane assembly quality is improved.
Drawings
FIG. 1 is a flow chart of a method for detecting multiple circular holes on the surface of an aircraft based on feature learning according to the present invention;
FIG. 2 is a schematic view of a circular hole feature point scan according to the present invention;
FIG. 3 is a schematic structural diagram of a three-dimensional point cloud feature learning network according to the present invention;
FIG. 4 is a schematic diagram of a circular hole detected by the method for detecting multiple circular holes on the surface of an airplane.
Detailed Description
The technical solution of the present invention is further described in detail below with reference to the accompanying drawings.
Fig. 1 is a flowchart of a method for detecting multiple circular holes on an aircraft surface based on feature learning according to the present invention, and the method for detecting multiple circular holes on an aircraft surface includes the following steps:
(1) because the requirement on the detection precision of the round holes on the surface of the airplane is high, if the field of view of the scanner is large, the scanning resolution is reduced, the large field of view cannot be selected during scanning, and the point cloud data of all round holes cannot be acquired at one time. Therefore, the scanning method for acquiring the point cloud data of all round holes on the surface of the airplane specifically comprises the following substeps:
(1.1) fixing the scanner on a mechanical arm, and adjusting the pose of the scanner by using the mechanical arm to enable the distance from the surface of the circular hole to the scanner to meet the parameter requirement of the scanner;
(1.2) controlling the mechanical arm to move the scanner to enable the round holes on the surface of the airplane to be located in the field of view of the scanner, and then collecting point cloud data of all round holes in the field of view of the scanner;
and (1.3) repeatedly adjusting the visual field range of the scanner through the mechanical arm until point cloud data of all round holes on the surface of the airplane are obtained.
(2) As shown in fig. 2, which is a schematic diagram of scanning of the feature points of the circular hole in the invention, because the inner wall points of the circular hole of the actually scanned point cloud data are incomplete and have certain randomness, which may cause incomplete neighborhood of the points of the circular hole, thereby increasing the difficulty of extracting the boundary feature points, the invention extracts all the boundary feature points of the circular hole by inputting the point cloud data into the three-dimensional point cloud feature learning network, classifies all the extracted boundary feature points of the circular hole onto the circular hole, learns the normal direction of the corresponding circular hole and the fitting weight of the boundary feature points according to the classification result and the point cloud data, and improves the stability and accuracy of extracting the boundary feature points of the circular hole by detecting whether each point cloud data is the boundary feature point.
Fig. 3 is a schematic diagram of a three-dimensional point cloud feature learning network structure related in the present invention, the three-dimensional point cloud feature learning network structure includes: the sensor comprises a first graph volume layer, a first multilayer sensor, a first maximum pooling layer, a second graph volume layer, a second multilayer sensor, a second maximum pooling layer, a Transformer, a first full-link layer, a second full-link layer, a third maximum pooling layer and a third full-link layer, wherein the first graph volume layer, the first multilayer sensor and the first maximum pooling layer are sequentially connected, the second graph volume layer, the second multilayer sensor and the second maximum pooling layer are sequentially connected, the first maximum pooling layer and the second maximum pooling layer are both connected with the Transformer, the Transformer is respectively connected with the first full-link layer, the second full-link layer and the third full-link layer, and the second full-link layer and the third maximum pooling layer are connected.
Specifically, step (2) includes the following substeps:
(2.1) using two spheres with different radiuses to query the scanned point cloud data to obtain a local neighborhood point set and a global neighborhood point set of each round hole point cloud data, wherein the radius of the global neighborhood point set queried by the spheres is 2 times of the radius of the maximum round hole, and the 2 times of radius of the maximum round hole can cover the whole round hole, so that the boundary points obtain the global structure information of the whole round hole, and whether the point cloud data belong to the round hole is judged; the radius of the local neighborhood point set is 1/4 of the maximum round hole radius, and the maximum round hole radius of 1/4 can only cover the local area of the boundary of a single round hole, so that local structure information is obtained, and whether the point cloud data of the broken round hole is located on the boundary of the round hole or not is accurately judged; by two ball query methods with different radiuses, boundary points which only belong to the round hole can be extracted. The method comprises the steps of obtaining local features of each round hole point cloud data for a local area point set through a first graph convolution layer, a first multilayer sensor and a first maximum pooling layer, obtaining global features of each round hole point cloud data for a global neighborhood point set through a second graph convolution layer, a second multilayer sensor and a second maximum pooling layer, obtaining fusion features by fusing the local features and the global features of each round hole point cloud data through a Transformer, reducing the fusion features of each round hole point cloud data to two dimensions through a first full connection layer, calculating the probability that each round hole point cloud data is a round hole boundary common point and a round hole boundary feature point by utilizing a softmax function, and if the probability of the round hole boundary feature point is larger than that of the round hole boundary common point, using the round hole point cloud data as the round hole boundary feature point to form a round hole boundary point set.
(2.2) classifying all extracted boundary feature points of the round holes onto the round holes according to the distribution structure of the round holes on the surface of the airplane, wherein the classification method of the boundary feature points of the round holes adopted in the invention comprises the following steps: a classification method, a RANSAC method or a clustering method based on the position relation of the circular holes.
And (2.3) inputting the fusion features corresponding to the circular hole boundary feature points classified to the circular holes into the second full-connection layer and the third maximum pooling layer, outputting three-dimensional data, and unitizing the output three-dimensional data to obtain the normal direction of the circular holes. Through learning the normal direction, combine round hole central point to confirm a plane, with the boundary point projection of round hole to this plane on fitting, turn into the two-dimensional circle fitting problem with three-dimensional circle fitting, reduced the degree of difficulty of circle fitting.
And (2.4) inputting the fusion features corresponding to the circular hole boundary feature points classified on the circular holes into the third full-connection layer, and obtaining the fitting weight of the boundary feature points through a softmax function. Round hole boundary point fitting round hole radius is different from the conventional round fitting of some given points, and round hole boundary points are mainly distributed on the outer side of a round hole boundary circle, the optimal result of the conventional round fitting is not the radius of the round hole, and the weight acquired by the learning method is combined with the structural information of the round hole boundary, so that key points in the boundary points can be acquired and higher weight is given, and the fitting result has higher precision relative to the real aperture.
(3) And calculating corresponding round hole parameters by adopting a least square method according to the normal direction and the fitting weight of the round hole obtained by learning, training a three-dimensional point cloud characteristic learning network by taking the difference between the calculated round hole parameters and the real round hole parameters as a loss function until the loss function is converged, and finishing the training of the three-dimensional point cloud characteristic learning network. The calculation method of each round hole parameter in the invention specifically comprises the following steps: determining a projection plane according to the normal direction of a round hole and the three-dimensional coordinate mean value of a boundary characteristic point in a three-dimensional space point cloud coordinate system, establishing a two-dimensional coordinate system, projecting the boundary characteristic point of the round hole onto the two-dimensional coordinate system to obtain a two-dimensional boundary characteristic point, fitting the two-dimensional boundary characteristic point into the round hole by using the fitting weight of the round hole boundary characteristic point through a weighted least square method to obtain the parameter of the round hole, wherein the parameter of the round hole comprises a radius and a circle center, and the circle center needs to be rotated back into the three-dimensional space point cloud coordinate system to obtain the three-dimensional coordinate of the circle center because the circle center is the two-dimensional point on the projection plane.
(4) And scanning the point cloud data of the round holes on the surface of the airplane again, inputting the point cloud data into the trained three-dimensional point cloud characteristic learning network, and outputting parameters corresponding to the round holes.
As shown in fig. 4, which is a schematic diagram of detecting circular holes by using the method for detecting multiple circular holes on the surface of an airplane, the method for detecting multiple circular holes on the surface of an airplane accurately extracts all circular hole boundary points from real scanning data of the multiple circular holes on the surface of the airplane and accurately and stably fits circular hole parameters. The method for detecting the multiple round holes on the surface of the airplane has the following advantages that the precision of the obtained round holes is compared with that of the obtained round holes by other methods: the radius error of the round hole detected by the method is 0.016643 mm, and the variance of the radius error is 0.000384 mm2Other methods include: the method comprises an ultra method, a linear least square method, a random sampling consistency algorithm, a repeated least square method and commercial detection software PolyWorks, wherein the measured radius errors are 0.027201, 0.028264, 0.052397, 0.176154 and 0.0267 respectively, and the variances of the radius errors are 0.001113, 0.001166, 0.005844, 0.077724 and 0.001045 respectively, so that the method for detecting the multiple round holes on the surface of the airplane can detect the round hole parameters most accurately and stably.
The above is only a preferred embodiment of the present invention, and the scope of the present invention is not limited to the above embodiment, and any technical solutions that fall under the spirit of the present invention fall within the scope of the present invention. It should be noted that modifications and embellishments within the scope of the invention may be made by those skilled in the art without departing from the principle of the invention.
Claims (9)
1. A method for detecting multiple circular holes on the surface of an airplane based on feature learning is characterized by comprising the following steps:
(1) scanning to obtain point cloud data of all round holes on the surface of the airplane;
(2) inputting point cloud data into a three-dimensional point cloud feature learning network to extract all round hole boundary feature points, classifying all the extracted round hole boundary feature points onto round holes, and learning the normal direction of the corresponding round holes and the fitting weight of the boundary feature points according to the classification result and the point cloud data;
(3) calculating corresponding round hole parameters of the normal direction and the fitting weight of the round hole obtained by learning by adopting a least square method, training a three-dimensional point cloud characteristic learning network by taking the difference between the calculated round hole parameters and the real round hole parameters as a loss function until the loss function is converged, and finishing the training of the three-dimensional point cloud characteristic learning network;
(4) and scanning the point cloud data of the round holes on the surface of the airplane again, inputting the point cloud data into the trained three-dimensional point cloud characteristic learning network, and outputting parameters corresponding to the round holes.
2. The method for detecting multiple circular holes on the surface of an aircraft based on feature learning as claimed in claim 1, wherein the step (1) comprises the following sub-steps:
(1.1) fixing the scanner on a mechanical arm, and adjusting the pose of the scanner by using the mechanical arm to enable the distance from the surface of the circular hole to the scanner to meet the parameter requirement of the scanner;
(1.2) controlling the mechanical arm to move the scanner to enable the round holes on the surface of the airplane to be located in the field of view of the scanner, and then collecting point cloud data of all round holes in the field of view of the scanner;
and (1.3) repeatedly adjusting the visual field range of the scanner through the mechanical arm until point cloud data of all round holes on the surface of the airplane are obtained.
3. The method for detecting multiple circular holes on the surface of an airplane based on feature learning of claim 1, wherein the three-dimensional point cloud feature learning network comprises: the sensor comprises a first graph volume layer, a first multilayer sensor, a first maximum pooling layer, a second graph volume layer, a second multilayer sensor, a second maximum pooling layer, a Transformer, a first full-link layer, a second full-link layer, a third maximum pooling layer and a third full-link layer, wherein the first graph volume layer, the first multilayer sensor and the first maximum pooling layer are sequentially connected, the second graph volume layer, the second multilayer sensor and the second maximum pooling layer are sequentially connected, the first maximum pooling layer and the second maximum pooling layer are both connected with the Transformer, the Transformer is respectively connected with the first full-link layer, the second full-link layer and the third full-link layer, and the second full-link layer and the third maximum pooling layer are connected.
4. The method for detecting the multiple circular holes on the surface of the airplane based on the feature learning as claimed in claim 1, wherein the specific process of extracting the boundary feature points of all the circular holes to be detected in the step (2) is as follows: the method comprises the steps of inquiring scanned point cloud data by using two spheres with different radiuses to obtain a local neighborhood point set and a global neighborhood point set of each round hole point cloud data, obtaining local features of each round hole point cloud data by passing the local neighborhood point set through a first graph volume layer, a first multilayer sensor and a first maximum pooling layer, obtaining global features of each round hole point cloud data by passing the global neighborhood point set through a second graph volume layer, a second multilayer sensor and a second maximum pooling layer, obtaining fusion features by fusing the local features and the global features of each round hole point cloud data through a Transformer, reducing the fusion features of each round hole point cloud data to two dimensions through a first full connection layer, calculating by using a softmax function to obtain the probability that each round hole data is a round hole boundary common point and a round hole boundary feature point, and if the probability of the round hole boundary feature point is greater than that of the round hole boundary common point, and taking the round hole point cloud data as round hole boundary feature points, and forming a round hole boundary point set by the round hole boundary feature points.
5. The method for detecting the multiple circular holes on the surface of the airplane based on the feature learning as claimed in claim 4, wherein the radius of the global neighborhood point set queried by a sphere is 2 times of the radius of the maximum circular hole, and the radius of the local neighborhood point set queried by the sphere is 1/4 times of the radius of the maximum circular hole.
6. The method for detecting multiple circular holes in the surface of an airplane based on feature learning as claimed in claim 1, wherein the method for classifying the feature points on the boundaries of the circular holes comprises the following steps: a classification method, a RANSAC method or a clustering method based on the position relation of the circular holes.
7. The method for detecting multiple circular holes in the surface of an aircraft based on feature learning as claimed in claim 1, wherein the normal learning method for each circular hole is specifically as follows: and inputting the fusion features corresponding to the circular hole boundary feature points classified to the circular holes into the second full-connection layer and the third maximum pooling layer, outputting three-dimensional data, and unitizing the output three-dimensional data to obtain the normal direction of the circular holes.
8. The method for detecting multiple circular holes in the surface of an airplane based on feature learning according to claim 1, wherein the learning method of the fitting weight of the boundary feature point of each circular hole is specifically as follows: and inputting the fusion features corresponding to the circular hole boundary feature points classified to the circular holes into the third full-connection layer, and obtaining the fitting weight of the boundary feature points through a softmax function.
9. The method for detecting multiple circular holes in the surface of an aircraft based on feature learning as claimed in claim 1, wherein the method comprises the following steps: the calculation method of each round hole parameter specifically comprises the following steps: determining a projection plane according to the normal direction of the round hole and the three-dimensional coordinate mean value of the boundary characteristic point in the three-dimensional space point cloud coordinate system, establishing a two-dimensional coordinate system, projecting the boundary characteristic point of the round hole onto the two-dimensional coordinate system to obtain a two-dimensional boundary characteristic point, fitting the two-dimensional boundary characteristic point into the round hole by using the fitting weight of the round hole boundary characteristic point through a weighted least square method to obtain the parameter of the round hole, wherein the parameter of the round hole comprises a radius and a circle center, and rotating the circle center into the three-dimensional space point cloud coordinate system to obtain the three-dimensional coordinate of the circle center.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110934836.9A CN113674236A (en) | 2021-08-16 | 2021-08-16 | Airplane surface multi-circular-hole detection method based on feature learning |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202110934836.9A CN113674236A (en) | 2021-08-16 | 2021-08-16 | Airplane surface multi-circular-hole detection method based on feature learning |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113674236A true CN113674236A (en) | 2021-11-19 |
Family
ID=78542910
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202110934836.9A Pending CN113674236A (en) | 2021-08-16 | 2021-08-16 | Airplane surface multi-circular-hole detection method based on feature learning |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113674236A (en) |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114881955A (en) * | 2022-04-28 | 2022-08-09 | 厦门微亚智能科技有限公司 | Slice-based annular point cloud defect extraction method and device and equipment storage medium |
| CN115049842A (en) * | 2022-06-16 | 2022-09-13 | 南京航空航天大学深圳研究院 | Aircraft skin image damage detection and 2D-3D positioning method |
| CN115953589A (en) * | 2023-03-13 | 2023-04-11 | 南京航空航天大学 | Engine cylinder block aperture size measuring method based on depth camera |
| CN116011107A (en) * | 2023-01-10 | 2023-04-25 | 南京航空航天大学 | Method, device and system for extracting hole characteristics of large composite material component |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003247954A (en) * | 2002-02-26 | 2003-09-05 | Sumitomo Metal Mining Co Ltd | Defect detection method for round body circumference |
| CN106091923A (en) * | 2016-05-30 | 2016-11-09 | 武汉理工大学 | The central point rapid assay methods of industrial bolt circular hole based on three-dimensional laser scanning technique |
| CN110986778A (en) * | 2019-12-25 | 2020-04-10 | 电子科技大学 | System and method for detecting size of rivet hole of carbon fiber composite material |
| CN111681206A (en) * | 2020-05-08 | 2020-09-18 | 上海工程技术大学 | Method for detecting size of special-shaped hole of spinneret plate |
| CN111912335A (en) * | 2020-06-30 | 2020-11-10 | 成都飞机工业（集团）有限责任公司 | Airplane surface datum hole identification method suitable for robot drilling and riveting system |
-
2021
- 2021-08-16 CN CN202110934836.9A patent/CN113674236A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2003247954A (en) * | 2002-02-26 | 2003-09-05 | Sumitomo Metal Mining Co Ltd | Defect detection method for round body circumference |
| CN106091923A (en) * | 2016-05-30 | 2016-11-09 | 武汉理工大学 | The central point rapid assay methods of industrial bolt circular hole based on three-dimensional laser scanning technique |
| CN110986778A (en) * | 2019-12-25 | 2020-04-10 | 电子科技大学 | System and method for detecting size of rivet hole of carbon fiber composite material |
| CN111681206A (en) * | 2020-05-08 | 2020-09-18 | 上海工程技术大学 | Method for detecting size of special-shaped hole of spinneret plate |
| CN111912335A (en) * | 2020-06-30 | 2020-11-10 | 成都飞机工业（集团）有限责任公司 | Airplane surface datum hole identification method suitable for robot drilling and riveting system |
Non-Patent Citations (1)
| Title |
|---|
| 杨爽: "《基于深度学习的复合材料圆孔检测方法研究》", 《CNKI硕士学位论文》 * |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114881955A (en) * | 2022-04-28 | 2022-08-09 | 厦门微亚智能科技有限公司 | Slice-based annular point cloud defect extraction method and device and equipment storage medium |
| CN114881955B (en) * | 2022-04-28 | 2023-05-12 | 厦门微亚智能科技有限公司 | Annular point cloud defect extraction method, device and equipment storage medium based on slice |
| CN115049842A (en) * | 2022-06-16 | 2022-09-13 | 南京航空航天大学深圳研究院 | Aircraft skin image damage detection and 2D-3D positioning method |
| CN115049842B (en) * | 2022-06-16 | 2023-11-17 | 南京航空航天大学深圳研究院 | Method for detecting damage of aircraft skin image and positioning 2D-3D |
| CN116011107A (en) * | 2023-01-10 | 2023-04-25 | 南京航空航天大学 | Method, device and system for extracting hole characteristics of large composite material component |
| CN115953589A (en) * | 2023-03-13 | 2023-04-11 | 南京航空航天大学 | Engine cylinder block aperture size measuring method based on depth camera |
| CN115953589B (en) * | 2023-03-13 | 2023-05-16 | 南京航空航天大学 | Engine cylinder block aperture size measurement method based on depth camera |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113674236A (en) | Airplane surface multi-circular-hole detection method based on feature learning | |
| CN108509918B (en) | Target detection and tracking method fusing laser point cloud and image | |
| CN109444911B (en) | Unmanned ship water surface target detection, identification and positioning method based on monocular camera and laser radar information fusion | |
| CN109598765B (en) | Monocular camera and millimeter wave radar external parameter combined calibration method based on spherical calibration object | |
| CN109300162A (en) | A kind of multi-line laser radar and camera combined calibrating method based on fining radar scanning marginal point | |
| CN110533722A (en) | A kind of the robot fast relocation method and system of view-based access control model dictionary | |
| CN102494663B (en) | Measuring system of swing angle of swing nozzle and measuring method of swing angle | |
| CN114626470B (en) | Aircraft skin key feature detection method based on multi-type geometric feature operator | |
| CN111476286A (en) | Map construction method for mobile robot | |
| JP6756889B1 (en) | Vortex detector, vortex detection method, program and trained model | |
| KR102228939B1 (en) | System and method for automatic 3-dimensional scanning | |
| US20230104047A1 (en) | Track fusion method and device for unmanned surface vehicle | |
| Pan et al. | Automatic recognition and location system for electric vehicle charging port in complex environment | |
| CN110186375A (en) | Intelligent high-speed rail white body assemble welding feature detection device and detection method | |
| CN109270079B (en) | Workpiece surface defect accurate detection method based on point cloud model | |
| Zhao et al. | Vision-based adaptive stereo measurement of pins on multi-type electrical connectors | |
| CN115690746A (en) | Non-blind area sensing method and system based on vehicle-road cooperation | |
| CN112734619B (en) | Free-form surface coverage viewpoint automatic sampling method for detecting feature scanning uncertainty | |
| CN113295142B (en) | Terrain scanning analysis method and device based on FARO scanner and point cloud | |
| Yuan et al. | DSmT-based ultrasonic detection model for estimating indoor environment contour | |
| CN110310322A (en) | Method for detecting assembly surface of 10-micron-level high-precision device | |
| CN117067226A (en) | Steel bridge rust detection robot and rust detection method | |
| CN117292181A (en) | Sheet metal part hole group classification and full-size measurement method based on 3D point cloud processing | |
| CN110554405A (en) | normal scanning registration method and system based on cluster combination | |
| CN115457130A (en) | Electric vehicle charging port detection and positioning method based on depth key point regression |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| CB03 | Change of inventor or designer information | ||
| CB03 | Change of inventor or designer information |
Inventor after: Wang Jun Inventor after: Wei Zeyong Inventor after: Chen Honghua Inventor after: Yi Cheng Inventor before: Wang Jun Inventor before: Wei Zeyong Inventor before: Chen Honghua |
|
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20211119 |