CN116476070B - Method for adjusting scanning measurement path of large-scale barrel part local characteristic robot - Google Patents
Method for adjusting scanning measurement path of large-scale barrel part local characteristic robot Download PDFInfo
- Publication number
- CN116476070B CN116476070B CN202310577100.XA CN202310577100A CN116476070B CN 116476070 B CN116476070 B CN 116476070B CN 202310577100 A CN202310577100 A CN 202310577100A CN 116476070 B CN116476070 B CN 116476070B
- Authority
- CN
- China
- Prior art keywords
- point cloud
- obtaining
- pose
- robot
- theoretical
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
- 238000005259 measurement Methods 0.000 title claims abstract description 74
- 238000000034 method Methods 0.000 title claims abstract description 53
- 230000000295 complement effect Effects 0.000 claims description 34
- 238000006243 chemical reaction Methods 0.000 claims description 28
- 239000011159 matrix material Substances 0.000 claims description 28
- 230000008569 process Effects 0.000 claims description 18
- 238000000605 extraction Methods 0.000 claims description 13
- 238000004364 calculation method Methods 0.000 claims description 12
- 238000011176 pooling Methods 0.000 claims description 11
- 230000009466 transformation Effects 0.000 claims description 8
- 238000004088 simulation Methods 0.000 claims description 7
- 238000004422 calculation algorithm Methods 0.000 claims description 6
- 238000013135 deep learning Methods 0.000 claims description 6
- 238000005070 sampling Methods 0.000 claims description 5
- 230000000007 visual effect Effects 0.000 description 7
- 230000011218 segmentation Effects 0.000 description 6
- 238000004519 manufacturing process Methods 0.000 description 4
- 238000012545 processing Methods 0.000 description 4
- 238000009826 distribution Methods 0.000 description 2
- 230000000694 effects Effects 0.000 description 2
- 238000012935 Averaging Methods 0.000 description 1
- 238000010923 batch production Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000012512 characterization method Methods 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000010586 diagram Methods 0.000 description 1
- 238000006073 displacement reaction Methods 0.000 description 1
- 239000012636 effector Substances 0.000 description 1
- 230000006870 function Effects 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 238000012795 verification Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
- G01B11/00—Measuring arrangements characterised by the use of optical techniques
- G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J11/00—Manipulators not otherwise provided for
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1628—Programme controls characterised by the control loop
- B25J9/163—Programme controls characterised by the control loop learning, adaptive, model based, rule based expert control
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1679—Programme controls characterised by the tasks executed
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
- Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
- Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]
Landscapes
- Engineering & Computer Science (AREA)
- Robotics (AREA)
- Mechanical Engineering (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Manipulator (AREA)
- Image Processing (AREA)
Abstract
The invention discloses a method for adjusting a scanning measurement path of a large-scale barrel part local characteristic robot, which comprises the following steps: constructing a large-scale barrel robot scanning measurement field, obtaining a large-scale barrel theoretical pose based on a theoretical model, obtaining a theoretical scanning measurement path based on the large-scale barrel theoretical pose, and obtaining three-dimensional point cloud data of the large-scale barrel based on the theoretical scanning measurement path by the robot; judging the integrity of local characteristic point clouds of the large cylinder based on the three-dimensional point cloud data of the large cylinder to obtain a judging result; and obtaining a robot motion program based on the judgment result, and obtaining a final robot scanning measurement path based on the robot motion program. The invention can effectively ensure that the integrity of the point cloud of the measuring result can be ensured under the condition that the actual pose of the large cylinder deviates from the theoretical pose.
Description
Technical Field
The invention relates to the technical field of automatic three-dimensional scanning measurement of large-scale components, in particular to a method for adjusting a scanning measurement path of a large-scale barrel part local characteristic robot.
Background
The large-scale cylinder is a product part which is widely applied in the aerospace field, the local characteristics of the large-scale cylinder are key parts which are distributed on the surface of the large-scale cylinder and are used for being assembled with external equipment, such as an assembly interface, a payload support and the like, the local characteristics have the characteristic of discrete distribution, and the production and the manufacture of the large-scale cylinder are difficult in consideration of the large-scale characteristics of the large-scale cylinder. In order to meet the manufacturing process requirements of the local characteristics of the large-sized barrel, an efficient and accurate large-size measuring means for the local characteristics of the large-sized barrel is necessary.
The three-dimensional scanning measurement refers to a measurement means for acquiring the surface morphology of a workpiece through laser or structured light, and accurate pose and allowance information can be provided for workpiece processing through a three-dimensional point cloud. The robot scanning measurement refers to clamping three-dimensional scanning measurement equipment at the tail end of a robot, and the robot is used for assisting in realizing an automatic measurement method of scanning in a large range, has the characteristics of high measurement efficiency and capability of acquiring the surface information of a complete workpiece, and can effectively meet the measurement requirement of local characteristics of a large cylinder. However, the current path planning about robot scanning measurement is basically based on a theoretical CAD model, and when a certain deviation occurs between the pose of an actual workpiece and the pose of the theoretical CAD model, a situation that a local area cannot be scanned is caused, which causes the acquired point cloud to be missing, and affects the integrity of the scanning result. For the scan measurement of the local features of the large-scale barrel, the main factors causing the point cloud missing can be summarized as the following points: (1) The clamping process of the large cylinder part is mostly manually operated, and the initial pose of the large cylinder part has larger uncertainty; (2) The size of the local feature is smaller than the whole size of the large barrel, and the small pose deviation of the large barrel can lead to larger displacement of the feature; (3) The local features are composed of holes, planes and regular curved surfaces, the structure is relatively complex, and the pose deviation can increase the measurement difficulty.
Therefore, in the large-scale member robot scan measurement, the scan measurement path planned based on the theoretical model cannot be adapted to the actual state of the large-scale barrel.
Disclosure of Invention
The invention provides a method for adjusting a scanning measurement path of a large-scale barrel part local characteristic robot, which aims at solving the problem that a measurement path planned based on a workpiece theoretical model cannot effectively adapt to the actual state of a workpiece and further causes incomplete scanning results, namely point cloud loss in the scanning measurement process of the large-scale barrel part local characteristic robot.
The invention provides a method for adjusting a scanning measurement path of a large-scale barrel part local characteristic robot, which comprises the following steps:
constructing a large-scale barrel robot scanning measurement field, obtaining a large-scale barrel theoretical pose based on a theoretical model, obtaining a theoretical scanning measurement path based on the large-scale barrel theoretical pose, and obtaining three-dimensional point cloud data of the large-scale barrel based on the theoretical scanning measurement path by a robot;
judging the integrity of local characteristic point clouds of the large cylinder based on the three-dimensional point cloud data of the large cylinder to obtain a judging result;
and obtaining a robot motion program based on the judging result, and obtaining a final scanning measurement path of the robot based on the robot motion program.
Preferably, the point cloud of the large-scale barrel surface-mounted parts forms a local characteristic point cloud of the large-scale barrel;
the parts on the surface of the large cylinder part comprise parts consisting of a plane, a regular curved surface and holes.
Preferably, the process of obtaining the final scan measurement path of the robot includes:
s1, obtaining a theoretical pose of a large-scale cylinder based on a theoretical model, and obtaining a theoretical scanning measurement path and three-dimensional point cloud data of the large-scale cylinder based on the theoretical pose of the large-scale cylinder;
s2, judging the integrity of the local characteristic point cloud of the large barrel based on a manual observation mode, and obtaining a judgment result;
when the judging result is complete, ending the measurement to obtain the theoretical scanning measurement path, and obtaining a final robot scanning measurement path based on the theoretical scanning measurement path;
when the judging result is incomplete, executing S3;
s3, dividing the three-dimensional point cloud data of the large barrel part to obtain incomplete local characteristic point clouds; obtaining a standard local characteristic point cloud based on the theoretical pose of the large cylinder;
s4, constructing a point cloud completion model, and completing the incomplete local characteristic point cloud based on the point cloud completion model to obtain a complete local characteristic point cloud;
s5, registering the complementary local feature point cloud based on the standard local feature point cloud to obtain the actual pose of the large cylinder;
s6, modifying a workpiece coordinate system in robot path simulation based on the actual pose of the large cylinder to obtain an adjustment path;
s7, judging the accessibility of the adjustment path based on simulation, and obtaining a final scanning measurement path of the robot based on the accessibility of the adjustment path;
when the adjustment path is reachable, generating a robot motion program based on the adjustment path, and obtaining a final robot scanning measurement path based on the robot motion program;
and when the path is not reachable, adjusting the large cylinder based on the actual pose of the large cylinder, and circularly executing S1-S7 until the path is reachable, so as to obtain the final scanning and measuring path of the robot.
Preferably, the specific process of S3 includes:
dividing the three-dimensional point cloud data of the large barrel based on a random sampling consistency method to obtain a plurality of local characteristic point clouds;
and clustering the local feature point clouds based on a K-means method to obtain a plurality of clustering centers and a plurality of local feature point clouds corresponding to the clustering centers.
Preferably, the process of obtaining the standard local characteristic point cloud based on the theoretical pose of the large barrel comprises the following steps:
obtaining a local characteristic theoretical pose based on the large-scale barrel theoretical pose;
obtaining a local feature CAD model based on the local feature theoretical pose;
and obtaining a standard local feature point cloud based on the local feature CAD model.
Preferably, the point cloud completion model is constructed based on a deep learning algorithm, wherein the point cloud completion model comprises: standard feature extraction module, coding module and decoding module.
Preferably, the process of complementing the incomplete local feature point cloud based on the point cloud complementing model to obtain the complemented local feature point cloud includes:
inputting the standard local feature point cloud and the incomplete local feature point cloud into the standard feature extraction module, and obtaining a first result through multi-layer convolution, matrix multiplication and multi-layer convolution calculation;
inputting the incomplete local feature point cloud into the coding module, and obtaining a second result through multi-layer convolution and pooling calculation;
combining the first result with the second result, and performing data expansion and maximum pooling treatment to obtain a third result;
inputting the third result into the decoding module, and obtaining a sparse complement result through multi-layer convolution;
and up-sampling the sparse complement result to obtain a dense complement result.
Preferably, the specific process of S5 includes:
registering the dense complement result and the standard local feature point cloud based on a SACA-ICP method to obtain a first pose conversion matrix, and carrying out pose conversion on the sparse complement result based on the first pose conversion matrix to obtain a sparse complement result after pose conversion;
registering the sparse complement result after pose transformation with the standard local feature point cloud based on an ICP method to obtain a second pose transformation matrix;
obtaining a third pose conversion matrix based on the first pose conversion matrix and the second pose conversion matrix;
and obtaining the actual pose of the large cylinder based on the third pose conversion matrix.
The invention has the following technical effects:
the invention provides a method for adjusting the scanning measurement path of a large-scale barrel part local characteristic robot, which aims at the problem that the measurement path planned based on a theoretical model cannot adapt to the actual state of the large-scale barrel part and further causes point cloud deficiency in the robot scanning measurement of the large-scale barrel part, and can effectively ensure the integrity of the point cloud of the measurement result under the condition that the actual pose and the theoretical pose of the large-scale barrel part deviate.
According to the invention, the incomplete point cloud of the large barrel is segmented, complemented and registered, so that the calculation of the actual pose of the large barrel based on the incomplete point cloud is realized.
The invention provides a point cloud completion model fused with theoretical model data, and accurate point cloud completion effect can be realized based on a deep learning method by fusing theoretical model point cloud data of local characteristics of a large barrel.
The invention has simple operation process, and no additional assistance of operators is needed in the implementation process of the specific process.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions of the prior art, the drawings that are needed in the embodiments will be briefly described below, it being obvious that the drawings in the following description are only some embodiments of the present invention, and that other drawings may be obtained according to these drawings without inventive effort for a person skilled in the art.
FIG. 1 is a flow chart of a method for adjusting a scanning measurement path of a large-scale drum part local characteristic robot in an embodiment of the invention;
FIG. 2 is a diagram of a partial characterization robot scan measurement system for a large cartridge in an embodiment of the present invention;
FIG. 3 is a flow of point cloud segmentation of a large cartridge in an embodiment of the present invention;
FIG. 4 is a point cloud completion model principle of fusing theoretical model data in an embodiment of the present invention;
fig. 5 is a method of registration of local features of a large cartridge in an embodiment of the invention.
Detailed Description
The following description of the embodiments of the present invention will be made clearly and completely with reference to the accompanying drawings, in which it is apparent that the embodiments described are only some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
Example 1
The embodiment provides a method for adjusting a scanning measurement path of a large-scale barrel part local characteristic robot, which specifically comprises the following steps:
s1, constructing a large-scale barrel robot scanning measuring field, planning a robot scanning measuring path based on a large-scale barrel theoretical pose, and completing robot scanning measuring operation on local characteristics of the large-scale barrel based on the path to obtain three-dimensional point cloud data of the large-scale barrel.
S2, judging whether the local characteristic point cloud in the three-dimensional point cloud data of the large barrel is complete, ending measurement if the local characteristic point cloud is complete, and executing the step S3 if the local characteristic point cloud is incomplete.
S3, carrying out segmentation processing on the large barrel part point cloud obtained in the step S2, and extracting incomplete local characteristic point cloud.
And S4, establishing a point cloud complement model fused with theoretical model data based on a deep learning method, and complementing the local feature point cloud extracted in the step S3, wherein the local feature point cloud comprises a sparse complement result and a dense complement result, so that a complete local feature point cloud is obtained.
And S5, registering the complete local characteristic point cloud after the completion of the S4 with the local characteristic point cloud under the theoretical pose of the large-scale cylinder, and calculating to obtain the actual pose of the large-scale cylinder.
S6, modifying a workpiece coordinate system in a robot scanning measurement path based on the actual pose of the large cylinder obtained by calculation in the S5, judging whether the modified path is reachable through simulation, generating a robot motion program if the modified path is reachable, carrying out robot scanning measurement operation by adopting the program, adjusting the large cylinder according to the actual pose if the robot motion program is not reachable, and repeatedly executing the S1 after adjustment.
Specifically, the local characteristic of the large-scale cylinder is a part which consists of a plane, a regular curved surface, a hole and the like and is installed or attached to the surface of the large-scale cylinder.
Specifically, in step S1, the large-scale drum robot scanning measurement field is composed of an industrial robot, a guide rail, a visual tracker, a laser scanner and a large-scale drum.
Specifically, step S3 includes:
s3.1, dividing the large barrel part point cloud by adopting a random sampling consistency (RANSAC) method. Because the local features are distributed on the cylindrical surface of the large cylinder, a cylindrical equation of the surface of the large cylinder can be fitted by using the RANSAC, and then the local feature point cloud is extracted as an independent integral point cloud.
S3.2, clustering the extraction result of the S3.1 by adopting a K-Means (K-Means) method, so that a plurality of groups of different local characteristic point clouds and clustering centers can be obtained, and the category of each group of characteristic point clouds is distinguished according to the position distribution of each group of point cloud clustering centers in space.
Specifically, in step S4, the point cloud completion model fused with the theoretical model data is constructed based on a deep learning algorithm, and includes a standard feature extraction module, an encoding module and a decoding module, where the input of the model includes a standard local feature point cloud and an incomplete local feature point cloud obtained in step S3, and the output includes a sparse completion result and a dense completion result.
Furthermore, the standard local feature point cloud is a point cloud generated by a CAD model of the local feature of the large-scale barrel under the theoretical pose.
Furthermore, the standard feature extraction module takes the standard local feature point cloud and the incomplete local feature point cloud as inputs, the standard local feature point cloud and the incomplete local feature point cloud are respectively subjected to multi-layer convolution calculation, the obtained results are multiplied by a matrix, and finally, the multi-layer convolution calculation is performed again, so that the results are output. The coding module takes incomplete local feature point cloud as input, and after multi-layer convolution and pooling calculation, the result is combined with the output of the standard feature extraction module, and data expansion and maximum pooling processing are carried out to output the result. The decoding module takes the output of the encoding module as input, outputs a sparse complement result after multi-layer convolution, and up-samples the sparse complement result to obtain a dense complement result.
Specifically, step S5 includes:
s5.1, registering the dense complement result with a standard local feature point cloud by adopting a SACA-ICP method to obtain a pose conversion matrix T1, and carrying out pose conversion on the sparse complement result according to the conversion matrix T1.
And S5.2, registering the sparse complement result obtained in the step S5.1 after pose transformation with a standard local feature point cloud by adopting an ICP method to obtain a pose transformation matrix T2.
S5.3, multiplying T1 obtained in S5.1 and T2 obtained in S5.2 to obtain a conversion matrix of the actual pose and the theoretical pose of the large-scale cylinder, wherein the conversion matrix is used for representing the actual pose of the large-scale cylinder.
Example two
Fig. 1 is a schematic flow chart of a method for adjusting a scanning and measuring path of a large-scale drum local feature robot according to an embodiment of the present invention, and the method for adjusting a scanning and measuring path of a large-scale drum local feature robot mainly includes the following steps:
s1, measuring local characteristics of a large cylinder by adopting a scanning path planned based on a theoretical model:
the robot scanning and measuring system adopted by the invention is shown in fig. 2, and comprises an industrial robot, a guide rail, a scanner, a visual tracker and a large barrel. The local features to be measured comprise a first local feature and a second local feature, and are arranged on the surface of the workpiece. The coordinate systems in the robot scanning measurement system mainly include a world coordinate system { W }, a robot base coordinate system { B }, a robot end effector coordinate system { E }, a scanner coordinate system { S }, a vision tracker coordinate system { V }, and a large-scale drum coordinate system { L }. The invention adopts homogeneous conversion matrix representation for the conversion relation of the coordinate system. For point set P obtained by scanner scanning i Expressed in world coordinate system asWherein->And V S T represents the homogeneous transformation matrix between the visual tracker coordinate system and the world and scanner coordinate systems, respectively. The visual tracker acquires the position of the visual target point in the three-dimensional space by tracking the visual target point on the scanner in real time, so as to obtainTo->But->Then it is determined by calibration of the visual tracker.
The nature of the robot scan path is a collection of multiple scan viewpoints, each corresponding to a determined robot pose. Under the determined scanning pose, the following relation existsIn the method, in the process of the invention,respectively representing the conversion relations between the coordinate systems. />The pose of each scanning viewpoint of the plan, i.e., the positional relationship between the scanner and the workpiece, is also represented. In practical use, for each +.> The values of (2) are all determined, and +.>Because the unknown of the implementation state of the large-scale component is uncertain in batch production, the premise of ensuring whether the planned scanning path is accurate in practical application is that the pose of the workpiece after each clamping is consistent with that in a CAD model.
The measurement path is planned based on a theoretical model, and the basic principle that the whole measurement range covers the local features and the scanner does not interfere with the large barrel part is followed. Finally, the scanning measurement is accomplished by movement of the scanner relative to the large cartridge.
S2, judging the cloud state of the local characteristic points of the large barrel part:
and judging whether the local characteristic point cloud of the large barrel part obtained by scanning is complete, if so, ending the measurement, and if not, executing S3, wherein the judgment mode is mainly manual observation.
S3, large-scale barrel part local feature point cloud segmentation processing:
because the scanning result of the large barrel part is an integral point cloud, each local characteristic point cloud needs to be separated from the integral point cloud, the invention adopts a method based on RANSAC and K-Means to carry out point cloud segmentation, as shown in figure 3. First, the RANSAC is adopted to the original point cloud P o And (5) dividing. Since the local features are distributed on the cylindrical surface, the cylindrical equation of the large-scale barrel can be easily fitted by using RANSAC, and the local features are extracted from the surface to obtain P f . Subsequently, the K-Means algorithm is used for P f Clustering to obtain i groups of local feature point clouds, wherein i is the number of local features on the surface of the large cylinder part, and each point cloud is formed by P fi And (3) representing. At the same time, a cluster center C can be obtained i . According to C i Determining a local feature point cloud P at a position in space fi And the corresponding relation with local characteristics in the theoretical model.
S4, carrying out complement treatment on the partial characteristic point cloud of the segmented incomplete large barrel part:
the invention provides a point cloud complement model fused with theoretical model data based on a deep learning method, and the model architecture is shown in figure 4. The model comprises three modules, namely a standard feature extraction module, an encoding module and a decoding module, and the inputs of the model are a standard local feature point cloud P and an incomplete local feature point cloud P. Wherein P is the result of the segmentation of the S3 point cloud, the data dimension is [ x,3], and x is the number of the actual points. P is a standard local feature point cloud generated from the theoretical pose of the large cartridge CAD model. The data dimension of P is [1024,3].
In the standard feature extraction module, P is taken as input, and then the standard point cloud features are obtained through multi-layer convolution, and the number of convolution channels is (128, 64,3). The purpose of this step is to extract the shape information of the complete local feature point cloud. And the P input is subjected to multi-layer convolution and then subjected to maximum pooling, the number of convolution channels is (64, 128, 256), and the obtained result isThen through multi-layer convolution with channel number (256, 512,9), the data is changed to be [3, 3]]And obtaining T-net which represents the position information of the actual residual point cloud. The result obtained by multiplying the standard point cloud features and the T-net is processed by a global feature extraction module to obtain global features G n The network structure of the global feature extraction module is the same as the structure of the encoder 1 of the encoding module. G n The method comprises the shape information of the theoretical model of the feature and the position information of the actual model.
The input of the coding module is P, the module is divided into an encoder 1 and an encoder 2, after P is input into the encoder 1, firstly, the characteristic vector is obtained after multi-layer convolution with the channel number of (128, 256), then the maximum pooling is carried out, the result of the maximum pooling is unfolded and spliced with the characteristic vector, and the global characteristic G of the incomplete point cloud is obtained after multi-layer convolution with the channel number of (512, 1024) and the maximum pooling treatment is carried out again e1 . While the encoder 2 is designed to integrate the output G of the standard feature extraction module n And G e1 The result is spread after splicing, and then G is obtained after convolution and maximum pooling with the channel number of 2048, 1024 e2 。
The input of the decoding module is the output of the encoding module, and after convolution with the number of channels (1024, 1024, 3072), the dimension of the result is modified to obtain a dimension [1024,3]]Sparse complement result P of (2) C . Will P C Performing folding operation, namely dimension expansion, and G e2 And the dimension expansion result of (2) is spliced with the randomly generated two-dimensional grids to obtain the dimension of [16384, 1029 ]]Is subjected to a multi-layer convolution of the number of channels (512, 512,3), the result of which is equal to P C After addition of the folding results of (a) gives a dimension of [8192,3 ]]Is the dense complement result P of (2) D . Calculating two partial losses in the decoding module, P respectively C And corresponding sparse target resultsIs a bulldozing Distance EMD (Earth Mover's Distance):
p D And corresponding dense target resultsChamfering distance CD (Chamfer Distance) of (c):
the final loss function of the model is therefore:
s5, completing the S4 to form a complete local feature point cloud P C And P D Registering with a local characteristic point cloud under the theoretical pose of the large-scale cylinder, namely a standard local characteristic point cloud, and calculating to obtain the actual pose of the large-scale cylinder:
the point cloud registration process utilizes the point cloud complement result, P C And P D The registration accuracy can be effectively improved through twice fine registration, and the specific steps are shown in fig. 5. First, generating dense standard local feature point cloud according to theoretical modelThe point cloud dimension is [8192,3 ]]. The SACA-ICP algorithm is adopted to complete the density complement result P after the complement D And (3) regulating->Coarse registration is carried out to obtain a first registration result conversion matrix T 1 The method comprises the steps of carrying out a first treatment on the surface of the Then generating sparse standard local characteristic point cloud according to the theoretical model>Will->And T is 1 Multiplying the obtained result by P C Performing ICP calculation to obtain a second accurate registration junctionFruit T 2 Final registration result T f ＝T 1 ·T 2 。
The algorithm utilizes two output point clouds of the point cloud complement model provided by the invention, and can improve registration accuracy through secondary registration of the point clouds with different densities. The registration result is the pose calculation result of a single local feature, and the whole pose calculation of the large-scale barrel part is represented by averaging the transformation matrixes of all local features, namelyWherein T represents the pose conversion matrix of the actual pose and the theoretical model of the large-scale barrel, and n represents the number of local features.
S6, modifying a workpiece coordinate system in robot path simulation according to the actual pose of the large cylinder, and finishing path adjustment:
in the first measurement of the large-scale cylinder, the robot path is planned according to the theoretical model, the pose conversion matrix of the theoretical model and the actual pose of the large-scale cylinder is obtained through the steps, and the movement amount and the rotation angle (x, y, z, alpha, beta, gamma) of the actual workpiece coordinate system of the large-scale cylinder relative to the theoretical coordinate system on three coordinate axes of x, y and z can be calculated according to the conversion matrix. Therefore, in the robot simulation software, the workpiece coordinate system of the large cylinder can be adjusted according to (x, y, z, alpha, beta, gamma) on the basis of the path planned by the theory, then the original path is automatically adjusted along with the change of the workpiece coordinate system, finally simulation verification is carried out on the newly generated path, if the path is reachable in space by the robot, a robot motion program is generated to execute further measurement operation, and if the path is not reachable, the actual pose of the large cylinder needs to be adjusted.
In summary, the method for adjusting the scanning measurement path of the large-scale cylinder part local characteristic robot aims at solving the problem that the measurement point cloud is incomplete due to the fact that the actual pose is deviated from the theoretical model in the process of scanning measurement of the large-scale cylinder part, and the method for adjusting the measurement path according to the actual pose by acquiring the actual pose of the large-scale cylinder part through operations such as segmentation, complementation and registration of the incomplete point cloud is provided, so that the accuracy of the scanning measurement result of the large-scale cylinder part robot can be effectively ensured, and the manufacturing quality and efficiency of the large-scale cylinder part are improved.
The foregoing has shown and described the basic principles, principal features and advantages of the invention. It will be understood by those skilled in the art that the present invention is not limited to the embodiments described above, and that the above embodiments and descriptions are merely illustrative of the principles of the present invention, and various changes and modifications may be made without departing from the spirit and scope of the invention, which is defined in the appended claims. The scope of the invention is defined by the appended claims and equivalents thereof.
Claims (2)
1. The method for adjusting the scanning measurement path of the large-scale barrel part local characteristic robot is characterized by comprising the following steps of:
constructing a large-scale barrel robot scanning measurement field, obtaining a large-scale barrel theoretical pose based on a theoretical model, obtaining a theoretical scanning measurement path based on the large-scale barrel theoretical pose, and obtaining three-dimensional point cloud data of the large-scale barrel based on the theoretical scanning measurement path by a robot;
judging the integrity of local characteristic point clouds of the large cylinder based on the three-dimensional point cloud data of the large cylinder to obtain a judging result;
obtaining a robot motion program based on the judging result, and obtaining a final robot scanning measurement path based on the robot motion program;
the process of obtaining the final scan measurement path of the robot includes:
s1, obtaining a theoretical pose of a large-scale cylinder based on a theoretical model, and obtaining a theoretical scanning measurement path and three-dimensional point cloud data of the large-scale cylinder based on the theoretical pose of the large-scale cylinder;
s2, judging the integrity of the local characteristic point cloud of the large barrel based on a manual observation mode, and obtaining a judgment result;
when the judging result is complete, ending the measurement to obtain the theoretical scanning measurement path, and obtaining a final robot scanning measurement path based on the theoretical scanning measurement path;
when the judging result is incomplete, executing S3;
s3, dividing the three-dimensional point cloud data of the large barrel part to obtain incomplete local characteristic point clouds; obtaining a standard local characteristic point cloud based on the theoretical pose of the large cylinder;
s4, constructing a point cloud completion model, and completing the incomplete local characteristic point cloud based on the point cloud completion model to obtain a complete local characteristic point cloud;
s5, registering the complementary local feature point cloud based on the standard local feature point cloud to obtain the actual pose of the large cylinder;
s6, modifying a workpiece coordinate system in robot path simulation based on the actual pose of the large cylinder to obtain an adjustment path;
s7, judging the accessibility of the adjustment path based on simulation, and obtaining a final scanning measurement path of the robot based on the accessibility of the adjustment path;
when the adjustment path is reachable, generating a robot motion program based on the adjustment path, and obtaining a final robot scanning measurement path based on the robot motion program;
when the path is not reachable, adjusting the large cylinder based on the actual pose of the large cylinder, and circularly executing S1-S7 until the path is reachable, so as to obtain a final scanning measurement path of the robot;
the specific process of the S3 comprises the following steps:
dividing the three-dimensional point cloud data of the large barrel based on a random sampling consistency method to obtain a plurality of local characteristic point clouds;
clustering a plurality of local feature point clouds based on a K-means method to obtain a plurality of clustering centers and a plurality of local feature point clouds corresponding to the clustering centers;
the process for obtaining the standard local characteristic point cloud based on the theoretical pose of the large barrel comprises the following steps:
obtaining a local characteristic theoretical pose based on the large-scale barrel theoretical pose;
obtaining a local feature CAD model based on the local feature theoretical pose;
obtaining a standard local feature point cloud based on the local feature CAD model;
the point cloud completion model is constructed based on a deep learning algorithm, wherein the point cloud completion model comprises: the device comprises a standard feature extraction module, an encoding module and a decoding module;
the incomplete local feature point cloud is complemented based on the point cloud complement model, and the process for obtaining the complemented local feature point cloud comprises the following steps:
inputting the standard local feature point cloud and the incomplete local feature point cloud into the standard feature extraction module, and obtaining a first result through multi-layer convolution, matrix multiplication and multi-layer convolution calculation;
inputting the incomplete local feature point cloud into the coding module, and obtaining a second result through multi-layer convolution and pooling calculation;
combining the first result with the second result, and performing data expansion and maximum pooling treatment to obtain a third result;
inputting the third result into the decoding module, and obtaining a sparse complement result through multi-layer convolution;
up-sampling the sparse complement result to obtain a dense complement result;
the specific process of S5 includes:
registering the dense complement result and the standard local feature point cloud based on a SACA-ICP method to obtain a first pose conversion matrix, and carrying out pose conversion on the sparse complement result based on the first pose conversion matrix to obtain a sparse complement result after pose conversion;
registering the sparse complement result after pose transformation with the standard local feature point cloud based on an ICP method to obtain a second pose transformation matrix;
obtaining a third pose conversion matrix based on the first pose conversion matrix and the second pose conversion matrix;
and obtaining the actual pose of the large cylinder based on the third pose conversion matrix.
2. The method for adjusting the scanning measurement path of the large-scale drum part local characteristic robot according to claim 1, wherein the point cloud of the large-scale drum part surface-mounted part forms the local characteristic point cloud of the large-scale drum part;
the parts on the surface of the large cylinder part comprise parts consisting of a plane, a regular curved surface and holes.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310577100.XA CN116476070B (en) | 2023-05-22 | 2023-05-22 | Method for adjusting scanning measurement path of large-scale barrel part local characteristic robot |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202310577100.XA CN116476070B (en) | 2023-05-22 | 2023-05-22 | Method for adjusting scanning measurement path of large-scale barrel part local characteristic robot |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN116476070A CN116476070A (en) | 2023-07-25 |
| CN116476070B true CN116476070B (en) | 2023-11-10 |
Family
ID=87223340
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202310577100.XA Active CN116476070B (en) | 2023-05-22 | 2023-05-22 | Method for adjusting scanning measurement path of large-scale barrel part local characteristic robot |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN116476070B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN117444968A (en) * | 2023-11-02 | 2024-01-26 | 北京航空航天大学 | Method for planning scanning measurement path of large-scale barrel part local characteristic robot |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109341591A (en) * | 2018-11-12 | 2019-02-15 | 杭州思看科技有限公司 | A kind of edge detection method and system based on handheld three-dimensional scanner |
| CN110246186A (en) * | 2019-04-15 | 2019-09-17 | 深圳市易尚展示股份有限公司 | A kind of automatized three-dimensional colour imaging and measurement method |
| CN112161619A (en) * | 2020-09-16 | 2021-01-01 | 杭州思锐迪科技有限公司 | Pose detection method, three-dimensional scanning path planning method and detection system |
| CN112325796A (en) * | 2020-10-26 | 2021-02-05 | 上海交通大学 | Large-scale workpiece profile measuring method based on auxiliary positioning multi-view point cloud splicing |
| CN114139353A (en) * | 2021-11-11 | 2022-03-04 | 北京魔鬼鱼科技有限公司 | Method, system and computer program for optimizing simulated imaging perception data |
| WO2022165973A1 (en) * | 2021-02-05 | 2022-08-11 | 杭州思看科技有限公司 | Three-dimensional scanning method and system, electronic device, and computer equipment |
| CN115056213A (en) * | 2022-02-18 | 2022-09-16 | 上海大学 | Robot track self-adaptive correction method for large complex component |
-
2023
- 2023-05-22 CN CN202310577100.XA patent/CN116476070B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109341591A (en) * | 2018-11-12 | 2019-02-15 | 杭州思看科技有限公司 | A kind of edge detection method and system based on handheld three-dimensional scanner |
| CN110246186A (en) * | 2019-04-15 | 2019-09-17 | 深圳市易尚展示股份有限公司 | A kind of automatized three-dimensional colour imaging and measurement method |
| CN112161619A (en) * | 2020-09-16 | 2021-01-01 | 杭州思锐迪科技有限公司 | Pose detection method, three-dimensional scanning path planning method and detection system |
| CN112325796A (en) * | 2020-10-26 | 2021-02-05 | 上海交通大学 | Large-scale workpiece profile measuring method based on auxiliary positioning multi-view point cloud splicing |
| WO2022165973A1 (en) * | 2021-02-05 | 2022-08-11 | 杭州思看科技有限公司 | Three-dimensional scanning method and system, electronic device, and computer equipment |
| CN114139353A (en) * | 2021-11-11 | 2022-03-04 | 北京魔鬼鱼科技有限公司 | Method, system and computer program for optimizing simulated imaging perception data |
| CN115056213A (en) * | 2022-02-18 | 2022-09-16 | 上海大学 | Robot track self-adaptive correction method for large complex component |
Non-Patent Citations (1)
| Title |
|---|
| 基于机器人的飞机大型结构三维自动化检测方法;陈允全;李泷杲;黄翔;李琦;林晶;;航空制造技术(10);第53-59页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN116476070A (en) | 2023-07-25 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN112927357B (en) | 3D object reconstruction method based on dynamic graph network | |
| CN111080627A (en) | 2D +3D large airplane appearance defect detection and analysis method based on deep learning | |
| US7133551B2 (en) | Semi-automatic reconstruction method of 3-D building models using building outline segments | |
| CN110189399B (en) | Indoor three-dimensional layout reconstruction method and system | |
| CN104019799B (en) | A kind of relative orientation method utilizing local parameter optimization to calculate basis matrix | |
| Wei et al. | A non-contact measurement method of ship block using image-based 3D reconstruction technology | |
| US20230206603A1 (en) | High-precision point cloud completion method based on deep learning and device thereof | |
| CN104778688A (en) | Method and device for registering point cloud data | |
| WO2021052283A1 (en) | Method for processing three-dimensional point cloud data and computing device | |
| CN115345822A (en) | Automatic three-dimensional detection method for surface structure light of aviation complex part | |
| CN116476070B (en) | Method for adjusting scanning measurement path of large-scale barrel part local characteristic robot | |
| CN116402866A (en) | Point cloud-based part digital twin geometric modeling and error assessment method and system | |
| CN105654483A (en) | Three-dimensional point cloud full-automatic registration method | |
| CN110111237B (en) | Three-dimensional model coordinate conversion method based on Brep model | |
| Moemen et al. | 3-D reconstruction and measurement system based on multimobile robot machine vision | |
| CN111400830A (en) | Machining and calibrating method and device for three-dimensional blank workpiece | |
| CN113362463A (en) | Workpiece three-dimensional reconstruction method based on Gaussian mixture model | |
| Adallah et al. | 3D point cloud analysis for automatic inspection of aeronautical mechanical assemblies | |
| CN110838146A (en) | Homonymy point matching method, system, device and medium for coplanar cross-ratio constraint | |
| CN115423854B (en) | Multi-view point cloud registration and point cloud fusion method based on multi-scale feature extraction | |
| Zhang et al. | 4-D spatiotemporal detection and modeling of free-bending pipelines in cluttered 3-D point cloud | |
| Filaretov et al. | Method of combination of three-dimensional models of details with their CAD-models at the presence of deformations | |
| CN118070434B (en) | Method and system for constructing process information model of automobile part | |
| CN117953164B (en) | Method and system for improving drawing measurement quality | |
| CN114494429B (en) | Large-scale uncontrolled three-dimensional adjustment net geometric positioning gross error detection and processing method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |