CN115283172A - Robot automatic spraying method based on point cloud processing - Google Patents
Robot automatic spraying method based on point cloud processing Download PDFInfo
- Publication number
- CN115283172A CN115283172A CN202210870010.5A CN202210870010A CN115283172A CN 115283172 A CN115283172 A CN 115283172A CN 202210870010 A CN202210870010 A CN 202210870010A CN 115283172 A CN115283172 A CN 115283172A
- Authority
- CN
- China
- Prior art keywords
- point cloud
- spraying
- template
- robot
- point
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
- 238000005507 spraying Methods 0.000 title claims abstract description 128
- 238000000034 method Methods 0.000 claims abstract description 75
- 230000008569 process Effects 0.000 claims abstract description 52
- 239000007921 spray Substances 0.000 claims abstract description 39
- 238000007781 pre-processing Methods 0.000 claims abstract description 22
- 238000000605 extraction Methods 0.000 claims abstract description 9
- 230000009471 action Effects 0.000 claims abstract description 6
- 239000013598 vector Substances 0.000 claims description 41
- 230000009466 transformation Effects 0.000 claims description 38
- 238000004364 calculation method Methods 0.000 claims description 29
- 239000011159 matrix material Substances 0.000 claims description 28
- 238000005457 optimization Methods 0.000 claims description 19
- 238000004519 manufacturing process Methods 0.000 claims description 17
- 238000001914 filtration Methods 0.000 claims description 14
- 239000003973 paint Substances 0.000 claims description 14
- 238000003672 processing method Methods 0.000 claims description 8
- 238000005070 sampling Methods 0.000 claims description 5
- 238000002360 preparation method Methods 0.000 claims 1
- 238000005516 engineering process Methods 0.000 abstract description 3
- 230000011218 segmentation Effects 0.000 description 8
- 238000007592 spray painting technique Methods 0.000 description 8
- 150000001875 compounds Chemical class 0.000 description 7
- 238000010586 diagram Methods 0.000 description 4
- 238000004422 calculation algorithm Methods 0.000 description 3
- 230000008859 change Effects 0.000 description 2
- 238000003776 cleavage reaction Methods 0.000 description 2
- 230000001788 irregular Effects 0.000 description 2
- 238000005259 measurement Methods 0.000 description 2
- 230000003287 optical effect Effects 0.000 description 2
- 238000010422 painting Methods 0.000 description 2
- 238000007591 painting process Methods 0.000 description 2
- 238000005192 partition Methods 0.000 description 2
- 238000002203 pretreatment Methods 0.000 description 2
- 230000007017 scission Effects 0.000 description 2
- 238000004904 shortening Methods 0.000 description 2
- 238000007619 statistical method Methods 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 238000013473 artificial intelligence Methods 0.000 description 1
- 230000009286 beneficial effect Effects 0.000 description 1
- 239000002131 composite material Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000005520 cutting process Methods 0.000 description 1
- 238000000354 decomposition reaction Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 230000003631 expected effect Effects 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000009776 industrial production Methods 0.000 description 1
- 238000000691 measurement method Methods 0.000 description 1
- 238000012986 modification Methods 0.000 description 1
- 230000004048 modification Effects 0.000 description 1
- 230000001105 regulatory effect Effects 0.000 description 1
- 238000012876 topography Methods 0.000 description 1
- 230000001131 transforming effect Effects 0.000 description 1
- 238000003466 welding Methods 0.000 description 1
Images
Classifications
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B05—SPRAYING OR ATOMISING IN GENERAL; APPLYING FLUENT MATERIALS TO SURFACES, IN GENERAL
- B05B—SPRAYING APPARATUS; ATOMISING APPARATUS; NOZZLES
- B05B13/00—Machines or plants for applying liquids or other fluent materials to surfaces of objects or other work by spraying, not covered by groups B05B1/00 - B05B11/00
- B05B13/02—Means for supporting work; Arrangement or mounting of spray heads; Adaptation or arrangement of means for feeding work
- B05B13/04—Means for supporting work; Arrangement or mounting of spray heads; Adaptation or arrangement of means for feeding work the spray heads being moved during spraying operation
- B05B13/0431—Means for supporting work; Arrangement or mounting of spray heads; Adaptation or arrangement of means for feeding work the spray heads being moved during spraying operation with spray heads moved by robots or articulated arms, e.g. for applying liquid or other fluent material to 3D-surfaces
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
- G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Robotics (AREA)
- Application Of Or Painting With Fluid Materials (AREA)
- Manipulator (AREA)
Abstract
The invention discloses a robot automatic spraying method based on point cloud processing, wherein hardware components comprise a six-axis robot, three-dimensional reconstruction equipment, a target free-form surface and a spray gun, and the six-axis robot is used for executing automatic spraying action according to a point cloud processing result and technological parameter requirements; the data processing process comprises preprocessing and template making, point cloud feature extraction and description, point cloud registration, spraying template alignment, spraying track acquisition, spraying parameter and attitude adjustment, is used for processing free-form surface point cloud data and template matching, can update a template database in real time and automatically match a proper spraying template according to the point cloud features, and guides the configuration of a spraying process and spray gun motion parameters. The invention uses three-dimensional reconstruction and point cloud processing technology to model and process the characteristics of the free-form surface spraying object, establishes the spraying process database and matches with a proper spraying template, realizes the automatic spraying of the vision-guided robot, can reduce the uncertainty of manual spraying operation, and effectively improves the spraying efficiency.
Description
Technical Field
The invention belongs to the field of artificial intelligence, and particularly relates to a robot automatic spraying method based on point cloud processing.
Background
The spraying robot needs to acquire the compound facial line information of a target workpiece in operation, and the point cloud is a simple and visual data structure and can express the three-dimensional geometric information of a target object. With the progress of computer hardware manufacturing technology, three-dimensional reconstruction and its vision system are gradually applied in industrial production. The method is a non-contact, active, rapid and stable measurement method by utilizing a structured light system and adopting 3D point cloud data containing geometric information of a sprayed workpiece, and is widely applied to engineering practices such as welding seam detection and tracking, underwater measurement, target positioning, composite surface topography measurement and the like at present.
The key of the robot for realizing automatic spraying is to obtain a spraying path of the surface of a workpiece, and the common method is to extract a processing path by cutting a section and a block through a standard CAD model; due to the continuous iterative progress of the point cloud acquisition system, the amount of point cloud data obtained by single acquisition is increased continuously, and the requirements on point cloud preprocessing, feature extraction, point cloud registration and template library manufacturing are stricter. For example, peak and the like [6] provides a spraying track generation method based on a container box decomposition algorithm, the method acts on a standard CAD model, and point cloud data are insensitive; also, as disclosed in CN201811323907.6, the method for rapidly and intelligently programming a spraying robot for a planar/approximately planar workpiece is mainly based on spraying of a planar or approximately planar workpiece, and cannot achieve an expected effect on a free-form surface spraying object; further, as disclosed in CN201610657217.9, the method is to establish a film thickness distribution model of a cylindrical surface paint film by using a trajectory automatic planning module, and perform intersection with a point cloud model through a set of planes with angles of η to obtain three-dimensional cross-sectional profile data, and the optical axis of a calibration camera i is perpendicular to an X-Y plane of a robot coordinate system, and the optical axis of a calibration camera ii is perpendicular to a Y-Z plane of the robot coordinate system, which is complicated in structure.
Disclosure of Invention
The invention provides a robot automatic spraying method based on point cloud processing, which overcomes the defects of the prior art.
In order to solve the technical problems, the invention is realized by the following technical scheme:
the invention relates to a robot automatic spraying method based on point cloud processing, which is realized by loading a data processing method on a hardware system; the hardware system comprises a six-axis robot, three-dimensional reconstruction equipment, a target free-form surface and a spray gun, and is used for acquiring the geometric information of the free-form surface and executing automatic spraying action according to the point cloud processing result and the technological parameter requirement;
the data processing method is used for processing free-form surface point cloud data and matching templates, can update a template database in real time, automatically match a proper spraying template according to point cloud characteristics, and guide the configuration of a spraying process and spray gun motion parameters; the data processing method comprises preprocessing, template making, point cloud feature extraction and description, point cloud registration, spraying template alignment, spraying track acquisition, spraying parameter and attitude adjustment, is used for processing free-form surface point cloud data and template matching, can update a template database in real time and automatically match a proper spraying template according to the point cloud features, and guides the configuration of a spraying process and spray gun motion parameters; the target free-form surface is an irregular, arbitrarily extending surface.
Further, the three-dimensional reconstruction equipment and the spray gun are installed on a six-axis robot; the six-axis robot is an industrial robot having three translational degrees of freedom and three rotational degrees of freedom; the expression mode of the pose of the end flange plate in the robot base coordinate system is as follows:
E R B ＝R(r z )·R(r y )·R(r x )
E t B ＝[t 1 t 2 t 3 ] T ＝[x y z] T
in the formula (I), the compound is shown in the specification, E T B representing the position and orientation transformation matrix of the tail end flange plate under the robot base coordinate system, wherein E R B Represents the posture of the user, E t B representing the location. r is ij I =1,2,3, j =1,2,3 represents the attitude matrix E R B The elements of (a) and (b),t i i =1,2,3 represents an element of the position vector. x, y, z, r x ,r y ,r z Represents the pose description of the robot under a world coordinate system, wherein x, y and z represent positions and r x ,r y ,r z Representing a gesture;
the three-dimensional reconstruction equipment is a structured light camera capable of rapidly acquiring surface point cloud; the expression mode of the shot point cloud data in the camera coordinate system is as follows:
O P C ＝[X O Y O Z O ] T
in the formula (I), the compound is shown in the specification, O P C representing point cloud data in a camera coordinate system, where X O ,Y O ,Z O Representing point cloud data coordinate values. Through hand-eye calibration, a pose transformation matrix between a camera coordinate system and a robot terminal coordinate system can be established C T E And then obtaining the representation of the point cloud data in the robot base coordinate system:
E P O ＝ B T E E T C · C P O
in the formula (I), the compound is shown in the specification, E P O representing a representation of the point cloud data in a robot base coordinate system. And establishing a complete three-dimensional reconstruction model.
The spray gun is an automatic device which can be combined with a six-axis robot for use, parameters can be adjusted through a program, the spray gun is arranged at the tail end of the six-axis robot, and the pose transformation relation between a spray head of the spray gun and the tail end of the robot is recorded as Tool T E The method can be obtained by calibrating a TCP tool coordinate system.
Further, the preprocessing and template making process includes the steps of removing invalid points from input original point cloud data, down-sampling, radius filtering, statistical filtering and normal calculation, extracting a spraying track based on a bounding box method, and storing the spraying track and the processed point cloud into a process database.
The point cloud preprocessing part comprises invalid point removal, down-sampling, radius filtering, statistical filtering and normal calculation, and is used for acquiring an original point cloud P O The point cloud obtained by the preprocessing is recorded as P O-pre-T And recording the extracted spraying track as Path T The parameter of the painting process corresponding to the painting track is recorded as ζ T And the kinematic parameter of the robot corresponding to the spraying track is recorded as ν T. Accordingly, the expression χ of the spraying template can be obtained Tem :
χ Tem ＝{P O-pre-T ,Path T ,ζ T ,ν T }；
Further, the point cloud feature extraction and description process is to extract features of local normal vectors, local radii and local densities of the free-form surface point cloud and the template point cloud to be registered, calculate corresponding sub-feature histograms and describe point cloud features by using the total feature histogram;
the local normal vectors represent the curvature features of the free-form surface, and the curvature features can be deeply expressed by adopting the included angles among the normal vectors as indexes for evaluating the local normal vector features, and the calculation is as follows:
in the formula, con (p) i ) Covariance matrix representing point cloud, where p i Represents any point in the point cloud and,
representing the average point of the point cloud. n is a radical of an alkyl radical i And
represents two different normal vectors, where n i Is a vector of a main normal vector, and,
is n i The neighborhood normal vector of (1), both calculated according to Con (p) i ) Minimum of (2)And obtaining a unit vector of the feature vector corresponding to the feature value. Theta here j And e (0, pi) represents the local normal vector characteristics.
The local radius may represent a curvature variation characteristic of the free-form surface, and the local depth dj thereof is calculated as follows:
where r represents the query point neighborhood sphere radius, n i Representing normal vectors at query points, p i Representing any point in the point cloud (i.e. the query point),
representing neighborhood points of the query point;
the local density can represent the density degree and the point density of the free-form surface point cloud. The calculation method is as follows:
ρ j representing a local density feature, n i Representing normal vectors at query points, p i Representing any point in the point cloud (i.e. the query point),
representing neighborhood points of the query points; on the basis of feature calculation, three sub-histograms are obtained by applying a statistical method. And finally, performing one-dimensional direction splicing on the three sub-histograms to obtain a total histogram to represent the local characteristics of the point cloud.
Further, the point cloud registration process comprises the steps of carrying out feature histogram calculation, key point query, corresponding relation estimation and optimization on the point cloud of the free-form surface and the template point cloud to be registered, and iteratively registering and calculating a transformation matrix;
recording the transformation matrix between the point cloud of free-form surface to be registered and the point cloud of template Tem T raw For the matrixTo solve, the following objective function is generally constructed to solve:
wherein F (\9633;) represents the objective function, P T-x Representing the point cloud to be registered in the template library, P O Representing the point cloud to be registered, Tem R raw representing a rotational transformation of the two point clouds, Tem t raw representing the translational motion of the two point clouds. Due to the uncertainty of the point cloud scale and the spatial distance, the estimation of the transformation matrix is generally divided into coarse estimation and fine estimation, which are also called coarse registration and fine registration. The former has the function of shortening the distance between two points of cloud data, so that the registration time is greatly saved; the latter aims to improve the accuracy of the pose transformation matrix.
Further, the point cloud registration process comprises the steps of carrying out feature histogram calculation, key point query, corresponding relation estimation and optimization on the point cloud of the free-form surface and the point cloud of the template to be registered, and iteratively registering and calculating a transformation matrix; and in the spraying template alignment process, the cloud coordinate system of the point to be registered and the template point cloud coordinate system are aligned and transformed according to the transformation matrix. The transformation is as follows:
P O' ＝ Tem T raw ·P O
in the formula P O' Representing the point cloud under the template point cloud coordinate system, P O Representing the original point cloud of the point, Tem T raw representing two coordinate system transformation matrices.
Furthermore, the process of acquiring the spraying track is to call the spraying track corresponding to the index of the spraying template in the process database on the basis of finishing the alignment transformation; namely, finding the spraying track corresponding to the index in the template library, as follows:
Path T ＝θ(P O-pre-T )
in which theta (-) represents the spray trajectory for finding the corresponding template, P O-pre-T Point cloud data representing a corresponding template, path T Representing the spray trajectories of the corresponding index in the template library.
Furthermore, the spray painting parameter and posture adjustment process is to call the spray painting parameters and robot motion parameters of the corresponding indexes of the spray painting template in the process database on the basis of completing the alignment transformation, and send the spray painting parameters and robot motion parameters to the six-axis robot to complete the posture adjustment of the spray gun; namely, finding out the paint spraying parameters and the robot motion parameters corresponding to the indexes in the template library as follows:
in the formula
Representing the paint parameters and robot motion parameters for finding the corresponding template, P O-pre-T Point cloud data, ζ, representing corresponding templates T ,ν T Representing paint spraying parameters and robot motion parameters of corresponding indexes in the template library; on the basis, the paint spraying parameters and the robot motion parameters are issued to the robot to complete the spraying action.
Further, the normal calculation is to calculate a spraying point, the process needs to be judged, if the normal calculation is qualified, the next processing is continued, and if the normal calculation is not qualified, the preprocessing and template manufacturing process is ended;
the spray points are calculated as follows:
in the formula p sprayi Representing the position of the spray point, n sprayi Representing the attitude of the spray gun at the point of application, p i Representing the locus point on the free-form surface, h representing the spraying height, n i Represents the normal vector at the locus point, \9633; (. Cndot.) represents the regularization of the vector.
The process of extracting the spraying track based on the bounding box method comprises the steps of calculating the main direction of point cloud, calculating and dividing intervals, dividing to obtain a point cloud sequence, extracting track points from the point cloud sequence, optimizing the redundancy of the track points to obtain a spraying track and finally forming a spraying template together with the preprocessed point cloud on the basis of finishing the preprocessing of the point cloud of the free-form surface; the point cloud main direction is calculated by adopting a PCA method, the partition interval is obtained by carrying out equal-distance partition according to a boundary plane parallel to the main direction, and the formed plane sequence is as follows:
wherein PlaneSeq represents a sequence of a cleavage Plane, plane i Represents a segmentation plane, vector (·) represents an arbitrary vector in the plane, n PCA Representing the PCA vector, d (·,) representing the distance between two parallel planes, δ representing the paint width, n seg Representing the direction of division, d max Representing the maximum distance in the direction of the segmentation.
Combining with the segmentation plane, adopting a direct filtering method to obtain the track points on the free curved surface:
P pathi ＝PassT(PlaneSeq)
in the formula P pathi Representing the tracing points on the free-form surface, and PassT (DEG) representing the straight-through filtering; according to the distance condition between two points and the distance condition between three points, the track points after redundancy optimization can be obtained:
P i ＝Opt(P pathi ,d 2con ,d 3con )
in the formula P i Represents the track points on the optimized free-form surface, opt (-) represents the optimization function, d 2con ,d 3con Respectively representing a two-point distance condition and a three-point distance condition;
further, the spraying template needs to be judged, if the spraying template is qualified, the next processing is continued, otherwise, the preprocessing and template manufacturing process is ended, namely, the following form is adopted:
Δ＝con Δ (pre,tem)
wherein Δ represents the condition threshold of the pre-treatment and template fabrication process, con Δ (-) represents a threshold computation function, pre, tem represent the pre-processing and template fabrication processes, respectively; the track point redundancy optimizationJudging operation is needed, if the product is qualified, the next processing is continued, otherwise, the pretreatment and template manufacturing process is ended; namely, it is determined by the following form:
▽＝con ▽ (Opt)
where ^ represents the conditional threshold, con, of track point redundancy optimization ▽ (. Cndot.) represents a threshold computation function.
Compared with the prior art, the invention has the following beneficial effects:
the invention applies three-dimensional reconstruction and point cloud processing technology, models the free-form surface spraying object and processes the characteristics, establishes the spraying process database and matches with a proper spraying template, realizes automatic spraying of a visual guidance robot, can reduce the uncertainty of manual spraying operation, and effectively improves the spraying efficiency.
Of course, it is not necessary for any product in which the invention is practiced to achieve all of the above-described advantages at the same time.
Drawings
In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of a robot automatic spraying method based on point cloud processing according to the present invention;
FIG. 2 is a flow chart of the pretreatment and template fabrication of the present invention;
FIG. 3 is a flow chart of the present invention for extracting and describing the point cloud features;
FIG. 4 is a schematic diagram of the point cloud local feature histogram construction of the present invention;
FIG. 5 is a schematic diagram of the point cloud registration of the present invention;
FIG. 6 is a flow chart of the method for extracting the spraying trajectory based on the bounding box of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
Specific example 1:
please refer to fig. 1-6;
referring to fig. 1, the robot automatic spraying method based on point cloud processing of the present invention is implemented by loading a hardware system based on a data processing method; the hardware system comprises a six-axis robot, three-dimensional reconstruction equipment, a target free-form surface and a spray gun, and is used for acquiring the geometric information of the free-form surface and executing automatic spraying action according to the point cloud processing result and the technological parameter requirement; assembling a three-dimensional reconstruction device by a six-axis robot, performing three-dimensional modeling on a target free-form surface to obtain point cloud data, and recording the point cloud data as P O Or E P O ；
The data processing method is used for processing the free-form surface point cloud data and matching templates, can update the template database in real time, automatically match a proper spraying template according to the point cloud characteristics, and guide the configuration of a spraying process and spray gun motion parameters; the data processing method comprises preprocessing, template making, point cloud feature extraction and description, point cloud registration, spraying template alignment, spraying track acquisition, spraying parameter and attitude adjustment, is used for processing free-form surface point cloud data and template matching, can update a template database in real time and automatically match a proper spraying template according to the point cloud features, and guides the configuration of a spraying process and spray gun motion parameters; the target free-form surface is an irregular and randomly-extending curved surface;
and (4) preprocessing and template manufacturing are carried out by inputting the point cloud data of the free-form surface to obtain the spraying template. Then, acquiring a feature histogram of point cloud to be registered and template point cloud through point cloud feature extraction and description, performing point cloud registration on the basis to obtain a pose transformation matrix between the two point clouds, performing spray template alignment to obtain corresponding spray parameters, acquiring a spray track according to an index and a spray template obtained by combining pretreatment and template manufacturing, and performing spray painting parameter and attitude adjustment; after the spraying parameters and the motion parameters are obtained, the six-axis robot assembles the spray gun to perform spraying operation on the target free-form surface, and automatic spraying is achieved.
The three-dimensional reconstruction equipment and the spray gun are arranged on the six-axis robot; a six-axis robot is an industrial robot having three translational degrees of freedom and three rotational degrees of freedom; the expression mode of the pose of the end flange plate in the robot base coordinate system is as follows:
E R B ＝R(r z )·R(r y )·R(r x )
E t B ＝[t 1 t 2 t 3 ] T ＝[x y z] T
in the formula (I), the compound is shown in the specification, E T B representing the position and orientation transformation matrix of the tail end flange plate under the robot base coordinate system, wherein E R B Represents the posture of the user, E t B representing a location. r is ij I =1,2,3, j =1,2,3 represents the attitude matrix E R B Element of (1), t i I =1,2,3 represents an element of the position vector. x, y, z, r x ,r y ,r z Represents the pose description of the robot under a world coordinate system, wherein x, y and z represent positions and r x ,r y ,r z Represents a gesture;
the three-dimensional reconstruction equipment is a structured light camera capable of quickly acquiring a surface point cloud; the expression mode of the shot point cloud data in the camera coordinate system is as follows:
O P C ＝[X O Y O Z O ] T
in the formula (I), the compound is shown in the specification, O P C representing point cloud data in a camera coordinate system, where X O ,Y O ,Z O Representing point cloud data coordinate values. Through the calibration of hands and eyes, can buildPose transformation matrix between stereo camera coordinate system and robot end coordinate system C T E And then obtaining the representation of the point cloud data in the robot base coordinate system:
E P O ＝ B T E E T C · C P O
in the formula (I), the compound is shown in the specification, E P O representing a representation of the point cloud data in a robot-based coordinate system. And establishing a complete three-dimensional reconstruction model.
The spray gun is an automatic device which can be combined with six-axis robot for use, and its parameters can be regulated by program, and the spray gun is mounted at the tail end of six-axis robot, and the position-posture change relationship between its spray head and tail end of robot can be recorded as Tool T E And the coordinate system can be obtained by calibrating a TCP tool coordinate system.
Fig. 2 shows a flow chart of the preprocessing and template making. The process comprises the steps of starting from inputting original point cloud data, removing invalid points, down-sampling, radius filtering, statistical filtering and normal line calculation, judging whether using conditions are met or not, if so, extracting a spraying track based on a bounding box method, and if not, stopping processing if the template is unqualified. On the basis of extracting the spraying track based on the bounding box method, a spraying template is formed by the spraying track and the processed point cloud, whether the using condition is met or not is judged, if the using condition is met, the spraying template is stored in a process database, and if the using condition is not met, the spraying template is unqualified, and the spraying template is stopped.
The point cloud preprocessing part comprises invalid point removal, down-sampling, radius filtering, statistical filtering, normal calculation and acquisition of the original point cloud P O The point cloud obtained by the preprocessing is recorded as P O-pre-T And recording the extracted spraying track as Path T The parameter of the painting process corresponding to the painting track is recorded as ζ T And the kinematic parameter of the robot corresponding to the spraying track is recorded as ν T. Accordingly, the expression χ of the spraying template can be obtained Tem :
χ Tem ＝{P O-pre-T ,Path T ,ζ T ,ν T }；
Referring to FIG. 3, a flow chart of the preprocessing and template fabrication of the present invention is shown. The process comprises two processing objects of free-form surface point cloud to be registered and template point cloud; and performing point cloud feature calculation on the free-form surface point cloud to be registered and the template point cloud to obtain three features of a local normal vector, a local radius and local density, and generating a sub-feature histogram to describe the three features of the local normal vector, the local radius and the local density. On the basis, the three sub-feature histograms are spliced to form a total feature histogram.
As shown in fig. 4, a schematic diagram is constructed for the point cloud local feature histogram of the present invention; wherein, fig. 4 (a) represents a local radius feature, fig. 4 (b) represents a local normal vector feature, fig. 4 (c) represents a local density feature, and fig. 4 (d) represents a total feature histogram generated by three local features; the point cloud feature extraction and description process comprises the steps of extracting features of local normal vectors, local radiuses and local densities of the free-form surface point cloud and the template point cloud to be registered, calculating corresponding sub-feature histograms, and describing point cloud features by using the total feature histogram;
the local normal vectors represent the curvature features of the free-form surface, and the curvature features can be deeply expressed by adopting the included angles among the normal vectors as indexes for evaluating the local normal vector features, and the calculation is as follows:
in the formula, con (p) i ) Covariance matrix representing point cloud, where p i Represents any point in the point cloud and,
representing the average point of the point cloud. n is a radical of an alkyl radical i And
represents two different normal vectors, where n i Is a principal normal vector，
Is that n i The neighborhood normal vector of (1), both calculated according to Con (p) i ) The minimum feature value of (2) is obtained corresponding to the unit vector of the feature vector. Theta here j E (0, pi) represents the local normal vector characteristics.
The local radius may represent the curvature change characteristic of the free-form surface, and the local depth dj thereof is calculated as follows:
where r represents the query point neighborhood sphere radius, n i Representing normal vectors, p, at query points i Representing any point in the point cloud (i.e., the query point), p i j Representing neighborhood points of the query point;
the local density can represent the density degree and the point density of the free-form surface point cloud. The calculation method is as follows:
ρ j represents the local density feature, n i Representing normal vectors at query points, p i Representing any point in the point cloud (i.e. the query point),
representing neighborhood points of the query point; on the basis of feature calculation, a statistical method is applied to obtain three sub-histograms. And finally, performing one-dimensional direction splicing on the three sub-histograms to obtain a total histogram to represent the local characteristics of the point cloud.
Please refer to fig. 5, which is a schematic diagram of point cloud registration according to the present invention. The process comprises two processing objects of a free-form surface point cloud to be registered and a template point cloud; and performing point cloud feature histogram calculation and key point query on the free-form surface point cloud and the template point cloud to be registered to obtain a preliminary corresponding point pair, and then performing corresponding relation estimation and optimization to obtain a transformation matrix, wherein the step is coarse registration. On the basis, an ICP algorithm is adopted, iterative registration is carried out, a transformation matrix is calculated, and a final point cloud transformation relation is obtained.
The point cloud registration process comprises the steps of performing feature histogram calculation, key point query, corresponding relation estimation and optimization on the point cloud of the free-form surface to be registered and the point cloud of the template, and performing iterative registration calculation on a transformation matrix;
recording the transformation matrix between the point cloud of free-form surface to be registered and the point cloud of template Tem T raw For solving this matrix, the following objective function is generally constructed for solving:
wherein F (\9633;) represents the objective function, P T-x Representing the point cloud to be registered in the template library, P O Representing the point cloud to be registered, Tem R raw representing a rotational transformation of the two point clouds, Tem t raw representing the translational motion of the two point clouds. Due to uncertainty of point cloud scale and spatial distance, the estimation of the transformation matrix is generally divided into coarse estimation and fine estimation, which are also called coarse registration and fine registration. The former has the function of shortening the distance between two points of cloud data, so that the registration time is greatly saved; the purpose of the latter is to improve the pose transformation matrix precision.
The point cloud registration process comprises the steps of performing feature histogram calculation, key point query, corresponding relation estimation and optimization on free-form surface point cloud and template point cloud to be registered, and iteratively registering and calculating a transformation matrix; and in the process of aligning the spraying template, aligning and transforming the cloud coordinate system of the point to be registered and the point cloud coordinate system of the template according to the transformation matrix. The transformation is as follows:
P O' ＝ Tem T raw ·P O
in the formula P O' Representing the point cloud under the template point cloud coordinate system, P O Representing the original point cloud of the point, Tem T raw representing two coordinate system transformation matrices.
The process of acquiring the spraying track is to call the spraying track indexed correspondingly by the spraying template in the process database on the basis of completing the alignment transformation; namely, finding the spraying track corresponding to the index in the template library, as follows:
in which theta (-) represents the spray trajectory for finding the corresponding template, P O-pre-T Point cloud data representing the corresponding template, path T Representing the spray trajectories of the corresponding index in the template library.
The spray painting parameter and posture adjusting process is that on the basis of finishing alignment transformation, spray painting parameters and robot motion parameters corresponding to indexes of spray painting templates in a process database are called and issued to a six-axis robot to finish spray gun posture adjustment; namely, finding out the paint spraying parameters and the robot motion parameters corresponding to the indexes in the template library as follows:
in the formula
Representing paint spraying parameters and robot motion parameters, P, for finding corresponding templates O-pre-T Point cloud data, ζ, representing corresponding templates T ,ν T Representing paint spraying parameters and robot motion parameters of corresponding indexes in the template library; on the basis, the paint spraying parameters and the robot motion parameters are issued to the robot to complete the spraying action.
The normal calculation is to calculate a spraying point, the process needs to be judged, if the spraying point is qualified, the next processing is continued, and if the spraying point is not qualified, the preprocessing and template manufacturing process is ended;
the spray points are calculated as follows:
in the formula p sprayi Representing the position of the spray point, n sprayi Representing the attitude of the spray gun at the point of application, p i Represents the track point on the free-form surface, h represents the spraying height, n i Representing a normal vector at a locus, \9633 (·) representing regularization of the vector;
and (3) carrying out point cloud feature histogram calculation and key point query on the point cloud of the free-form surface and the point cloud of the template to be registered to obtain a preliminary corresponding point pair, and then carrying out corresponding relation estimation and optimization to obtain a transformation matrix, wherein the step is called coarse registration. On the basis, an ICP algorithm is adopted, iterative registration is carried out, a transformation matrix is calculated, and a final point cloud transformation relation is obtained.
Referring to fig. 6, a flow chart of extracting a spraying trajectory based on a bounding box method according to the present invention is shown. The process comprises the steps of obtaining free-form surface point clouds through pretreatment, calculating a point cloud main direction, calculating a segmentation interval, obtaining a point cloud sequence through segmentation, extracting track points of the point cloud sequence, optimizing track point redundancy, judging whether using conditions are met or not, obtaining a spraying track if the using conditions are met, and stopping processing if the template is not qualified. And finally, forming a spraying template together with the preprocessed point cloud. Calculating the main direction of the point cloud by adopting a PCA method, wherein the calculation division interval is obtained by carrying out equal-distance division according to a boundary plane parallel to the main direction, and the plane sequence is as follows:
wherein PlaneSeq represents a sequence of cleavage planes, plane i Representing the segmentation plane, vector (·) representing an arbitrary vector in the plane, n PCA Representing the PCA vector, d (·,) representing the distance between two parallel planes, δ representing the paint width, n seg Representing the direction of division, d max Representing the maximum distance in the direction of segmentation.
Combining with the segmentation plane, adopting a direct filtering method to obtain the track points on the free curved surface:
P pathi ＝PassT(PlaneSeq)
in the formula P pathi Representing the tracing points on the free-form surface, and PassT (DEG) representing the straight-through filtering; according to the distance condition between two points and the distance condition between three points, the track points after redundancy optimization can be obtained:
P i ＝Opt(P pathi ,d 2con ,d 3con )
in the formula P i Represents the track points on the optimized free-form surface, opt (-) represents the optimization function, d 2con ,d 3con Respectively representing a two-point distance condition and a three-point distance condition; are included within the scope of the invention; wherein, the spraying template needs to be judged, if qualified, the next processing is continued, otherwise, the pretreatment and template manufacturing process is ended, namely, the following form is adopted:
Δ＝con Δ (pre,tem)
wherein Δ represents the condition threshold of the pre-treatment and template fabrication process, con Δ (-) represents a threshold computation function, pre, tem represent the pre-processing and template fabrication processes, respectively; the track point redundancy optimization needs to be judged, if the track point redundancy optimization is qualified, the next processing is carried out, and if the track point redundancy optimization is not qualified, the preprocessing and template manufacturing processes are finished; namely, it is determined by the following form:
▽＝con ▽ (Opt)
where ^ represents the conditional threshold, con, of track point redundancy optimization ▽ (. Cndot.) represents a threshold calculation function.
The preferred embodiments of the invention disclosed above are intended to be illustrative only. The preferred embodiments are not intended to be exhaustive or to limit the invention to the precise embodiments disclosed. Obviously, many modifications and variations are possible in light of the above teaching. The embodiments were chosen and described in order to best explain the principles of the invention and the practical application, to thereby enable others skilled in the art to best utilize the invention. The invention is limited only by the claims and their full scope and equivalents.
Claims (10)
1. A robot automatic spraying method based on point cloud processing is realized by loading a data processing method on a hardware system, and is characterized in that:
the hardware system comprises a six-axis robot, three-dimensional reconstruction equipment, a target free-form surface and a spray gun, and is used for acquiring geometric information of the free-form surface and executing automatic spraying action according to a point cloud processing result and technological parameter requirements;
the data processing method comprises preprocessing, template making, point cloud feature extraction and description, point cloud registration, spraying template alignment, spraying track acquisition, spraying parameter and posture adjustment, is used for processing free-form surface point cloud data and template matching, can update a template database in real time, automatically matches a proper spraying template according to the point cloud features, and guides the configuration of a spraying process and spray gun motion parameters.
2. The robot automatic spraying method based on point cloud processing as claimed in claim 1, characterized in that the three-dimensional reconstruction device and the spray gun are mounted on a six-axis robot; the six-axis robot is an industrial robot having three translational degrees of freedom and three rotational degrees of freedom; the three-dimensional reconstruction equipment is a structured light camera capable of rapidly acquiring the surface point cloud.
3. The method of claim 1, wherein the preprocessing and template preparation process comprises the steps of removing invalid points from the input original point cloud data, down-sampling, radius filtering, statistical filtering, and normal calculation, and extracting a spraying trajectory based on a bounding box method, and storing the spraying trajectory and the processed point cloud in a process database.
4. The method of claim 1, wherein the point cloud feature extraction and description process comprises extracting features of local normal vectors, local radii and local densities of the point cloud of the free-form surface and the template point cloud to be registered, calculating corresponding sub-feature histograms, and describing the point cloud features by using the total feature histogram.
5. The robot automatic spraying method based on point cloud processing as claimed in claim 1, wherein the point cloud registration process comprises feature histogram calculation, key point query, correspondence estimation and optimization of the point cloud of the free-form surface and the point cloud of the template to be registered, and iterative registration calculation of a transformation matrix.
6. The robot automatic spraying method based on point cloud processing as claimed in claim 1, wherein the point cloud registration process is to perform feature histogram calculation, key point query, correspondence estimation and optimization on the point cloud of the free-form surface and the template point cloud to be registered, and iteratively register and calculate a transformation matrix.
7. The method of claim 1, wherein the acquiring of the spraying trajectory is based on the alignment transformation, and the spraying trajectory indexed by the spraying template in the process database is retrieved.
8. The method of claim 1, wherein the paint spraying parameter and attitude adjustment process comprises retrieving paint spraying parameters and robot motion parameters indexed in correspondence with a spray template in a process database based on completion of the alignment transformation, and issuing the paint spraying parameters and robot motion parameters to a six-axis robot to complete the attitude adjustment of the spray gun.
9. The robot automatic spraying method based on point cloud processing as claimed in claim 3, characterized in that the normal calculation is to calculate the spraying point, the process needs to be judged, if qualified, the next processing is continued, otherwise, the pretreatment and template making process is ended; the process of extracting the spraying track based on the bounding box method includes the steps of calculating the main direction of point cloud, calculating and dividing intervals, dividing to obtain a point cloud sequence, extracting track points from the point cloud sequence, optimizing track point redundancy to obtain a spraying track, and finally forming a spraying template together with the preprocessed point cloud.
10. The automatic robot spraying method based on point cloud processing of claim 9, wherein the spraying template needs to be judged, if qualified, the next processing is continued, otherwise, the pretreatment and template making process is ended; and judging operation is needed for the track point redundancy optimization, if the track point redundancy optimization is qualified, the next processing is continued, and otherwise, the preprocessing and template manufacturing process is ended.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210870010.5A CN115283172B (en) | 2022-07-22 | 2022-07-22 | Robot automatic spraying method based on point cloud processing |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202210870010.5A CN115283172B (en) | 2022-07-22 | 2022-07-22 | Robot automatic spraying method based on point cloud processing |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN115283172A true CN115283172A (en) | 2022-11-04 |
| CN115283172B CN115283172B (en) | 2024-05-17 |
Family
ID=83825018
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202210870010.5A Active CN115283172B (en) | 2022-07-22 | 2022-07-22 | Robot automatic spraying method based on point cloud processing |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115283172B (en) |
Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105976312A (en) * | 2016-05-30 | 2016-09-28 | 北京建筑大学 | Point cloud automatic registering method based on point characteristic histogram |
| WO2020134254A1 (en) * | 2018-12-27 | 2020-07-02 | 南京芊玥机器人科技有限公司 | Method employing reinforcement learning to optimize trajectory of spray painting robot |
| CN111915677A (en) * | 2020-07-08 | 2020-11-10 | 哈尔滨工程大学 | Ship pose estimation method based on three-dimensional point cloud characteristics |
| CN112263052A (en) * | 2020-11-13 | 2021-01-26 | 宁波三体智能科技有限公司 | Method and system for automatically mapping vamp glue spraying path based on visual data |
| CN112934518A (en) * | 2021-01-25 | 2021-06-11 | 山东华锐智能技术有限公司 | Automatic spraying device and method based on point cloud |
| CN113304970A (en) * | 2021-05-12 | 2021-08-27 | 深圳群宾精密工业有限公司 | Method for calculating vamp spraying track through 3D point cloud data |
| CN114474041A (en) * | 2021-12-07 | 2022-05-13 | 新拓三维技术(深圳)有限公司 | Welding automation intelligent guiding method and system based on cooperative robot |
-
2022
- 2022-07-22 CN CN202210870010.5A patent/CN115283172B/en active Active
Patent Citations (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN105976312A (en) * | 2016-05-30 | 2016-09-28 | 北京建筑大学 | Point cloud automatic registering method based on point characteristic histogram |
| WO2020134254A1 (en) * | 2018-12-27 | 2020-07-02 | 南京芊玥机器人科技有限公司 | Method employing reinforcement learning to optimize trajectory of spray painting robot |
| CN111915677A (en) * | 2020-07-08 | 2020-11-10 | 哈尔滨工程大学 | Ship pose estimation method based on three-dimensional point cloud characteristics |
| CN112263052A (en) * | 2020-11-13 | 2021-01-26 | 宁波三体智能科技有限公司 | Method and system for automatically mapping vamp glue spraying path based on visual data |
| CN112934518A (en) * | 2021-01-25 | 2021-06-11 | 山东华锐智能技术有限公司 | Automatic spraying device and method based on point cloud |
| CN113304970A (en) * | 2021-05-12 | 2021-08-27 | 深圳群宾精密工业有限公司 | Method for calculating vamp spraying track through 3D point cloud data |
| CN114474041A (en) * | 2021-12-07 | 2022-05-13 | 新拓三维技术(深圳)有限公司 | Welding automation intelligent guiding method and system based on cooperative robot |
Also Published As
| Publication number | Publication date |
|---|---|
| CN115283172B (en) | 2024-05-17 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110227876A (en) | Robot welding autonomous path planning method based on 3D point cloud data | |
| CN108876852B (en) | Online real-time object identification and positioning method based on 3D vision | |
| CN112907735B (en) | Flexible cable identification and three-dimensional reconstruction method based on point cloud | |
| CN113421291B (en) | Workpiece position alignment method using point cloud registration technology and three-dimensional reconstruction technology | |
| Lin et al. | Robotic grasping with multi-view image acquisition and model-based pose estimation | |
| CN111986219B (en) | Matching method of three-dimensional point cloud and free-form surface model | |
| CN111530671A (en) | Intelligent robot spraying method based on spraying system | |
| CN113781561B (en) | Target pose estimation method based on self-adaptive Gaussian weight quick point feature histogram | |
| CN113920061A (en) | Industrial robot operation method and device, electronic equipment and storage medium | |
| CN114055255A (en) | Large-scale complex component surface polishing path planning method based on real-time point cloud | |
| CN107563432A (en) | A kind of robot multiple-objective recognition methods of view-based access control model shape | |
| CN115685988A (en) | Method and device for automatically generating free-form surface spraying path | |
| CN115937468A (en) | Automatic generation method for machining program of countless-module robot | |
| Ma et al. | An efficient and robust complex weld seam feature point extraction method for seam tracking and posture adjustment | |
| Cheng et al. | Trajectory planning method with grinding compensation strategy for robotic propeller blade sharpening application | |
| CN115283172B (en) | Robot automatic spraying method based on point cloud processing | |
| CN116604212A (en) | Robot weld joint identification method and system based on area array structured light | |
| CN108469729B (en) | Human body target identification and following method based on RGB-D information | |
| Filaretov et al. | An new approach for automatization of cutting of flexible items by using multilink manipulators with vision system | |
| Filaretov et al. | Method of combination of three-dimensional models of details with their CAD-models at the presence of deformations | |
| CN115971004A (en) | Intelligent putty spraying method and system for carriage | |
| Yang et al. | Research on the feature smoothing algorithm for point cloud data of large complex surfaces based on multichannel convolutional neural network | |
| Dong et al. | Pose estimation of components in 3c products based on point cloud registration | |
| Wu et al. | A novel approach for porcupine crab identification and processing based on point cloud segmentation | |
| CN114782617A (en) | Method and system for high-precision automatic glue spraying based on machine vision |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |