WO2019010916A1

WO2019010916A1 - Workpiece three-dimensional point cloud data smoothing method

Info

Publication number: WO2019010916A1
Application number: PCT/CN2017/116107
Authority: WO
Inventors: 敬淑义; 尹章芹; 毛东辉; 王杰高
Original assignee: 南京埃斯顿机器人工程有限公司
Priority date: 2017-07-10
Filing date: 2017-12-14
Publication date: 2019-01-17
Also published as: CN107274376A

Abstract

A workpiece three-dimensional point cloud data smoothing method. A Savitzky-Golay algorithm is utilized for workpiece three-dimensional point cloud data smoothing processing. The algorithm employs a method for combined processing of a grayscale image and a point cloud image, reduces the data volume of raw data processing, and increases the amount of time for processing data. A polynomial fitting method is utilized for smoothing point cloud data and retaining the geometric invariance of a point cloud.

Description

Three-dimensional point cloud data smoothing filtering method for workpiece

Technical field

The invention relates to a method for processing three-dimensional point cloud data of a workpiece, in particular to a three-dimensional point cloud data smoothing filtering method for a workpiece. It can be combined with robots in the industrial field to obtain relevant information of workpieces, and belongs to the field of industrial robot vision applications.

Background technique

The vision system works with the robot to become the "hand-eye system." In the modern intelligent industrial application, the "hand-eye system" is used to replace the production line workers to complete the production line flow operation, production line assembly, or use the visual system to complete the cumbersome work of material measurement, quality inspection, etc., to improve the robot's perception and adaptability to the environment. With the visual control of the robot, it is not necessary to teach or offline programming the movement track of the industrial robot in advance, which can save a lot of programming time, improve production efficiency and processing quality, and liberate labor from production labor to a certain extent.

Stereoscopic vision is an important topic in the field of computer vision. Its purpose is to reconstruct the three-dimensional geometric information of the scene. Stereoscopic research has important application value, including autonomous navigation systems for mobile robots, aerospace and remote sensing measurements, and industrial automation systems.

There are many methods for acquiring three-dimensional information based on sensors, which are mainly divided into active and passive. In the industrial field, active three-dimensional methods are generally used to obtain depth information, that is, stereoscopic depth information is acquired by means of structured light or optical coding, and the stereoscopic depth information is obtained. It also becomes a three-dimensional point cloud information.

After acquiring the 3D point cloud data, the point cloud data needs to be filtered. The current point cloud filtering algorithms mainly include: LIDAR data morphology filtering algorithm, filtering algorithm based on slope change, and the like. The main problem of the LIDAR data morphology filtering algorithm is the artificial selection of the slope threshold and the square effect of the detailed terrain. If the threshold is set too large, the noise may be smoothed out, and the details of the workpiece are smoothed, and the general applicability of the algorithm is poor. The principle of the filtering algorithm based on the slope change is to determine the optimal filter function according to the contour of the workpiece. Although the algorithm is simple, it is necessary to know the contour change information of the workpiece and the set window information in advance, and the applicability of the algorithm is poor.

Summary of the invention

The object of the present invention is to overcome the defects of the prior art and provide a three-dimensional point cloud data smoothing filtering method for a workpiece. The method of the invention is based on Savitzky-Golay filtering, which eliminates the maturity of the original data, improves the quality of the point cloud processing data, and can more accurately reflect the original appearance of the object.

The invention provides a three-dimensional point cloud data smoothing filtering method for a workpiece, comprising the following steps:

Step 1: Collect point cloud data of the workpiece

Based on the shape of the workpiece, a perspective is selected to collect the 3D contour point cloud data of the workpiece.

Step 2: The collected workpiece point cloud data is obtained according to the three-dimensional coordinate value of the point, and the point cloud is projected onto the viewing plane.

Step 3: Obtain a projected grayscale image, and perform binarization threshold processing on the grayscale image to obtain a binary image.

Step 4: Open and close the binary image.

Step 5: Remove image noise with Savitzky-Golay filtering

1) Determine the values of M and N. N represents the number of points in the field, and M represents the number of times of the polynomial. The value of N is the same. The smaller the M value is, the better the smoothing effect is. However, in order to ensure that the filtered point cloud and the origin cloud are not much different, M=3 is generally taken. M is the same, the larger N is, the more the detail is lost, so the values of M and N are determined by the correlation of curvature.

(1) Calculate the curvature C _i (i = 1, 2, ..., n) of the point cloud and the curvature derivative dC _i (i = 1, 2, ..., n-1) using M-time spline interpolation, n represents the total number of point clouds.

(2) Let N _max = n / 4, N _min = 5; if (n / 4) < 5 is satisfied, then N = 5, and if the number of point clouds n is less than 5, the processing is terminated.

(3) Determine the value of N; if the curvature value of a point C _i <Q ₁ , then it can be considered that this point and its field point are in a straight line, and then find the point in the field that satisfies dC _i <Q ₂ and determine After the point, if n ₁ >n ₂ , then N=2n ₁ +1; otherwise: N=2n ₂ +1, if the curvature value of a point C _i >Q ₁ , then the curvature change dC _{i of} the point is in its domain Looking for satisfaction

or

Point, where Q, Q ₁ , Q ₂ are empirical values of the curvature segmentation threshold, if n ₁ >n ₂ , then N=2n ₁ +1; otherwise: N=2n ₂ +1, determining N and M value. n ₁ represents the number of points whose curvature is smaller than and close to Q ₂ , and n ₂ represents the number of points whose curvature value is larger than and close to Q ₂ .

2) After determining the values of N and M, the Savitzky-Golay filter is used to remove the noise of the image. For the N points in the field of any point x _i , the M-order polynomial is fitted, and the least squares is used to determine the coefficients of the polynomial. Polynomial The value at x _i is the smooth value g _i of the corresponding point; N is greater than the order M of the polynomial; the M-order polynomial p _i (x) fitted by the scan data x _{i is} expressed as:

Where x represents any point in the N point clouds other than the point x _i .

Assuming that x _i+1 -x _i =Δx for any x _i , the fitting polynomial needs to calculate the coefficient b _k in (1-1) to make it optimal, ie:

Matrix representation of the coefficients of b _k :

Other parameters are represented by vectors as:

with

Y represents the filter value,

Filter the values of in ₁ , point i, and point ₂ respectively.

From equations (1-3) and (1-4), equation (1-2) can be written as:

Equation (1-5) can be expressed as:

A ^T Ab _k =A ^T Y (1-6)

Since A ^T A is a positive definite matrix and there is an inverse matrix, the coefficient b _k :

b _k =(A ^T A) ^-1 A ^T Y (1-7)

Step 6: The watershed segmentation recognition method is used for segmentation and recognition, and the image region is divided into several parts to obtain the largest region, and the color map after denoising is obtained based on the color map of the maximum region cropping.

Step 7: Calculate the three-dimensional coordinate values of the points in the point cloud data according to the points in the color map to obtain the denoised point cloud.

Step 8: Determine the noise condition in the point cloud data. If there is, change the angle of view, filter again. If not, output the result, the advantages of the present invention compared to other algorithms:

1. Keep the geometric features unchanged (algorithm filtering processing, using the fitting algorithm to remove the noise in the image and keep the geometric characteristics of the workpiece unchanged);

2. The accuracy and speed are higher than other operators; (based on the grayscale image, the point cloud data of the casting is obtained, and the amount of data calculation is reduced, plus Faster data processing speed, using the method of fitting calculation, to provide the accuracy of point cloud computing. )

3. Applicability is strong (data fitting calculation, suitable for different castings, strong adaptability).

DRAWINGS

Figure 1 is a flow chart of the Savitzky-Golay algorithm.

2 is a block diagram of a method for smoothing and filtering a three-dimensional point cloud data of a workpiece of the present invention.

Figure 3 is a grayscale diagram of a workpiece sample.

Figure 4 is an effect diagram before Savitzky-Golay filtering.

Figure 5 is an effect diagram of Savitzky-Golay filtering.

Detailed ways

The present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

Embodiments, the present invention uses a steel pipe casting as a workpiece, and specifically describes a three-dimensional point cloud data smoothing filtering method of the workpiece proposed by the present invention, as shown in FIG. 2, and the specific implementation process is as follows:

Step 1: Load the point cloud data of the workpiece, and use vc to read the point cloud data of the iron pipe casting on the vs2013 software platform.

Step 2: Select a projection direction, call the pcl (point cloud library) point cloud algorithm library, use the greedy projection triangulation method, calculate the point cloud normal vector, and put the normal vector and the point cloud coordinates together to determine the point cloud projection. Angle and direction.

Step 3: Load the grayscale image, as shown in Figure 3. The read grayscale image is loaded on the vs2013 software platform, and the grayscale image is binarized.

Step 4: The opening and closing operation of the binary image is first expanded and then etched by a circular structure. Create a circular structure with a radius of 3, use a circular structure to etch the binary image, and then use a circular structure to swell the etched image to obtain a processed binary image to determine the data area to be processed. .

Step 5: Remove the noise of the image with Savitzky-Golay filtering.

Using Savitzky-Golay filter needs to first determine several parameters: the number of polynomial fitting M, the total number of points in the field of point X _i N, the number of points on either side of the point X _i n ₁ and n _2. In order to achieve a better level of accuracy and speed of calculation, M generally takes 2, 3, 4. This embodiment M=3,

N cannot be set too large, otherwise it will affect the speed of processing data, but it can not affect the accuracy of the fitting, and N is the base, so set the value of N to 47.

Based on the principle of vector value consistency in the directions of the parties, let n ₁ = n ₂ .

After determining the values of N and M, the Savitzky-Golay filter is used to remove the noise of the image:

For the N=47 points in the field of any point x _i , the M=3 order polynomial is fitted, and the least squares is used to determine the coefficient of the polynomial. The value of the polynomial at x _i is the smooth value g _i of the corresponding point. The M=3rd degree polynomial p _i (x) fitted by the scan data x _i can be expressed as:

Matrix representation of the coefficients of b _k :

Other parameters are represented by vectors as:

with

From equations (1-3) and (1-4), equation (1-2) can be written as:

Equation (1-5) can be expressed as:

A ^T Ab _k =A ^T Y (1-6)

b _k =(A ^T A) ^-1 A ^T Y (1-7)

Figure 4 is a filtered effect diagram of Figure 5.

Step 6: Using the watershed segmentation identification method to perform segmentation and segmentation, divide the image region into several parts, obtain the largest region, process the filtered image, and obtain the maximum effective region.

Step 7: Calculate the three-dimensional coordinate values of the points in the point cloud data according to the points in the gray scale image to obtain the denoised point cloud. Reverse the three-dimensional coordinates of the point cloud to obtain the denoised point cloud.

Step 8: Determine if the denoising is acceptable. If it is qualified, output point cloud. If it is not qualified, select one again to clear The angle of the contour of the object is filtered again.

Table 1 shows the comparison of the algorithm in this paper with the commonly used threshold filtering algorithm in time.

Table 1

点云数据量Point cloud data volume	阈值滤波平均花费的时间(s)Average time spent on threshold filtering (s)	本文算法的平均花费的时间(s)The average time spent on the algorithm (s)
1942019420	124124	3030
199013199013	508508	131131
798006798006	12501250	240240
19015801901580	3345033450	387387

The experimental data is obtained on a CPU with a CPU of 3.0 GHz, a memory of 4 G, and a system of win7.

The meanings of the following nouns in the context of the present invention are:

Watershed segmentation identification method: a mathematical morphology segmentation method based on topological theory. The basic idea is to regard the image as a geomorphological topological feature. The gray value of each pixel in the image indicates the altitude of the point. Each local minimum and its affected area are called catchment basins, while the boundary of the catchment basin forms a watershed. The concept and formation of the watershed can be illustrated by simulating the immersion process. On each local minimum surface, pierce a small hole, and then slowly immerse the entire model in the water. As the immersion deepens, the influence of each local minimum is slowly expanded outward, in two sets. The faucet at the confluence of the basin forms a watershed and divides the image into different areas.

The greedy projection triangulation method: firstly project the directed point cloud into a local coordinate plane, and then perform intra-plane triangulation in the coordinate plane, and obtain a triangular mesh surface model according to the topological relationship of the three points in the plane.

Opening and closing operation of the binary image by first expanding and then etching with a circular structure: taking a circular as a structural element, performing intersection and equal set operations on the region corresponding to the binary image at each pixel position, first etching The process of expansion is called the open operation. Corrosion is a process of eliminating boundary points and shrinking the boundary to the inside. Can be used to eliminate small and meaningless objects. Swelling is the process of merging all background points that come into contact with an object into the object, and expanding the boundary to the outside, which can be used to fill the voids in the object.

Claims

A three-dimensional point cloud data smoothing filtering method for workpieces, comprising the following steps:

Step 1. Collect point cloud data of the workpiece

Based on the shape of the workpiece, a perspective is selected to collect the three-dimensional contour point cloud data of the workpiece;

Step 2. Acquire the corresponding point cloud data according to the three-dimensional coordinate value of the point, and project the point cloud onto the viewing plane;

Step 3. Obtain a projected grayscale image, and perform a binarization threshold processing on the grayscale image to obtain a binary image;

Step 4. Perform an open and close operation on the binary image;

Step 5. Remove image noise with Savitzky-Golay filtering

1) Determine the values of M and N, where N is the number of points in the field, and M is the number of polynomials:

(1) Calculate the curvature C i (i = 1, 2, ..., n) of the point cloud and the curvature derivative dC i (i = 1, 2, ..., n-1) using M-time spline interpolation, n represents the total number of point clouds;

(2) Let N max = n / 4, N min = 5; if (n / 4) < 5, then N = 5, if the number of point clouds n is less than 5, the process ends;

(3) Determine the value of N; if the curvature value of a point C i <Q 1 , then it can be considered that this point and its field point are in a straight line, and then find the point in the field that satisfies dC i <Q 2 and determine After the point, if n 1 >n 2 , then N=2n 1 +1; otherwise: N=2n 2 +1; if the curvature value of a point C i >Q 1 , then the curvature change dC i of the point is in its domain Looking for satisfaction
or
Point, where Q, Q 1 , Q 2 are empirical values of the curvature segmentation threshold, if n 1 >n 2 , then
Otherwise: N = 2n 2 +1; determine the values of N and M; n 1 represents the number of points whose curvature is less than and close to Q 2 , and n 2 represents the number of points whose curvature value is greater than and close to Q 2 ;

2) After determining the values of N and M, remove the noise of the image with a Savitzky-Golay filter

For the N points in any field x i domain, the M-order polynomial is fitted, and the least squares is used to determine the coefficients of the polynomial. The value of the polynomial at x i is the smooth value g i of the corresponding point; N is greater than the polynomial The order M; the M-order polynomial p i (x) fitted by the scan data x i is expressed as:

b k =(A T A) -1 A T Y

Where x represents any point in the N point clouds other than the point x i ; assume that for any x i there is x i+1 -x i =Δx, Y represents the filtered value,
Filter the values of in 1 , point i, and point i+n 2 respectively;

Step 6. Using the watershed segmentation identification method to segment the region, divide the image region into several parts, obtain the largest region, and obtain the color map after denoising based on the color map of the maximum region cropping;

Step 7. Calculate the three-dimensional coordinate values of the points in the point cloud data according to the points in the color map to obtain the denoised point cloud;

Step 8. Determine the noise condition in the point cloud data. If there is noise, change the angle of view, filter again. If there is no noise, output the last point cloud data.
The workpiece 3D point cloud data smoothing filtering method according to claim 1, wherein in step 5, M=3.