CN111539446A - 2D laser hole site detection method based on template matching - Google Patents

2D laser hole site detection method based on template matching Download PDF

Info

Publication number
CN111539446A
CN111539446A CN202010143880.3A CN202010143880A CN111539446A CN 111539446 A CN111539446 A CN 111539446A CN 202010143880 A CN202010143880 A CN 202010143880A CN 111539446 A CN111539446 A CN 111539446A
Authority
CN
China
Prior art keywords
hole
point cloud
template
point
points
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010143880.3A
Other languages
Chinese (zh)
Other versions
CN111539446B (en
Inventor
田威
庄志炜
石瀚斌
廖文和
张霖
李波
胡俊山
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nanjing University of Aeronautics and Astronautics
Original Assignee
Nanjing University of Aeronautics and Astronautics
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nanjing University of Aeronautics and Astronautics filed Critical Nanjing University of Aeronautics and Astronautics
Priority to CN202010143880.3A priority Critical patent/CN111539446B/en
Publication of CN111539446A publication Critical patent/CN111539446A/en
Application granted granted Critical
Publication of CN111539446B publication Critical patent/CN111539446B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G06V10/75—Organisation of the matching processes, e.g. simultaneous or sequential comparisons of image or video features; Coarse-fine approaches, e.g. multi-scale approaches; using context analysis; Selection of dictionaries
    • G06V10/751—Comparing pixel values or logical combinations thereof, or feature values having positional relevance, e.g. template matching
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G01B11/24—Measuring arrangements characterised by the use of optical techniques for measuring contours or curvatures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]

Abstract

The invention discloses a 2D laser hole site detection method based on template matching, belongs to the technical field of precision measurement, and can solve the problem that hole site identification and positioning on non-ideal scanning surfaces of high-light-reflection metal, strong-light-absorption composite materials and the like are difficult. The method comprises the following steps: combining a 2D laser profiler with a servo motor to obtain three-dimensional point cloud of the surface of a hole area of a workpiece; removing hole edge outliers by using an improved median filtering algorithm, and carrying out binarization processing on a point cloud z coordinate value; and self-adaptively constructing a template of the hole to be detected according to the point cloud density, executing template matching, judging hole characteristics by using an ROI (region of interest), finally detecting the position of the hole, and fitting a plane by using the point cloud of the hole neighborhood to calculate the normal direction of the hole. The invention can effectively detect the holes on the surface which is not ideal to be scanned, has strong robustness and meets the requirement of measurement precision.

Description

2D laser hole site detection method based on template matching
Technical Field
The invention belongs to the technical field of precision measurement, and particularly relates to a 2D laser hole site detection method based on template matching.
Background
In the digital design and manufacturing process of products, a numerical control system processes according to a theoretical mathematical model of the products, but due to errors in various aspects such as processing, manufacturing and tool positioning, the consistency between a processed workpiece and a theoretical digital model of the processed workpiece cannot be ensured, so that if a processing hole position is positioned only by the theoretical digital model, the precision cannot meet the assembly requirement, and in the actual processing process, the position of the processing hole position needs to be corrected according to the actual installation position of the workpiece. For example, in the automatic assembly of aircraft components, a plurality of groups of datum holes are usually formed in the surface of a product and used as references for positioning the product, and the actual positions of the datum holes need to be automatically measured before machining, so that the aircraft components are accurately positioned. Accurate identification and positioning of fiducial holes on a product is one of the key technologies and difficulties of automated assembly systems. Currently, most equipment adopts an industrial camera or a 2D laser profiler to detect hole sites.
The hole site detection technology based on monocular vision is mature and easy to realize, but is only suitable for detecting the plane position of a hole, the detection precision in the depth direction is not high, and the axis direction of the hole (namely the normal direction of the hole) cannot be measured. And is not suitable for many working conditions. There are some hole site detection technologies based on 2D laser scanning. The 2D laser profilometer is arranged on the linear motion mechanism to obtain the three-dimensional point cloud around the hole, and the position and normal information of the hole can be calculated by processing the point cloud to a certain degree. For example: the method comprises the following steps of scanning by using a 2D laser profiler to obtain point clouds of a plane to be measured, and setting a distance threshold value between all point cloud data points and a fitting plane to achieve the purpose of filtering hole edge noise points, wherein the scheme is only applicable to planes and not applicable to curved surfaces; and the method can be applied only when the 2D laser profiler can scan ideal point clouds, and when the point clouds have flaws, hole features cannot be well identified. For another example: the method is based on a reference hole detection method combining line laser scanning and image processing, three-dimensional point cloud data obtained by line laser scanning are converted into a two-dimensional gray scale image, and an edge detection operator is used for extracting a contour, so that the central position of a reference hole is obtained.
Disclosure of Invention
The invention aims to provide a 2D laser hole site detection method based on template matching, which can solve the problem that hole site identification and positioning on non-ideal scanning surfaces such as highly reflective metal, strong light-absorbing composite materials and the like are difficult, and improve the robustness of hole site detection.
In order to achieve the above purpose, the embodiment of the invention adopts the following technical scheme:
acquiring a point cloud S of a hole site area to be detected through a 2D laser profiler, wherein the 2D laser profiler is carried on a linear servo motor; preprocessing the collected point cloud S, and removing invalid points and outliers of the hole area to be detected; after the binarization processing is carried out on the z coordinate value of each point of the point cloud S, a hole template T (m, n) is constructed according to the density of the point cloud S, and template matching is carried out; aiming at the point cloud S after template matching, the characteristic of the hole to be detected is judged through the ROI sensitive interest area; and acquiring the position of the hole to be measured by using the characteristic discrimination result of the hole to be measured, and acquiring the normal direction of the hole to be measured.
In this embodiment, a 2D laser profiler is selected as a detection device, and a scheme for detecting 2D laser hole locations based on template matching is provided, where the 2D laser profiler is first used to scan and obtain initial point cloud information of a plane to be detected, then the obtained point cloud is subjected to preprocessing such as filtering and noise reduction, and then a z-coordinate value of the point cloud is binarized, a hole template is adaptively constructed and template matching is performed to identify hole features, and finally the position and direction of a hole are calculated. Compared with the prior art, the embodiment of the invention can realize that:
1) the template matching method is adopted to identify holes in the three-dimensional point cloud, so that the hole identification rate is improved;
2) the hole template self-adaptive construction algorithm can self-adaptively construct a hole template according to the point cloud density obtained by actual scanning, so that the matching degree of the hole is improved;
3) the three-dimensional point cloud is subjected to binarization processing, so that the calculation of template matching is simplified, and the matching efficiency is improved;
4) the actual position of the hole is calculated by adopting an averaging method, so that the position measurement precision of the hole is improved;
5) the point cloud in the neighborhood of the fitting hole is decomposed by adopting SVD, and the normal direction of the hole is replaced by the normal direction of the neighborhood plane, so that the normal direction measurement of the hole is realized, the influence of hole edge noise points on the normal direction measurement of the hole is reduced, and the normal direction measurement precision of the hole is improved;
6) and the ROI is utilized to judge the hole characteristics, so that the recognition accuracy of the hole characteristics is improved.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings needed to be used in the embodiments will be briefly described below, and it is obvious that the drawings in the following description are only some embodiments of the present invention, and it is obvious for those skilled in the art that other drawings can be obtained according to the drawings without creative efforts.
FIG. 1 is a flow chart of a 2D laser hole site detection method based on template matching according to the present invention;
FIG. 2 is a schematic view of hole site scanning;
FIG. 3 is a point cloud obtained by scanning the surface of a workpiece to be measured;
FIG. 4 is a point cloud after pre-processing;
FIG. 5 is a point cloud after z-coordinate binarization;
FIG. 6 is a hole template of a self-adapting construction;
FIG. 7 is a schematic diagram of template matching;
FIG. 8 shows the measurement results of the hole to be measured;
FIG. 9 is a hole normal measurement error;
FIG. 10 is a comparison of hole center distance measurement errors for two hole site detection algorithms;
FIG. 11 is a comparison of the point cloud scanned before and after spraying developer;
in the drawings, the numerical references indicate respectively: 2D laser profile instrument-1, workpiece-2 to be measured.
Detailed Description
In order to make the technical solutions of the present invention better understood, the present invention will be described in further detail with reference to the accompanying drawings and specific embodiments. Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like reference numerals refer to the same or similar elements or elements having the same or similar function throughout. The embodiments described below with reference to the accompanying drawings are illustrative only for the purpose of explaining the present invention, and are not to be construed as limiting the present invention. As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "comprises" and/or "comprising," when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and/or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and/or groups thereof. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may also be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or coupled. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items. It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. It will be further understood that terms, such as those defined in commonly used dictionaries, should be interpreted as having a meaning that is consistent with their meaning in the context of the prior art and will not be interpreted in an idealized or overly formal sense unless expressly so defined herein.
With the large-area use of composite materials on airplanes, the automatic drilling and riveting system puts higher requirements on hole site detection. When the 2D laser scanning hole is adopted, due to the strong light absorption characteristic of the composite material, the cloud of the surface point to be detected obtained by scanning has large-area flaws, the traditional hole position detection algorithm based on edge detection cannot be used on the composite material airplane component, in addition, the current hole position detection technology based on the 2D laser scanning can only be used for hole position measurement, and cannot be used for measuring the normal direction of the hole, and a new algorithm must be adopted to effectively detect the position and the normal direction of the hole.
In this embodiment, a 2D laser profiler is selected as a detection device, and a scheme for detecting 2D laser hole locations based on template matching is provided, as shown in fig. 1, first, initial point cloud information of a plane to be detected is obtained by scanning the 2D laser profiler, then, preprocessing such as filtering and noise reduction is performed on the obtained point cloud, then, a z-coordinate value of the point cloud is binarized, a hole template is adaptively constructed, template matching is performed to identify hole features, and finally, the position and normal direction of a hole are calculated. The specific logic comprises:
and acquiring a point cloud S of the hole site area to be detected through a 2D laser profiler.
And preprocessing the collected point cloud S, and removing invalid points and outliers of the hole area to be detected.
And after the binarization processing is carried out on the z coordinate value of each point of the point cloud S, constructing a hole template T (m, n) according to the density of the point cloud S and carrying out template matching. And (4) performing feature discrimination on the hole to be detected through the ROI aiming at the point cloud S after template matching.
And acquiring the position of the hole to be detected by using the characteristic discrimination result of the hole to be detected, and acquiring the normal direction of the hole to be detected.
Wherein, the 2D laser profile appearance is carried on linear servo motor. Each point in the point cloud S corresponds to a spatial three-dimensional coordinate value (x, y, Z), where the Z coordinate value is a coordinate value in the Z direction, the Z direction is a laser emission direction, the hole template T (m, n) is a matrix of m × n, m and n respectively represent the length and width of the matrix, and m and n are both positive integers. For example: each point in the cloud has a spatial three-dimensional coordinate value (x, Y, Z), which is a coordinate value in the Z direction (laser emission direction) and a Y coordinate value in the Y direction (laser profile direction). For convenience of description, the point cloud mentioned in the present embodiment is denoted by a character S, and may be referred to as the point cloud S in the present embodiment.
In this embodiment, the preprocessing the collected point cloud S includes: and obtaining isolated points and outliers in the point cloud S by using the filter F.
The kernel B of the filter F is rectangular, and the size of the kernel B is 15 multiplied by 3. And setting the z coordinate values of the isolated point and the outlier to zero. The filter F mentioned in this embodiment is a 15 × 3 two-dimensional matrix, also called convolution kernel. An image is filtered, i.e. for each pixel of the image, the product of its neighbourhood pixels and the corresponding elements of the filter matrix is calculated and then added up as the value of the pixel position. This completes the filtering process. Where 15 x 3 refers to the number of rows and columns, i.e. a matrix of 15 rows and 3 columns.
Specifically, the obtaining of the isolated points and the outliers in the point cloud S by using the filter F includes: and setting the y coordinate value of the invalid point to be 0, and then creating the filter F based on the median filtering principle. And carrying out convolution on the point cloud S and the inner core B, and scanning each point of the point cloud S one by one. For the scanned point P of the ith row and the jth column, all valid points in a 15 × 3 neighborhood with the point P as the center are extracted from the point cloud S.
Obtaining Z0(i, j) and the median z of the z-coordinate values of the effective pointsmAbsolute value Z of the difference ofd＝|Z0(i,j)-zmAnd if m is less than or equal to 10, judging as an isolated point. When Z isdNot less than 0.05mmAnd judging as the outlier.
The z-coordinate values of the invalid points are all 0, the z-coordinate values of the valid points are not zero, the number of the valid points in the neighborhood is m, and i and j are positive integers.
For example: setting the y coordinate value of the invalid point to 0 by using the condition that the z coordinate values of the invalid point are all 0:
Figure BDA0002400038510000061
creating a filter F based on the median filtering principle, wherein a kernel B of the filter F is rectangular, and the size of the kernel B is 15 × 3. convolving the point cloud S with the kernel B, scanning each point of the point cloud S one by one, for example, for the scanned point P in the ith row and the jth column, taking out all effective points (namely points with a z coordinate value not zero) in a 15 × 3 neighborhood with the point P as the center from the point cloud S, counting the number m of the effective points in the neighborhood, and counting the median z of the z coordinate values of the effective pointsmThen, Z is obtained0(i, j) and zmAbsolute value Z of the difference ofd＝|Z0(i,j)-zmAnd if m is less than or equal to 10, judging the point as an isolated point. When Z isdWhen the size is more than or equal to 0.05mm, the point is judged as an outlier. The zeroing the z-coordinate values of the outliers and the outliers comprises: the working process of the filter F is a process of setting the z-coordinate values of the isolated points and the outliers to zero, as follows:
Figure BDA0002400038510000071
further, this embodiment further includes:
after each point of the point cloud S is scanned one by one, a z coordinate value of a point whose z coordinate value is not 0 is set to 1.
Scanning the z coordinate of the point cloud S point by point, and setting the z coordinate of the point to 1 when the z coordinate is not 0, for example:
Figure BDA0002400038510000072
constructing a pore template T (m, n) according to the density of the point cloud S and performing templateMatching, including: distances Δ X and Δ Y of adjacent acquisition points in the X-direction and Y-direction are obtained, respectively. According to the row number H of the point cloud contained in the hole to be detectedrowAnd the number of columns HcolAdjusting the size of the template T (m, n) of the hole to be detected to be an odd number, constructing a matrix m × n with all elements being 1, then setting the point with the distance from the center point of the template T (m, n) being smaller than the radius of the hole to be detected to be 0, obtaining the final matrix form of the template T (m, n) of the hole to be detected, executing template matching, and obtaining the template T (m, n) and a subgraph SijThe difference of (m, n).
Wherein Δ X ═ X (X)0t-X0R)/(col-1)，Δy＝(Y0U-Y0D)/(row-1)X0LIs the X coordinate value, X, of the first row point cloud0RThe x value and Y value of the last row of point clouds0UIs the average value of non-zero Y values of the first row of point clouds, Y0DThe average value of the non-zero y values of the last row of point clouds, col the number of columns of the point clouds, and row the number of rows of the point clouds. The template T (m, n) of the hole to be measured is in the shape of an outer square and an inner circle, the size of the outer square is larger than that of the inner circle, and Hrow＝D/Δy、Hcol＝D/Δx、m＝Hrow+6、n＝Hcol+2 and D are the diameter of the hole to be measured. Sub-figure Sij(m, n) is the area of the template T (m, n) covering the point cloud S, i, j is the subgraph SijAnd (m, n) coordinate values of the center on the searched point cloud S, and when the difference value is at the minimum value, the position of the matched target hole is determined.
At X0L、Y0U、X0RIn the isoparametric, 0 in the lower corner mark is used for identifying point cloud data in an unprocessed state; l \ R \ U \ D is the first acronym of Left \ Right \ Up \ Down, which respectively refers to the Left, Right, upper and lower positions of the point cloud.
The size of the template is adjusted to an odd number, such as:
Figure BDA0002400038510000081
performing template matching, computing template T (m, n) and subgraph SijThe difference of (m, n) is represented by the formula:
Figure BDA0002400038510000082
the minimum value of E (i, j) is the position of the matched target hole. The acquisition method of the matching rate K of template matching is as follows:
Figure BDA0002400038510000083
where E (i, j) is the template T (m, n) and the sub-map S at the (i, j) positionij(m, n) bias values, i.e. template T (m, n) and subgraph SijAnd (m, n) subtracting and summing corresponding elements in (m, n).
Further, the feature discrimination of the hole to be measured through the ROI region of interest includes:
setting the central circular area of the hole template T (m, n) to be detected as an ROI (region of interest), and acquiring the matching rate K of the ROIr. When the template matching rate K is more than KtROI area matching ratio Kr＞KrtThen, the measured area is determined as a hole feature, where KtAs template matching rate threshold, KrtIs the threshold value of the ROI area matching rate, K is the matching rate of template matching,
Figure BDA0002400038510000084
the obtaining of the normal direction of the hole to be measured includes: using subgraph SijThe coordinate C (x) of the center of the circle is obtained by the average value of the coordinate values of all the points on (m, n)C,yC,zC). Using subgraph Sij(M, n) fitting a plane to the points, calculating a singular vector M (a, b, c) corresponding to the minimum singular value of the SVD transform of the covariance matrix, and using M (a, b, c) as the normal vector of the hole to be measured, wherein a space vector can be represented by 3 coordinate values, and the singular vector is represented by (a, b, c). Wherein the content of the first and second substances,
Figure BDA0002400038510000091
n is subgraph Sij(m, n) number of acquisition points xC、yCAnd zCCoordinate values, S, on the x, y and z axes, respectively, of the centre of the circleij(m, n) represents the subgraph.
In this embodiment, the method further includes: and when the z coordinate values of all the points in the first row of the point cloud S are all 0, determining that the upper edge of the hole area to be measured exceeds the scanning measurement range. And when the z coordinate values of all the points in the last row of the point cloud S are all 0, judging that the lower edge of the hole area to be measured exceeds the scanning measurement range.
Further, the method also comprises the following steps: after the point cloud S is obtained through scanning, firstly, rough template matching is carried out, and the rough position P of the hole is obtained through identificationC. Then to PCAnd carrying out filtering pretreatment on the point cloud in the neighborhood, and finally carrying out template fine matching to obtain the accurate position P of the hole. When the rough matching of the template is carried out, the matching step length is 10, and the matching calculation is executed once every 10 points. When the template is finely matched, the matching step length is 1, and the matching calculation is executed once every 1 point.
In a preferred embodiment, the template matching rate threshold K ist60%, the ROI area matching rate threshold KrtThe content was 90%.
Compared with the prior art, the embodiment has the advantages that:
7) the template matching method is adopted to identify holes in the three-dimensional point cloud, so that the hole identification rate is improved;
8) the hole template self-adaptive construction algorithm can self-adaptively construct a hole template according to the point cloud density obtained by actual scanning, so that the matching degree of the hole is improved;
9) the three-dimensional point cloud is subjected to binarization processing, so that the calculation of template matching is simplified, and the matching efficiency is improved;
10) the actual position of the hole is calculated by adopting an averaging method, so that the position measurement precision of the hole is improved;
11) the point cloud in the neighborhood of the hole is fitted by SVD decomposition, the normal direction of the hole is replaced by the normal direction of the neighborhood plane,
the normal measurement of the hole is realized, the influence of hole edge noise points on the normal measurement of the hole is reduced, and the normal measurement precision of the hole is improved;
12) and the ROI is utilized to judge the hole characteristics, so that the recognition accuracy of the hole characteristics is improved.
Specifically, for example, the process may also be implemented as steps S1 to S6, where:
step S1, as shown in fig. 2, the 2D laser profiler 1 is driven by a linear servo motor and moves linearly at a uniform speed along the X direction, the speed of the servo motor is V, the sampling period of the profiler is T, the numerical value of the profiler and the numerical value of the servo motor encoder are simultaneously acquired at intervals of T, the displacement in the X direction can be calculated by the numerical value of the encoder, and finally, the point cloud S on the surface of the workpiece 2 to be measured can be obtained, as shown in fig. 3.
Step S2, point cloud pretreatment is carried out by adopting an improved median filtering algorithm, invalid points and outliers of the hole area to be detected are removed, and the point cloud after pretreatment is shown in figure 4.
Step S3, the z-coordinate value of the point cloud S is binarized as shown in fig. 5.
Step S4, adaptively configuring a pore template T (m, n) according to the point cloud density, where the pore template T (m, n) is as shown in fig. 6, and performing template matching, as shown in fig. 7.
And step S5, using the ROI to judge the hole characteristics.
In step S6, the positions of the holes are calculated by averaging, the normal direction of the holes is fitted by SVD decomposition, and the measurement result is shown in fig. 8.
In a further embodiment, in step S2, the modified median filtering algorithm specifically includes the following steps:
s21) sets the y-coordinate value of the invalid point to 0 using the condition that the z-coordinate values of the invalid point are all 0:
Figure BDA0002400038510000101
s22) creating a filter F based on the median filtering principle, the kernel B of the filter F being rectangular in shape, the kernel B having a size of 15 × 3.
S23) convolving the point cloud S with the kernel B, and scanning each point of the point cloud S one by one, for example, for the scanned ith row and jth columnThe point P in (b) is obtained by extracting all the effective points (i.e., points whose z-coordinate value is not zero) in the neighborhood of 15 × 3 centered around the point P from the point cloud S, counting the number m of the effective points in the neighborhood, and counting the median z-coordinate value of the z-coordinate values of the effective pointsmThen, Z is obtained0(i, j) and zmAbsolute value Z of the difference ofd＝|Z0(i,j)-zmAnd if m is less than or equal to 10, judging the point as an isolated point. When Z isdWhen the size is more than or equal to 0.05mm, the point is judged as an outlier. The working process of the filter F is a process of setting the z-coordinate values of the isolated points and the outliers to zero, as follows:
Figure RE-GDA0002568655020000111
in a further embodiment, the specific method of step S3 is:
scanning the point-by-point z coordinate of the point cloud, and setting the z coordinate value of the point to 1 when the z coordinate value is not 0, as follows:
Figure BDA0002400038510000112
in a further embodiment, the specific steps of step S4 are:
s41) calculating distances Δ X and Δ Y of adjacent acquisition points in the X and Y directions by the following method:
in the X direction: Δ X ═ X0L-X0R)/(col-1)， (4)
Y direction: Δ Y ═ Y0U-Y0D)/(row-1) (5)
Wherein, X0LIs the x coordinate value of the first row of point clouds. X0RThe x coordinate value of the last row of point clouds. Y is0UIs the average of the non-zero y values of the first row of point clouds. Y is0DIs the average of the non-zero y values of the last row of point clouds.
col is the number of columns of the point cloud. row is the number of rows in the point cloud.
S42) calculating the line number H of the point cloud contained in the hole to be measuredrowAnd the number of columns Hcol：
Hrow＝D/Δy (6)
Hcol＝D/Δx (7)
Wherein D is the diameter of the hole to be measured.
S43), the template T (m, n) of the hole to be measured is in the shape of an outer square and an inner circle, the size of the outer square is slightly larger than that of the inner circle, the size (m, n) of the outer square is the size of the template, and the template T (m, n) is set as:
m＝Hrow+6 (8)
n＝Hcol+2 (9)
s44) adjusting the size of the template to an odd number by the following method:
Figure BDA0002400038510000121
Figure BDA0002400038510000122
s45) constructing a matrix of m × n with all 1 elements, and then setting a point where the distance from the center point of the template is smaller than the radius of the hole to be measured to 0, thereby obtaining a template T (m, n) of the hole to be measured.
S46), template matching is performed, and a template T (m, n) and a subgraph S are calculatedijThe difference of (m, n) is represented by the formula:
Figure BDA0002400038510000123
wherein, sub-diagram Sij(m, n) is the area where the template T (m, n) covers the searched point cloud S;
i, j is subgraph Sij(m, n) coordinates centered on the searched point cloud S.
The minimum value of E (i, j) is the position of the matched target hole. The matching rate K of template matching is calculated by the following formula:
Figure BDA0002400038510000124
in a further embodiment, the specific steps of step S5 are:
s51) setting the central circular area of the hole template T (m, n) to be detected as an ROI area of interest, and calculating the matching rate K of the ROI arear。
S52) matching the template with a threshold value KtThreshold value K of matching rate with ROI (region of interest)rtSimultaneously, as the distinguishing condition of the hole characteristics, when the template matching rate K is more than KtROI area matching ratio Kr＞KrtThen, the measured area is determined as a hole feature.
In a further embodiment, the specific steps of step S6 are:
s61) sub-graph SijCoordinate C (x) having the average value of coordinate values of all points on (m, n) as the center of circleC,yC,zC) As follows:
Figure BDA0002400038510000131
wherein N is a subfigure Sij(m, n) the number of acquisition points contained.
S62) decomposition by SVD and by sub-graph SijAnd (M, n) fitting a plane, and calculating a singular vector M (a, b, c) corresponding to the minimum singular value transformed by the covariance matrix SVD. The vector M (a, b, c) is taken as the normal vector to the hole to be measured.
In a further embodiment, when the z-coordinate values of all the points in the first row of the point cloud S are all 0, it is determined that the upper edge of the hole region to be measured exceeds the scanning measurement range. And when the z coordinate values of all the points in the last row of the point cloud S are 0, determining that the lower edge of the hole area to be measured exceeds the scanning measurement range.
In a further embodiment, after the point cloud S of the hole site region to be detected is obtained through scanning, template rough and rough matching is firstly carried out, and the rough position P of the hole is obtained through identificationCThen to position PCAnd carrying out filtering pretreatment on the point cloud in the neighborhood, and finally carrying out template fine matching so as to obtain the accurate position P of the hole. Rough position P of holeCIn the above description, P means position, C means circle, and the abbreviation is used as a code in mathematical expression.
In the step S52, the template matching rate threshold K ist60%, the ROI area matching rate threshold KrtThe content was 90%. Wherein, the parameter KtT in the lower right hand corner of (1), refers to threshold, parameter KrtR in the lower right hand corner of (1), refers to region of interest. In a further embodiment, when performing rough template matching, the matching step size is 10, i.e. the matching calculation is performed every 10 points. When the template is finely matched, the matching step is 1, namely, the matching calculation is performed every 1 point.
In order to verify the practicability and normal measurement precision of the template matching-based 2D laser hole site detection method, a unit normal vector N of the axis of a hole to be measured in a 2D contourgraph coordinate system is measured by using a laser tracker, a unit normal vector M of the axis of the hole to be measured in the 2D contourgraph coordinate system is fitted by the template matching-based hole site detection method, and the included angle phi between the vector N and the vector M is the normal measurement error of the hole to be measured. And measuring 10 holes, wherein the normal measurement error is shown in figure 9, the maximum value of phi is not more than 0.2 degrees, and the accuracy requirement of normal accuracy +/-0.5 degrees in airplane assembly is met.
In order to verify the practicability and the position measurement accuracy of the template matching-based 2D laser hole site detection method, 20 holes in 2 rows and 10 columns are drilled on a workpiece, 2 holes in each column are a group, the diameter of each hole is 4mm, and the hole distance is 10 mm. Firstly, measuring the center distance L of 2 holes of each group by using a center distance vernier caliper, then fixing a workpiece on an experimental platform, and respectively scanning each row of holes to obtain 10 groups of point cloud data Ai(i ═ 1,2, …, 10); then spraying developer on the surface of the workpiece (the sprayed developer can effectively improve the surface gloss of the workpiece, and the scanned point cloud is complete and flawless), and respectively scanning each row of holes to obtain 10 groups of point cloud data Bi(i ═ 1,2, …, 10); finally, the traditional hole site detection method based on edge detection and the hole site detection method based on template matching provided by the invention are adopted to carry out hole site detection on the point cloud obtained by scanning and calculate the center distance of 2 holes, and the experimental result is shown in table 1, wherein L is the center distanceHole center distance, L, measured by vernier caliper1The hole center distance, L, is measured by adopting the traditional hole position detection algorithm based on edge detection2The invention provides a hole center distance measured by adopting the template matching-based 2D laser hole position detection method. L is1、L2The difference value from L is the hole location detection error, as shown in fig. 10, where Δ LH1 is the detection error of the conventional hole location detection method based on edge detection after spraying developer, Δ LQ2 is the detection error of the hole location detection method based on template matching according to the present invention before spraying developer, and Δ LH2 is the detection error of the hole location detection method based on template matching according to the present invention after spraying developer.
Point clouds obtained by scanning before and after developer spraying are shown in fig. 11, and it can be seen that the point clouds obtained by scanning the surface of the workpiece 2 have a large number of defects before developer spraying, the whole workpiece is in a grid shape, and at the moment, the edge of the grid can be extracted by the hole position detection method based on edge detection, so that the position of the hole cannot be identified, and therefore, the method is not suitable any more; after the developer is sprayed, the point cloud is ideal, and the hole position detection method based on edge detection can identify the edge of a hole so as to calculate the hole position.
In the automatic drilling and riveting of the airplane parts, the detection precision of the reference hole is required to be within +/-0.5 mm, and the following information can be obtained from the data in fig. 10, fig. 11, table 1 and table 2:
TABLE 1 measurement of hole center distance
Figure BDA0002400038510000151
Figure BDA0002400038510000161
TABLE 2 statistics of measurement results of two hole site testing methods
Figure BDA0002400038510000162
(1) By comparing the success rate of hole site detection before and after spraying the developer by the two methods in table 1, it can be seen that the success rate of hole site detection based on edge detection for holes is 0% before spraying the developer, which indicates that the hole site detection based on edge detection is only suitable for ideal point cloud and has poor robustness; the detection success rate of the hole site detection method based on template matching is 100% before and after the developer is sprayed, and the hole site detection method based on template matching is proved to have strong robustness.
(2) By comparing the measurement errors delta LH1 and delta LH2 of the two hole site detection methods after the developer is sprayed, the measurement errors of the two methods are approximate, and the ideal point cloud is proved to have the same measurement precision by adopting the method based on the template matching and the method based on the edge detection and meet the requirement of the assembly precision of airplane parts.
(3) By comparing the measurement errors delta LQ2 and delta LH2 before and after the developer is sprayed, the measurement accuracy is reduced when the developer is not sprayed, but the assembly accuracy requirement of the airplane component can be still met, which means that the process step of spraying the developer can be omitted after the hole site detection method based on the template matching provided by the invention is adopted, and the airplane assembly efficiency is further improved.
The embodiments in the present specification are described in a progressive manner, and portions that are similar to each other in the embodiments are referred to each other, and each embodiment focuses on differences from other embodiments. In particular, for the apparatus embodiment, since it is substantially similar to the method embodiment, it is relatively simple to describe, and reference may be made to some descriptions of the method embodiment for relevant points. The above description is only a specific embodiment of the present invention, but the scope of the present invention is not limited thereto, and any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope of the present invention are included in the scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims (10)

1. A2D laser hole site detection method based on template matching is characterized by comprising the following steps:
acquiring point cloud of a hole site area to be detected through a 2D laser profiler, wherein the 2D laser profiler is carried on a linear servo motor;
preprocessing the collected point cloud, and removing invalid points and outliers of the hole area to be detected;
after the binarization processing is carried out on the Z coordinate value of each point of the point cloud, a pore template T (m, n) is constructed according to the density of the point cloud, and template matching is carried out, wherein each point in the point cloud corresponds to a space three-dimensional coordinate value (x, y, Z), the Z coordinate value is a coordinate value in the Z direction, the Z direction is a laser emission direction, the pore template T (m, n) is a matrix of m x n, and m and n are positive integers;
judging the characteristics of the hole to be detected aiming at the point cloud after template matching;
and acquiring the position of the hole to be measured by using the characteristic discrimination result of the hole to be measured, and acquiring the normal direction of the hole to be measured.
2. The method of claim 1, wherein the pre-processing the acquired point cloud comprises:
obtaining isolated points and outliers in the point cloud S by using a filter F, wherein a kernel B of the filter F is rectangular, and the size of the kernel B is 15 multiplied by 3;
and setting the z-coordinate values of the isolated points and the outliers to zero.
3. The method of claim 2, wherein the obtaining outliers and outliers in the point cloud S using the filter F comprises:
setting the y coordinate value of the invalid point to 0, and then creating the filter F based on the median filtering principle, wherein the z coordinate values of the invalid point are all 0;
convolving the point cloud S with the kernel B, and scanning each point of the point cloud S one by one;
for scanned points P in the ith row and the jth column, extracting all effective points in a 15 multiplied by 3 neighborhood with the points P as the center from the point cloud S, wherein the z-coordinate value of the effective points is not zero, the number of the effective points in the neighborhood is m, and i and j are both positive integers;
obtaining the Z coordinate value Z of the currently scanned P point0(i, j) and the median z of the z-coordinate values of the effective pointsmAbsolute value Z of the difference ofd＝|Z0(i,j)-zmIf m is less than or equal to 10, judging as an isolated point; when Z isdAnd judging as an outlier when the average particle size is more than or equal to 0.05 mm.
4. The method of claim 3, further comprising:
after each point of the point cloud S is scanned one by one, the z-coordinate value of a point whose z-coordinate value is not 0 is set to 1.
5. The method of claim 4, wherein constructing a pore template T (m, n) from the density of the point cloud S and performing template matching comprises:
distances Δ X and Δ Y of adjacent acquisition points in the X-direction and the Y-direction are acquired, respectively, where Δ X ═ X (X)0L-X0R)/(col-1),Δy＝(Y0U-Y0D)/(row-1)，X0LIs the X coordinate value, X, of the first row point cloud0RThe x value and Y value of the last row of point clouds0UIs the average value of non-zero Y values of the first row of point clouds, Y0DThe average value of the non-zero y values of the last row of point clouds, col is the row number of the point clouds, and row is the row number of the point clouds;
according to the number of lines H of the point cloud contained in the hole to be detectedrowAnd the number of columns HcolAnd acquiring the size of the template T (m, n) of the hole to be measured, wherein the template T (m, n) of the hole to be measured is in the shape of an outer square and an inner circle, the size of the outer square is larger than that of the inner circle, and Hrow＝D/Δy、Hcol＝D/Δx、m＝Hrow+6、n＝Hcol+2 and D are the diameter of the hole to be measured;
after the size of the template T (m, n) is adjusted to be an odd number, an m multiplied by n matrix with all elements being 1 is constructed, and then a point with the distance from the central point of the template T (m, n) being smaller than the radius of the hole to be detected is set to be 0, so that the final matrix form of the template T (m, n) of the hole to be detected is obtained;
performing template matching to obtain a template T (m, n) and a subgraph Sij(m, n) difference, wherein the subgraph Sij(m, n) is the area of the template T (m, n) covering the point cloud S, i, j is the subgraph SijAnd (m, n) coordinate values of the center on the searched point cloud S, and when the difference value is at the minimum value, the position of the matched target hole is determined.
6. The method of claim 5, wherein the performing the feature discrimination of the hole under test comprises:
setting the central circular area of the hole template T (m, n) to be detected as an ROI (region of interest), and acquiring the matching rate K of the ROIr；
When the template matching rate K is more than KtROI area matching ratio Kr＞KrtThen, the measured area is determined as a hole feature, where KtAs template matching rate threshold, KrtIs the threshold value of the ROI area matching rate, K is the matching rate of template matching,
Figure FDA0002400038500000031
e (i, j) is the template T (m, n) at position (i, j) and subgraph SijDeviation value of (m, n).
7. The method of claim 4, wherein said obtaining a normal to said hole under test comprises:
obtaining the coordinate C (x) of the center of circle by using the average value of the coordinate values of all points on the subgraphC,yC,zC) Wherein, in the step (A),
Figure FDA0002400038500000032
n is the number x of acquisition points contained in the subgraphC、yCAnd zCCoordinate values, S, on the x, y and z axes, respectively, of the centre of the circleij(m, n) represents the subgraph;
and fitting a plane by using the points on the subgraph, calculating a singular vector corresponding to the minimum singular value of the SVD transformation of all the points on the subgraph, and taking the singular vector as the normal vector of the hole to be measured.
8. The method of claim 1, further comprising:
when the z coordinate values of all the points in the first row of the point cloud S are 0, determining that the upper edge of the hole area to be measured exceeds the scanning measurement range;
and when the z coordinate values of all the points in the last row of the point cloud S are all 0, determining that the lower edge of the hole area to be measured exceeds the scanning measurement range.
9. The method of claim 1 or 8, further comprising:
after the point cloud S is obtained through scanning, firstly, rough template matching is carried out, and the rough position P of the hole is obtained through identificationC；
Then to PCFiltering and preprocessing the point cloud in the neighborhood, and finally performing template fine matching to obtain the accurate position P of the hole;
when the template is roughly matched, the matching step length is 10, and matching calculation is executed once every 10 points; when the template is finely matched, the matching step length is 1, and the matching calculation is executed once every 1 point.
10. The method of claim 6, wherein the template matching rate threshold K ist60%, the ROI area matching rate threshold KrtThe content was 90%.
CN202010143880.3A 2020-03-04 2020-03-04 Template matching-based 2D laser hole site detection method Active CN111539446B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010143880.3A CN111539446B (en) 2020-03-04 2020-03-04 Template matching-based 2D laser hole site detection method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010143880.3A CN111539446B (en) 2020-03-04 2020-03-04 Template matching-based 2D laser hole site detection method

Publications (2)

Publication Number Publication Date
CN111539446A true CN111539446A (en) 2020-08-14
CN111539446B CN111539446B (en) 2023-10-03

Family

ID=71978658

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010143880.3A Active CN111539446B (en) 2020-03-04 2020-03-04 Template matching-based 2D laser hole site detection method

Country Status (1)

Country Link
CN (1) CN111539446B (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112631201A (en) * 2020-12-28 2021-04-09 佛山科学技术学院 Hole searching control method and system for shaft hole assembly
CN113701626A (en) * 2021-08-10 2021-11-26 哈尔滨岛田大鹏工业股份有限公司 3D machine vision detection method for automobile longitudinal beam
CN113865508A (en) * 2021-09-28 2021-12-31 南京航空航天大学 Automatic detection device and method for through hole rate of acoustic lining of honeycomb sandwich composite material
CN114234796A (en) * 2021-10-26 2022-03-25 深圳市裕展精密科技有限公司 Hole detection method, hole detection device and hole detection equipment
CN114663438A (en) * 2022-05-26 2022-06-24 浙江银轮智能装备有限公司 Track detection method, system, apparatus, storage medium and computer program product

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102509383A (en) * 2011-11-28 2012-06-20 哈尔滨工业大学深圳研究生院 Feature detection and template matching-based mixed number identification method
CN108981604A (en) * 2018-07-11 2018-12-11 天津工业大学 A kind of precision component three-dimensional overall picture measurement method based on line laser
CN109147040A (en) * 2018-08-28 2019-01-04 浙江大学 Human body dot cloud hole method for repairing and mending based on template

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102509383A (en) * 2011-11-28 2012-06-20 哈尔滨工业大学深圳研究生院 Feature detection and template matching-based mixed number identification method
CN108981604A (en) * 2018-07-11 2018-12-11 天津工业大学 A kind of precision component three-dimensional overall picture measurement method based on line laser
CN109147040A (en) * 2018-08-28 2019-01-04 浙江大学 Human body dot cloud hole method for repairing and mending based on template

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112631201A (en) * 2020-12-28 2021-04-09 佛山科学技术学院 Hole searching control method and system for shaft hole assembly
CN113701626A (en) * 2021-08-10 2021-11-26 哈尔滨岛田大鹏工业股份有限公司 3D machine vision detection method for automobile longitudinal beam
CN113701626B (en) * 2021-08-10 2023-08-04 哈尔滨岛田大鹏工业股份有限公司 Automobile longitudinal beam 3D machine vision detection method
CN113865508A (en) * 2021-09-28 2021-12-31 南京航空航天大学 Automatic detection device and method for through hole rate of acoustic lining of honeycomb sandwich composite material
CN114234796A (en) * 2021-10-26 2022-03-25 深圳市裕展精密科技有限公司 Hole detection method, hole detection device and hole detection equipment
CN114663438A (en) * 2022-05-26 2022-06-24 浙江银轮智能装备有限公司 Track detection method, system, apparatus, storage medium and computer program product

Also Published As

Publication number Publication date
CN111539446B (en) 2023-10-03

Similar Documents

Publication Publication Date Title
CN111539446A (en) 2D laser hole site detection method based on template matching
CN109489580B (en) On-machine point cloud detection and compensation method for surface processing of aero-engine blade
CN107292927B (en) Binocular vision-based symmetric motion platform pose measurement method
CN109060836B (en) Machine vision-based high-pressure oil pipe joint external thread detection method
CN105021124B (en) A kind of planar part three-dimensional position and normal vector computational methods based on depth map
CN107341802B (en) Corner sub-pixel positioning method based on curvature and gray scale compounding
KR102056664B1 (en) Method for work using the sensor and system for performing thereof
US8103376B2 (en) System and method for the on-machine 2-D contour measurement
CN111609811A (en) Machine vision-based large-size plate forming online measurement system and method
CN106251353A (en) Weak texture workpiece and the recognition detection method and system of three-dimensional pose thereof
CN112614098B (en) Blank positioning and machining allowance analysis method based on augmented reality
CN113112496B (en) Sub-pixel shaft part size measurement method based on self-adaptive threshold
CN113324478A (en) Center extraction method of line structured light and three-dimensional measurement method of forge piece
CN111047588A (en) Imaging measurement method for size of shaft type small part
CN112880562A (en) Method and system for measuring pose error of tail end of mechanical arm
CN115482195B (en) Train part deformation detection method based on three-dimensional point cloud
CN113267180B (en) AGV forklift tray positioning and forking method based on 3D depth vision
CN113689415A (en) Steel pipe wall thickness online detection method based on machine vision
CN114037675A (en) Airplane sample plate defect detection method and device
Usamentiaga et al. Real-time inspection of long steel products using 3-D sensors: Calibration and registration
CN115546125A (en) Method for error detection and track deviation correction of additive manufacturing cladding layer based on point cloud information
CN116466649A (en) Machine tool machining system based on three-dimensional laser scanning analysis
CN114022441A (en) Defect detection method for irregular hardware
CN111366092B (en) Line structure light sensor pose adjusting method
Su et al. Neural networks for precise measurement in computer vision systems

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant