CN116385356A - Method and system for extracting regular hexagonal hole features based on laser vision - Google Patents

Method and system for extracting regular hexagonal hole features based on laser vision Download PDF

Info

Publication number
CN116385356A
CN116385356A CN202310140327.8A CN202310140327A CN116385356A CN 116385356 A CN116385356 A CN 116385356A CN 202310140327 A CN202310140327 A CN 202310140327A CN 116385356 A CN116385356 A CN 116385356A
Authority
CN
China
Prior art keywords
plane
hexagonal hole
fitting
image
dimensional
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202310140327.8A
Other languages
Chinese (zh)
Inventor
曹先锋
马继勇
刘强
李昂
江雅珍
朱帅臣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Jiaotong University
China National Heavy Duty Truck Group Jinan Power Co Ltd
Original Assignee
Shanghai Jiaotong University
China National Heavy Duty Truck Group Jinan Power Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Jiaotong University, China National Heavy Duty Truck Group Jinan Power Co Ltd filed Critical Shanghai Jiaotong University
Priority to CN202310140327.8A priority Critical patent/CN116385356A/en
Publication of CN116385356A publication Critical patent/CN116385356A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/0002Inspection of images, e.g. flaw detection
    • G06T7/0004Industrial image inspection
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00Image analysis
    • G06T7/60Analysis of geometric attributes
    • G06T7/62Analysis of geometric attributes of area, perimeter, diameter or volume
    • YGENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02PCLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/30Computing systems specially adapted for manufacturing

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Geometry (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The invention provides a method and a system for extracting regular hexagonal hole characteristics based on laser vision, wherein the method comprises the following steps: acquiring a plurality of images of the regular hexagonal hole scanned by the optical cutter lines, wherein each image has one optical cutter line; obtaining two-dimensional coordinates of the center of the optical cutter through image processing on each image, converting the two-dimensional coordinates into three-dimensional coordinates, and unifying the three-dimensional coordinates into a camera coordinate system to obtain three-dimensional point clouds of edge characteristics of the hexagonal hole and three-dimensional point clouds of a plane where the hexagon is located; and performing plane fitting on the edge characteristic three-dimensional point cloud of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located to obtain a fitting plane, and calculating the center point, the diameter of the circumscribed circle, the diameter of the inscribed circle, the diameter of the circumscribed circle, the normal vector and the direction vector of the hexagonal hole. Based on the method, a system for extracting the regular hexagonal hole features based on laser vision is also provided, the processing and the calculation of the acquired data are realized through the image and the point cloud algorithm, and the calculation result is used for guiding production and processing.

Description

Method and system for extracting regular hexagonal hole features based on laser vision
Technical Field
The invention belongs to the technical field of non-contact measurement of commercial vehicle body parts, and particularly relates to a method and a system for extracting regular hexagonal hole characteristics based on laser vision.
Background
In the processing process of automobile and aircraft parts, the sizes of the produced and processed parts are deviated due to the factors of site environment, processing precision, manual errors and the like. Timely detection of defective parts is of great significance to overall production, some tools and algorithms can be used for replacing manual detection to improve detection accuracy and efficiency, and overall quality can be guaranteed by acquiring and recording error data.
In the manufacture of automobile sheet metal, the hexagonal characteristic holes are very important hole characteristics although not common hole characteristics, one use is for the installation of plastic buckles, and the limit function can be achieved not only but also in a limited direction. Therefore, the method for realizing high-precision rapid detection of the hexagonal hole features has important significance for the production, processing and assembly processes. At present, when the method for extracting the characteristics of the hexagonal holes adopts an image mode to measure, the characteristic extraction is mainly realized through edge extraction, but the mode can generate a large amount of noise to influence the measurement accuracy due to external interference. In the hexagon detection method proposed in the prior art, a two-dimensional image detection method is adopted to detect the hexagon bolt, the hexagon detection of the component is divided into straight lines and circles and the calculation of the line end length and the straight line included angle through the Hough transformation principle, and the hexagon characteristic can be effectively detected by the method. But the detection accuracy is not high enough in the image domain. In the production and processing process of the prior art, the prior two-dimensional measurement mode has the defects of high measurement accuracy requirement, complex field environment and multiple interference factors, and the effect of the prior two-dimensional measurement mode is not ideal.
Disclosure of Invention
In order to solve the technical problems, the invention provides a method and a system for extracting regular hexagonal hole features based on laser vision. The laser vision sensor can be used for detecting the characteristics of the hexagonal holes on the machined parts, so that labor force is saved, and detection efficiency is improved.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the extraction method of the regular hexagon hole features based on laser vision comprises the following steps:
acquiring a plurality of images of the regular hexagonal hole scanned by the optical cutter lines, wherein each image has one optical cutter line;
obtaining two-dimensional coordinates of the center of the optical cutter through image processing on each image, converting the two-dimensional coordinates into three-dimensional coordinates, and simultaneously unifying the three-dimensional coordinates into a camera coordinate system to obtain three-dimensional point clouds of edge characteristics of the hexagonal hole and three-dimensional point clouds of a plane where the hexagon is located;
performing plane fitting on the three-dimensional point cloud of the edge characteristic of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located to obtain a fitting plane, screening out points far away from the fitting plane, projecting the three-dimensional point cloud of the edge characteristic to the fitting plane to calculate projection points, separating six sides, and then calculating the central point, the diameter of an circumscribed circle, the diameter of an inscribed circle, the diameter of an circumscribed circle, a normal vector and a direction vector of the hexagonal hole.
Further, the process of acquiring a plurality of images of the optical knife line scanning regular hexagonal hole comprises the following steps:
the swing type line laser sensor is adopted to scan the hexagonal hole to be measured, the image is shot through the image acquisition equipment uniformly according to the swing angle, and each image has one optical knife line.
Further, the image processing includes:
selecting a first target area in the image to be processed, and then preprocessing the selected first target area;
searching hexagonal hole contours in a selected first target area, and selecting an upper contour and a lower contour which are longest in the position of the image through circumscribing rectangles by each contour;
and selecting a second target area formed by the circumscribed rectangular area from the original image, and extracting the same second target area range by using a light knife to obtain a two-dimensional sub-pixel coordinate of the center of the light knife.
Further, the preprocessing includes:
firstly, carrying out linear smoothing filtering on a first target area by utilizing a Gaussian filtering algorithm;
after linear smoothing filtering, binarization processing is carried out;
after the binarization process, an erosion operation is performed on the region of the pixel above the threshold to find a local minimum using an on operation in the morphological operation.
Further, performing plane fitting on the three-dimensional point cloud of the edge feature of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located to obtain a fitted plane, and screening out points far from the fitted plane comprises the following steps:
performing plane fitting on the edge characteristic three-dimensional point cloud of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located by utilizing an SVD decomposition principle to obtain a fitted plane;
calculating the average value of the plane fitting error as the standard value of the screening out-of-plane point;
and circularly calculating the distance from each point to the fitting plane, comparing the distance with a standard value, and screening out points far away from the plane.
Further, the process of projecting the edge feature three-dimensional point cloud to the fitting plane to calculate a projection point includes: and projecting the edge characteristic three-dimensional point cloud to a fitting plane, calculating the distance between each hexagonal characteristic point and the fitting plane, and calculating the projection of each characteristic point on the fitting plane according to the normal vector and the distance of the plane.
Further, the process of separating six sides includes:
dividing the hexagon into two parts, each part comprising three sides; traversing all the characteristic points from a fourth point, wherein the fourth point takes three points forwards as a first set, and takes three points backwards as a second set; the first set and the second set are respectively subjected to straight line fitting;
calculating an included angle of the two straight lines, if the included angle does not exceed an angle deviation threshold value, locating the current point at the straight line part, otherwise, locating the current point as an alternative point of the inflection point region;
and each inflection point alternative area is subjected to secondary inflection point searching, and two points are selected each time to form a straight line. The point clouds on the six sides are drawn by simultaneous processing forward and backward.
Further, after six sides are separated, six sides are subjected to intersection point obtaining six corner points, and the six corner points are subjected to circumcircle fitting and six sides are subjected to inscription circle diameter obtaining.
Further, the regular hexagonal inscribed circle diameter=circumscribed circle diameter×cos30 °.
The invention also provides a system for extracting the regular hexagonal hole characteristics based on laser vision, which comprises an acquisition module, an image processing module and a fitting calculation module;
the acquisition module is used for acquiring a plurality of images of the regular hexagonal hole scanned by the optical cutter lines, and each image has one optical cutter line;
the image processing module is used for obtaining two-dimensional coordinates of the center of the optical cutter through image processing on each image, converting the two-dimensional coordinates into three-dimensional coordinates, and simultaneously unifying the three-dimensional coordinates into a camera coordinate system to obtain three-dimensional point clouds of edge characteristics of the hexagonal hole and three-dimensional point clouds of a plane where the hexagon is located;
the fitting calculation module is used for carrying out plane fitting on the edge characteristic three-dimensional point cloud of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located to obtain a fitting plane, projecting the edge characteristic three-dimensional point cloud to the fitting plane to calculate projection points after screening points far away from the fitting plane, separating six sides, and then calculating the center point, the circumscribed circle diameter, the inscribed circle diameter, the circumscribed circle diameter, the normal vector and the direction vector of the hexagonal hole.
The effects provided in the summary of the invention are merely effects of embodiments, not all effects of the invention, and one of the above technical solutions has the following advantages or beneficial effects:
the invention provides a method and a system for extracting regular hexagonal hole characteristics based on laser vision, wherein the method comprises the following steps: acquiring a plurality of images of the regular hexagonal hole scanned by the optical cutter lines, wherein each image has one optical cutter line; obtaining two-dimensional coordinates of the center of the optical cutter through image processing on each image, converting the two-dimensional coordinates into three-dimensional coordinates, and simultaneously unifying the three-dimensional coordinates into a camera coordinate system to obtain three-dimensional point clouds of edge characteristics of the hexagonal hole and three-dimensional point clouds of a plane where the hexagon is located; performing plane fitting on the three-dimensional point cloud of the edge characteristic of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located to obtain a fitting plane, screening out points far away from the fitting plane, projecting the three-dimensional point cloud of the edge characteristic to the fitting plane to calculate projection points, separating six sides, and then calculating the central point, the diameter of an circumscribed circle, the diameter of an inscribed circle, the diameter of an circumscribed circle, a normal vector and a direction vector of the hexagonal hole. A method for extracting regular hexagonal hole features based on laser vision is also provided. The method collects data to be processed by means of the sensor, processes the data through the image part and the point cloud part algorithm, and can calculate parameters of a regular hexagonal hole, including center point coordinates, circumscribed circle diameters, inscribed circle diameters and normal vector information, and the calculated result can help site workers evaluate whether machining is reasonable or not and is used for guiding production and machining.
Drawings
Fig. 1 is a flowchart of a method for extracting regular hexagonal hole features based on laser vision according to embodiment 1 of the present invention;
FIG. 2 is a photograph of a camera of example 1 of the present invention with a single line of light projected onto a hexagonal hole;
fig. 3 is a schematic view of a three-dimensional point cloud of a regular hexagonal hole according to embodiment 1 of the present invention;
fig. 4 is a schematic diagram of an extraction system of regular hexagonal hole features based on laser vision according to embodiment 2 of the present invention.
Detailed Description
In order to clearly illustrate the technical features of the present solution, the present invention will be described in detail below with reference to the following detailed description and the accompanying drawings. The following disclosure provides many different embodiments, or examples, for implementing different structures of the invention. In order to simplify the present disclosure, components and arrangements of specific examples are described below. Furthermore, the present invention may repeat reference numerals and/or letters in the various examples. This repetition is for the purpose of simplicity and clarity and does not in itself dictate a relationship between the various embodiments and/or configurations discussed. It should be noted that the components illustrated in the figures are not necessarily drawn to scale. Descriptions of well-known components and processing techniques and processes are omitted so as to not unnecessarily obscure the present invention.
Example 1
The embodiment 1 of the invention provides a method for extracting regular hexagonal hole characteristics based on laser vision, which is characterized in that data to be processed are acquired by means of a sensor, the data are processed through an image part and a point cloud part algorithm, and parameters of a regular hexagonal hole, including center point coordinates, circumscribed circle diameters, inscribed circle diameters and normal vector information, can be calculated after the processing.
Fig. 1 is a flowchart of a method for extracting regular hexagonal hole features based on laser vision according to embodiment 1 of the present invention;
in step S100, a plurality of images of the regular hexagonal hole scanned by the optical cutter line are acquired, and each image has only one optical cutter line;
the swing type line laser sensor is adopted to scan the hexagonal hole to be measured, the image is shot through the image acquisition equipment uniformly according to the swing angle, and each image has one optical knife line. For the hexagonal hole feature, the sensor captures a total of 61 images, with one optical line per image.
In step S200, obtaining a two-dimensional coordinate of a center of the optical cutter through image processing on each image, converting the two-dimensional coordinate into a three-dimensional coordinate, and simultaneously unifying the three-dimensional coordinate under a camera coordinate system to obtain a three-dimensional point cloud of edge characteristics of the hexagonal hole and a three-dimensional point cloud of a plane where the hexagon is located;
the image processing process comprises the following steps: selecting a first target area in the image to be processed, and then preprocessing the selected first target area; searching hexagonal hole contours in a selected first target area, and selecting an upper contour and a lower contour which are longest in the position of the image through circumscribing rectangles by each contour; and selecting a second target area formed by the circumscribed rectangular area from the original image, and extracting the same second target area range by using a light knife to obtain a two-dimensional sub-pixel coordinate of the center of the light knife.
In the present application: and selecting an ROI (region of interest) area, reducing the size of an image to be processed, and taking the first target area as the ROI area.
The process of preprocessing the selected first target area comprises the following steps: firstly, carrying out linear smoothing filtering on a first target area by utilizing a Gaussian filtering algorithm; after linear smoothing filtering, binarization processing is carried out; after the binarization process, an erosion operation is performed on the region of the pixel above the threshold to find a local minimum using an on operation in the morphological operation.
The image is first subjected to linear smoothing filtering by means of a gaussian filtering algorithm for eliminating gaussian noise. The Gaussian filter performs weighted average on the whole image, and the value of each pixel is obtained by performing weighted average on the value of each pixel and the values of other pixels in the field. Most of noise on the image can be eliminated through Gaussian filtering, and subsequent processing interference is reduced. And secondly, binarization processing is carried out, and the pixel value is changed into 1 which is higher than the threshold value and is changed into 0 which is lower than the threshold value, so that the subsequent contour searching is facilitated. The open operation in morphological operation is performed again, namely, the erosion operation is performed on the region with the pixel value of 1 of the image, and the local minimum value is found. Finally, a contour searching step is performed, wherein only a light knife area is left on the image after the preprocessing, as shown in fig. 2, the image shot by a camera when a single light knife line is projected on a hexagonal hole in embodiment 1 of the present invention, the light knives shown in fig. 2 are generally distributed in two areas, and the middle part is that the lower layer of the shot hole does not belong to the characteristics of the hexagonal hole. The found contours are not necessarily valid contours requiring subsequent processing.
The two profiles representing hexagonal holes, i.e. the upper and lower longest profiles, are selected.
Searching hexagonal hole contours in a selected first target area, and selecting an upper contour and a lower contour which are longest in the position of the image through circumscribing rectangles by each contour;
and selecting a second target area formed by the circumscribed rectangular area from the original image, and extracting the same second target area range by using a light knife to obtain a two-dimensional sub-pixel coordinate of the center of the light knife.
Firstly, the minimum circumscribed rectangle of each contour is obtained through an algorithm, the circumscribed rectangle is guaranteed to be the minimum circumscribed rectangle, the long form of the rectangle is guaranteed to be parallel to the length and width of the image, and the circumscribed rectangle is used as a region of interest (ROI). In order to facilitate the subsequent extraction of the center of the optical knife, the image is properly expanded in the width direction of the image, and the left and right are generally expanded by 8 pixels. And secondly, sequencing the outlines according to the perimeter, and selecting the perimeter larger than 100 pixels for processing so as to exclude some noise which is not eliminated from participating in subsequent calculation. And finally, selecting an upper contour area and a lower contour area through the position of the image of the circumscribed rectangle, so that not only can the upper contour and the lower contour be distinguished, but also the contour which does not belong to the middle part of the hexagonal hole in fig. 2 can be eliminated. And after multiplying 2 by the Y coordinate of the left upper corner of the circumscribed rectangle outline, if the Y coordinate is smaller than the width of the image minus the width of the circumscribed rectangle, the circumscribed rectangle is indicated to be positioned at the upper part of the image, otherwise, the circumscribed rectangle is positioned at the lower part of the image.
Firstly, the processing range of the circumscribed rectangle formed by the two selected contours draws the same ROI range at the same position on the original image. And secondly, extracting the center of the optical cutter by using a gray level gravity center method in each row within the ROI range, and finally obtaining the two-dimensional sub-pixel coordinates of the center of the optical cutter after each contour is processed from top to bottom.
And converting the two-dimensional pixel coordinates into three-dimensional point clouds under a sensor coordinate system by using calibration parameters of the sensor, wherein the left graph is all points as shown in fig. 3, and the right graph is extracted characteristic points. And finding out the characteristic points in the middle and lower parts of the upper half outline of the image, and finding out the characteristic points in the middle and upper parts of the lower half outline of the image.
In step S300, performing plane fitting on the three-dimensional point cloud of the edge feature of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located to obtain a fitted plane, screening out points far away from the fitted plane, projecting the three-dimensional point cloud of the edge feature to the fitted plane to calculate projection points, separating six sides, and then calculating the center point, the diameter of the circumscribed circle, the diameter of the inscribed circle, the diameter of the circumscribed circle, the normal vector and the direction vector of the hexagonal hole.
According to the method, the process of performing plane fitting on the edge characteristic three-dimensional point cloud of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located to obtain a fitting plane is as follows:
performing plane fitting on the edge characteristic three-dimensional point cloud of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located by utilizing an SVD decomposition method principle to obtain a fitting plane P; calculating the average value of the plane fitting error as the standard value of the screening out-of-plane point; and circularly calculating the distance from each point to the fitting plane P, comparing the distance with a standard value, and screening out points far away from the plane.
The process of projecting the edge characteristic three-dimensional point cloud to the fitting plane to calculate projection points comprises the following steps: and projecting the characteristic three-dimensional point cloud to a fitting plane P, calculating the distance from each hexagonal characteristic point to the fitting plane, and calculating the projection of each characteristic point on the fitting plane P according to the normal vector and the distance of the plane.
The 6 sides are separated, and the characteristic point of the hexagon is divided into an upper part and a lower part, wherein each part comprises 3 sides. The upper and lower parts are processed simultaneously. The inflection points of 3 edges are found out first, and then feature point clouds are separated according to the inflection points.
All feature points are traversed from the 4 th point, 3 points forward of the point are taken as one set, 3 points backward are taken as the other set, and the two sets are respectively subjected to linear fitting. Calculating the included angle of two straight lines ɑ If the included angle is ɑ A point approaching 0 degrees or 180 degrees (deviation threshold not exceeding 10 degrees) indicates that the point is a straight line portion, and conversely, the point is recorded as an alternative point to the inflection point region. According to the characteristic that the internal angle of the regular hexagon is 120 degrees, when the upper part and the lower part are simultaneously processed, the angle can be changed from being close to 0 degree (or 180 degrees) to being close to 120 degrees (or 60 degrees) to being close to 0 degree, the upper characteristic point and the lower characteristic point in the transformation process can be respectively changed twice, and two inflection point alternative areas can be recorded.
Each inflection point candidate region performs a secondary search for an inflection point. The judging mode is carried out in the area when the inflection point is found, but only 2 points are selected for each point selection to construct a straight line. After the secondary searching, a inflection point range with a smaller range can be obtained, and after points in the inflection point range are eliminated, the points before and after the inflection point are divided into points on 3 straight lines. The point clouds on 6 sides can be divided by processing the point clouds up and down simultaneously.
The 6 sides calculate the intersection point, all the characteristic points are projected on the same plane in the above steps, but each point is still a three-dimensional point, so that the straight line intersection point is inconvenient to solve, and therefore the three-dimensional point is required to be changed into a two-dimensional point to calculate the intersection point.
Firstly, arbitrarily taking two point construction unit direction vectors from characteristic points
Figure BDA0004087296970000081
X-axis as new coordinate system, in plane normal vector +.>
Figure BDA0004087296970000082
As Z axis +.>
Figure BDA0004087296970000083
The Y-axis can be constructed. Secondly, coordinates of three known points in the new and old coordinate systems are needed for realizing coordinate system conversion. Since 2 points have been selected when constructing the X-axis. Assuming that the selected point is P1, knowing the coordinates of the point in the old coordinate system and defining the point as an origin in the new coordinate system; the other point is P2, and the distance D1 of the point from the P1 is calculated, and then the point P2 is (D1, 0) in the new coordinate system; the third point cannot be directly selected and needs to be manually constructed by calculation, and the P1 point and the plane normal vector are known in the old coordinate system +.>
Figure BDA0004087296970000084
It is possible to construct the next point p3=p1+d2 of the old coordinate system +.>
Figure BDA0004087296970000085
Wherein D2 is a self-determined distance may be other than 0Then the P3 point is (0, d 2) in the new coordinate system. And finally, calculating a transformation relation matrix RT between the new coordinate system and the old coordinate system according to the three points, and transforming the characteristic points on the 6 sides into the new coordinate system through the transformation matrix RT.
The Z coordinates of all the characteristic points in the new coordinate system are the same and 0, so that the three-dimensional points can be changed into two-dimensional points by directly removing the Z values of the points, 6 straight lines on the XOY plane are respectively fitted to the two-dimensional points on 6 sides, the intersection point is obtained by solving the intersection point of every two straight lines, 6 two-dimensional angular points are obtained, and the six points are transformed into the points in the old coordinate system through the RT transformation matrix after the Z values of the six points are added with 0.
And 6 corner points are subjected to circumcircle fitting, and the RANSAC algorithm is utilized to carry out circle fitting on the 6 points. And randomly selecting 3 points to calculate a corresponding space circle equation, calculating the distances di between other points and a circle, setting a threshold value beta, if di < beta, considering the points to be in the circle, otherwise, recording the number of the points in the circle outside the circle. Repeating the steps, and selecting the model parameters corresponding to the spherical surface with the largest number of interior points; and calculating an iteration ending judgment factor according to the expected error rate, the optimal inner point number, the total sample number and the current iteration number at the end of each iteration, and determining whether to stop iteration according to the iteration number. And after iteration is finished, carrying out least square circle fitting on the inner points to obtain a final circle fitting coefficient. The diameter of the circumscribed circle of the hexagon and the midpoint coordinate can be obtained through the fitting.
The diameter of the inscribed circle is calculated on 6 sides, and the diameter of the inscribed circle of the regular hexagon=the diameter of the circumscribed circle×cos30 °.
The information required for regular hexagonal measurement includes: center point, circumscribed circle diameter, inscribed circle diameter, circumscribed circle diameter, normal vector, direction vector.
The embodiment 1 of the invention provides a method for extracting the characteristics of a regular hexagonal hole based on laser vision, which is characterized in that data to be processed are acquired by a sensor, the data are processed by an image part and a point cloud part algorithm, parameters of the regular hexagonal hole, including center point coordinates, circumscribed circle diameters, inscribed circle diameters and normal vector information, can be calculated after the processing, and the calculation result can help site workers to evaluate whether the processing is reasonable or not and is used for guiding production and processing.
Example 2
Based on the embodiment 1 of the present invention, a method for extracting regular hexagonal hole features based on laser vision is provided, and the embodiment 2 of the present invention provides a system for extracting regular hexagonal hole features based on laser vision, as shown in fig. 4, which is a schematic diagram of the system for extracting regular hexagonal hole features based on laser vision in the embodiment 2 of the present invention, where the system includes: the device comprises an acquisition module, an image processing module and a fitting calculation module;
the acquisition module is used for acquiring a plurality of images of the regular hexagonal hole scanned by the optical cutter lines, and each image has one optical cutter line;
the image processing module is used for obtaining two-dimensional coordinates of the center of the optical cutter through image processing on each image, converting the two-dimensional coordinates into three-dimensional coordinates, and simultaneously unifying the three-dimensional coordinates into a camera coordinate system to obtain three-dimensional point clouds of edge characteristics of the hexagonal hole and three-dimensional point clouds of a plane where the hexagon is located;
the fitting calculation module is used for carrying out plane fitting on the edge characteristic three-dimensional point cloud of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located to obtain a fitting plane, projecting the edge characteristic three-dimensional point cloud to the fitting plane to calculate projection points after screening points far away from the fitting plane, separating six sides, and then calculating the center point, the diameter of an circumscribed circle, the diameter of an inscribed circle, the diameter of an circumscribed circle, a normal vector and a direction vector of the hexagonal hole.
The process executed by the acquisition module comprises the following steps: the swing type line laser sensor is adopted to scan the hexagonal hole to be measured, the image is shot through the image acquisition equipment uniformly according to the swing angle, and each image has one optical knife line. For the hexagonal hole feature, the sensor captures a total of 61 images, with one optical line per image.
In the process executed by the image processing module, the image processing process comprises the following steps: selecting a first target area in the image to be processed, and then preprocessing the selected first target area; searching hexagonal hole contours in a selected first target area, and selecting an upper contour and a lower contour which are longest in the position of the image through circumscribing rectangles by each contour; and selecting a second target area formed by the circumscribed rectangular area from the original image, and extracting the same second target area range by using a light knife to obtain a two-dimensional sub-pixel coordinate of the center of the light knife.
In the present application: and selecting an ROI (region of interest) area, reducing the size of an image to be processed, and taking the first target area as the ROI area.
The process of preprocessing the selected first target area comprises the following steps: firstly, carrying out linear smoothing filtering on a first target area by utilizing a Gaussian filtering algorithm; after linear smoothing filtering, binarization processing is carried out; after the binarization process, an erosion operation is performed on the region of the pixel above the threshold to find a local minimum using an on operation in the morphological operation.
The image is first subjected to linear smoothing filtering by means of a gaussian filtering algorithm for eliminating gaussian noise. The Gaussian filter performs weighted average on the whole image, and the value of each pixel is obtained by performing weighted average on the value of each pixel and the values of other pixels in the field. Most of noise on the image can be eliminated through Gaussian filtering, and subsequent processing interference is reduced. And secondly, binarization processing is carried out, and the pixel value is changed into 1 which is higher than the threshold value and is changed into 0 which is lower than the threshold value, so that the subsequent contour searching is facilitated. The open operation in morphological operation is performed again, namely, the erosion operation is performed on the region with the pixel value of 1 of the image, and the local minimum value is found. Finally, a contour searching step is performed, wherein only a light knife area is left on the image after the preprocessing, as shown in fig. 2, the image shot by a camera when a single light knife line is projected on a hexagonal hole in embodiment 1 of the present invention, the light knives shown in fig. 2 are generally distributed in two areas, and the middle part is that the lower layer of the shot hole does not belong to the characteristics of the hexagonal hole. The found contours are not necessarily valid contours requiring subsequent processing.
The two profiles representing hexagonal holes, i.e. the upper and lower longest profiles, are selected.
Searching hexagonal hole contours in a selected first target area, and selecting an upper contour and a lower contour which are longest in the position of the image through circumscribing rectangles by each contour;
and selecting a second target area formed by the circumscribed rectangular area from the original image, and extracting the same second target area range by using a light knife to obtain a two-dimensional sub-pixel coordinate of the center of the light knife.
Firstly, the minimum circumscribed rectangle of each contour is obtained through an algorithm, the circumscribed rectangle is guaranteed to be the minimum circumscribed rectangle, the long form of the rectangle is guaranteed to be parallel to the length and width of the image, and the circumscribed rectangle is used as a region of interest (ROI). In order to facilitate the subsequent extraction of the center of the optical knife, the image is properly expanded in the width direction of the image, and the left and right are generally expanded by 8 pixels. And secondly, sequencing the outlines according to the perimeter, and selecting the perimeter larger than 100 pixels for processing so as to exclude some noise which is not eliminated from participating in subsequent calculation. And finally, selecting an upper contour area and a lower contour area through the position of the image of the circumscribed rectangle, so that not only can the upper contour and the lower contour be distinguished, but also the contour which does not belong to the middle part of the hexagonal hole in fig. 2 can be eliminated. And after multiplying 2 by the Y coordinate of the left upper corner of the circumscribed rectangle outline, if the Y coordinate is smaller than the width of the image minus the width of the circumscribed rectangle, the circumscribed rectangle is indicated to be positioned at the upper part of the image, otherwise, the circumscribed rectangle is positioned at the lower part of the image.
Firstly, the processing range of the circumscribed rectangle formed by the two selected contours draws the same ROI range at the same position on the original image. And secondly, extracting the center of the optical cutter by using a gray level gravity center method in each row within the ROI range, and finally obtaining the two-dimensional sub-pixel coordinates of the center of the optical cutter after each contour is processed from top to bottom.
And converting the two-dimensional pixel coordinates into three-dimensional point clouds under a sensor coordinate system by using calibration parameters of the sensor, wherein the left graph is all points as shown in fig. 3, and the right graph is extracted characteristic points. And finding out the characteristic points in the middle and lower parts of the upper half outline of the image, and finding out the characteristic points in the middle and upper parts of the lower half outline of the image.
In the process of executing the fitting calculation module, the process of carrying out plane fitting on the edge characteristic three-dimensional point cloud of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located to obtain a fitting plane is as follows:
performing plane fitting on the edge characteristic three-dimensional point cloud of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located by utilizing an SVD decomposition method principle to obtain a fitting plane P; calculating the average value of the plane fitting error as the standard value of the screening out-of-plane point; and circularly calculating the distance from each point to the fitting plane P, comparing the distance with a standard value, and screening out points far away from the plane.
The process of projecting the edge characteristic three-dimensional point cloud to the fitting plane to calculate projection points comprises the following steps: and projecting the characteristic three-dimensional point cloud to a fitting plane P, calculating the distance from each hexagonal characteristic point to the fitting plane, and calculating the projection of each characteristic point on the fitting plane P according to the normal vector and the distance of the plane.
The 6 sides are separated, and the characteristic point of the hexagon is divided into an upper part and a lower part, wherein each part comprises 3 sides. The upper and lower parts are processed simultaneously. The inflection points of 3 edges are found out first, and then feature point clouds are separated according to the inflection points.
All feature points are traversed from the 4 th point, 3 points forward of the point are taken as one set, 3 points backward are taken as the other set, and the two sets are respectively subjected to linear fitting. Calculating the included angle of two straight lines ɑ If the included angle is ɑ A point approaching 0 degrees or 180 degrees (deviation threshold not exceeding 10 degrees) indicates that the point is a straight line portion, and conversely, the point is recorded as an alternative point to the inflection point region. According to the characteristic that the internal angle of the regular hexagon is 120 degrees, when the upper part and the lower part are simultaneously processed, the angle can be changed from being close to 0 degree (or 180 degrees) to being close to 120 degrees (or 60 degrees) to being close to 0 degree, the upper characteristic point and the lower characteristic point in the transformation process can be respectively changed twice, and two inflection point alternative areas can be recorded.
Each inflection point candidate region performs a secondary search for an inflection point. The judging mode is carried out in the area when the inflection point is found, but only 2 points are selected for each point selection to construct a straight line. After the secondary searching, a inflection point range with a smaller range can be obtained, and after points in the inflection point range are eliminated, the points before and after the inflection point are divided into points on 3 straight lines. The point clouds on 6 sides can be divided by processing the point clouds up and down simultaneously.
The 6 sides calculate the intersection point, all the characteristic points are projected on the same plane in the above steps, but each point is still a three-dimensional point, so that the straight line intersection point is inconvenient to solve, and therefore the three-dimensional point is required to be changed into a two-dimensional point to calculate the intersection point.
Firstly, arbitrarily taking two point construction unit direction vectors from characteristic points
Figure BDA0004087296970000121
X-axis as new coordinate system, in plane normal vector +.>
Figure BDA0004087296970000122
As Z axis +.>
Figure BDA0004087296970000123
The Y-axis can be constructed. Secondly, coordinates of three known points in the new and old coordinate systems are needed for realizing coordinate system conversion. Since 2 points have been selected when constructing the X-axis. Assuming that the selected point is P1, knowing the coordinates of the point in the old coordinate system and defining the point as an origin in the new coordinate system; the other point is P2, and the distance D1 of the point from the P1 is calculated, and then the point P2 is (D1, 0) in the new coordinate system; the third point cannot be directly selected and needs to be manually constructed by calculation, and the P1 point and the plane normal vector are known in the old coordinate system +.>
Figure BDA0004087296970000124
It is possible to construct the next point p3=p1+d2 of the old coordinate system +.>
Figure BDA0004087296970000125
Where D2 is a self-defined distance and can be any number other than 0, then the P3 point is (0, D2) in the new coordinate system. And finally, calculating a transformation relation matrix RT between the new coordinate system and the old coordinate system according to the three points, and transforming the characteristic points on the 6 sides into the new coordinate system through the transformation matrix RT.
The Z coordinates of all the characteristic points in the new coordinate system are the same and 0, so that the three-dimensional points can be changed into two-dimensional points by directly removing the Z values of the points, 6 straight lines on the XOY plane are respectively fitted to the two-dimensional points on 6 sides, the intersection point is obtained by solving the intersection point of every two straight lines, 6 two-dimensional angular points are obtained, and the six points are transformed into the points in the old coordinate system through the RT transformation matrix after the Z values of the six points are added with 0.
And 6 corner points are subjected to circumcircle fitting, and the RANSAC algorithm is utilized to carry out circle fitting on the 6 points. And randomly selecting 3 points to calculate a corresponding space circle equation, calculating the distances di between other points and a circle, setting a threshold value beta, if di < beta, considering the points to be in the circle, otherwise, recording the number of the points in the circle outside the circle. Repeating the steps, and selecting the model parameters corresponding to the spherical surface with the largest number of interior points; and calculating an iteration ending judgment factor according to the expected error rate, the optimal inner point number, the total sample number and the current iteration number at the end of each iteration, and determining whether to stop iteration according to the iteration number. And after iteration is finished, carrying out least square circle fitting on the inner points to obtain a final circle fitting coefficient. The diameter of the circumscribed circle of the hexagon and the midpoint coordinate can be obtained through the fitting.
The diameter of the inscribed circle is calculated by 6 sides. Regular hexagonal inscribed circle diameter = circumscribed circle diameter x cos30 °.
The information required for regular hexagonal measurement includes: center point, circumscribed circle diameter, inscribed circle diameter, circumscribed circle diameter, normal vector, direction vector.
The embodiment 2 of the invention provides a regular hexagonal hole feature extraction system based on laser vision, data to be processed are collected by means of a sensor, the data are processed through an image part and a point cloud part algorithm, parameters of a regular hexagonal hole, including center point coordinates, circumscribed circle diameters, inscribed circle diameters and normal vector information, can be calculated after the processing, and a calculation result can help site workers to evaluate whether processing is reasonable or not and is used for guiding production and processing.
The description of the relevant part in the extraction system of the regular hexagonal hole feature based on the laser vision provided in the embodiment of the application may refer to the detailed description of the corresponding part in the extraction method of the regular hexagonal hole feature based on the laser vision provided in embodiment 1 of the application, which is not repeated here.
It is noted that relational terms such as first and second, and the like are used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Moreover, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements is inherent to. Without further limitation, an element defined by the phrase "comprising one … …" does not exclude the presence of other like elements in a process, method, article, or apparatus that comprises the element. In addition, the parts of the above technical solutions provided in the embodiments of the present application, which are consistent with the implementation principles of the corresponding technical solutions in the prior art, are not described in detail, so that redundant descriptions are avoided.
While the specific embodiments of the present invention have been described above with reference to the drawings, the scope of the present invention is not limited thereto. Other modifications and variations to the present invention will be apparent to those of skill in the art upon review of the foregoing description. It is not necessary here nor is it exhaustive of all embodiments. On the basis of the technical scheme of the invention, various modifications or variations which can be made by the person skilled in the art without the need of creative efforts are still within the protection scope of the invention.

Claims (10)

1. The method for extracting the regular hexagonal hole features based on laser vision is characterized by comprising the following steps of:
acquiring a plurality of images of the regular hexagonal hole scanned by the optical cutter lines, wherein each image has one optical cutter line;
obtaining two-dimensional coordinates of the center of the optical cutter through image processing on each image, converting the two-dimensional coordinates into three-dimensional coordinates, and simultaneously unifying the three-dimensional coordinates into a camera coordinate system to obtain three-dimensional point clouds of edge characteristics of the hexagonal hole and three-dimensional point clouds of a plane where the hexagon is located;
performing plane fitting on the three-dimensional point cloud of the edge characteristic of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located to obtain a fitting plane, screening out points far away from the fitting plane, projecting the three-dimensional point cloud of the edge characteristic to the fitting plane to calculate projection points, separating six sides, and then calculating the central point, the diameter of an circumscribed circle, the diameter of an inscribed circle, the diameter of an circumscribed circle, a normal vector and a direction vector of the hexagonal hole.
2. The method for extracting the regular hexagonal hole features based on laser vision according to claim 1, wherein the process of acquiring the plurality of images of the regular hexagonal hole by optical knife line scanning comprises the following steps:
the swing type line laser sensor is adopted to scan the hexagonal hole to be measured, the image is shot through the image acquisition equipment uniformly according to the swing angle, and each image has one optical knife line.
3. The method for extracting regular hexagonal hole features based on laser vision according to claim 1, wherein the image processing comprises:
selecting a first target area in the image to be processed, and then preprocessing the selected first target area;
searching hexagonal hole contours in a selected first target area, and selecting an upper contour and a lower contour which are longest in the position of the image through circumscribing rectangles by each contour;
and selecting a second target area formed by the circumscribed rectangular area from the original image, and extracting the same second target area range by using a light knife to obtain a two-dimensional sub-pixel coordinate of the center of the light knife.
4. A method for extracting regular hexagonal hole features based on laser vision according to claim 3, wherein the preprocessing comprises:
firstly, carrying out linear smoothing filtering on a first target area by utilizing a Gaussian filtering algorithm;
after linear smoothing filtering, binarization processing is carried out;
after the binarization process, an erosion operation is performed on the region of the pixel above the threshold to find a local minimum using an on operation in the morphological operation.
5. The method for extracting the regular hexagonal hole features based on the laser vision according to claim 1, wherein the plane fitting is performed on the edge feature three-dimensional point cloud of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located to obtain a fitting plane, and the process of screening out the points far from the fitting plane comprises the following steps:
performing plane fitting on the edge characteristic three-dimensional point cloud of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located by utilizing an SVD decomposition principle to obtain a fitted plane;
calculating the average value of the plane fitting error as the standard value of the screening out-of-plane point;
and circularly calculating the distance from each point to the fitting plane, comparing the distance with a standard value, and screening out points far away from the plane.
6. The method for extracting regular hexagonal hole features based on laser vision according to claim 5, wherein the process of projecting the edge feature three-dimensional point cloud to the fitting plane to calculate the projection points comprises: and projecting the edge characteristic three-dimensional point cloud to a fitting plane, calculating the distance between each hexagonal characteristic point and the fitting plane, and calculating the projection of each characteristic point on the fitting plane according to the normal vector and the distance of the plane.
7. The method for extracting regular hexagonal hole features based on laser vision according to claim 6, wherein the process of separating six sides comprises:
dividing the hexagon into two parts, each part comprising three sides; traversing all the characteristic points from a fourth point, wherein the fourth point takes three points forwards as a first set, and takes three points backwards as a second set; the first set and the second set are respectively subjected to straight line fitting;
calculating an included angle of the two straight lines, if the included angle does not exceed an angle deviation threshold value, locating the current point at the straight line part, otherwise, locating the current point as an alternative point of the inflection point region;
and each inflection point alternative area is subjected to secondary inflection point searching, and two points are selected each time to form a straight line. The point clouds on the six sides are drawn by simultaneous processing forward and backward.
8. The method for extracting the regular hexagonal hole features based on laser vision according to claim 7, wherein after six sides are separated, six sides are intersected to obtain six corner points, and the six corner points are subjected to circumcircle fitting and six sides are inscribed to obtain the diameter of the inscribed circle.
9. The method for extracting regular hexagonal hole features based on laser vision according to claim 8, wherein the regular hexagonal inscribed circle diameter = circumscribed circle diameter x cos30 °.
10. The extraction system of the regular hexagon hole features based on the laser vision is characterized by comprising an acquisition module, an image processing module and a fitting calculation module;
the acquisition module is used for acquiring a plurality of images of the regular hexagonal hole scanned by the optical cutter lines, and each image has one optical cutter line;
the image processing module is used for obtaining two-dimensional coordinates of the center of the optical cutter through image processing on each image, converting the two-dimensional coordinates into three-dimensional coordinates, and simultaneously unifying the three-dimensional coordinates into a camera coordinate system to obtain three-dimensional point clouds of edge characteristics of the hexagonal hole and three-dimensional point clouds of a plane where the hexagon is located;
the fitting calculation module is used for carrying out plane fitting on the edge characteristic three-dimensional point cloud of the hexagonal hole and the three-dimensional point cloud of the plane where the hexagon is located to obtain a fitting plane, projecting the edge characteristic three-dimensional point cloud to the fitting plane to calculate projection points after screening points far away from the fitting plane, separating six sides, and then calculating the center point, the circumscribed circle diameter, the inscribed circle diameter, the circumscribed circle diameter, the normal vector and the direction vector of the hexagonal hole.
CN202310140327.8A 2023-02-17 2023-02-17 Method and system for extracting regular hexagonal hole features based on laser vision Pending CN116385356A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202310140327.8A CN116385356A (en) 2023-02-17 2023-02-17 Method and system for extracting regular hexagonal hole features based on laser vision

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202310140327.8A CN116385356A (en) 2023-02-17 2023-02-17 Method and system for extracting regular hexagonal hole features based on laser vision

Publications (1)

Publication Number Publication Date
CN116385356A true CN116385356A (en) 2023-07-04

Family

ID=86970122

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202310140327.8A Pending CN116385356A (en) 2023-02-17 2023-02-17 Method and system for extracting regular hexagonal hole features based on laser vision

Country Status (1)

Country Link
CN (1) CN116385356A (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117710396A (en) * 2023-12-14 2024-03-15 安徽工布智造工业科技有限公司 3D point cloud-based recognition method for nonstandard parts in light steel industry

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117710396A (en) * 2023-12-14 2024-03-15 安徽工布智造工业科技有限公司 3D point cloud-based recognition method for nonstandard parts in light steel industry

Similar Documents

Publication Publication Date Title
CN111299815B (en) Visual detection and laser cutting trajectory planning method for low-gray rubber pad
CN111222516B (en) Method for extracting point cloud key outline features of printed circuit board
US5471541A (en) System for determining the pose of an object which utilizes range profiles and synethic profiles derived from a model
CN111640158B (en) End-to-end camera and laser radar external parameter calibration method based on corresponding mask
CN113393426B (en) Steel rolling plate surface defect detection method
CN113470090A (en) Multi-solid-state laser radar external reference calibration method based on SIFT-SHOT characteristics
CN111539446B (en) Template matching-based 2D laser hole site detection method
CN110232388B (en) Method for identifying honeycomb edge from honeycomb core surface measurement data
CN114037675A (en) Airplane sample plate defect detection method and device
CN113096094A (en) Three-dimensional object surface defect detection method
CN113781585B (en) Online detection method and system for surface defects of additive manufactured parts
CN112907601B (en) Automatic extraction method and device for tunnel arch point cloud based on feature transformation
CN110780276A (en) Tray identification method and system based on laser radar and electronic equipment
CN116385356A (en) Method and system for extracting regular hexagonal hole features based on laser vision
CN111968224A (en) Ship 3D scanning point cloud data processing method
CN116188544A (en) Point cloud registration method combining edge features
CN116310355A (en) Laser point cloud denoising and defect detection method for complex structural member
CN113870326B (en) Structural damage mapping, quantifying and visualizing method based on image and three-dimensional point cloud registration
CN114004899B (en) Pallet pose recognition method, storage medium and equipment
CN117706577A (en) Ship size measurement method based on laser radar three-dimensional point cloud algorithm
CN113705564A (en) Pointer type instrument identification reading method
Wu et al. A Systematic Point Cloud Edge Detection Framework for Automatic Aircraft Skin Milling
CN116579955A (en) New energy battery cell weld reflection point denoising and point cloud complement method and system
CN115971004A (en) Intelligent putty spraying method and system for carriage
CN113689478B (en) Alignment method, device and system of measuring equipment

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination