CN110689579B - Rapid monocular vision pose measurement method and measurement system based on cooperative target - Google Patents
Rapid monocular vision pose measurement method and measurement system based on cooperative target Download PDFInfo
- Publication number
- CN110689579B CN110689579B CN201910993526.7A CN201910993526A CN110689579B CN 110689579 B CN110689579 B CN 110689579B CN 201910993526 A CN201910993526 A CN 201910993526A CN 110689579 B CN110689579 B CN 110689579B
- Authority
- CN
- China
- Prior art keywords
- checkerboard
- target
- image
- rapid
- precision
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/13—Edge detection
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/136—Segmentation; Edge detection involving thresholding
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/194—Segmentation; Edge detection involving foreground-background segmentation
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20112—Image segmentation details
- G06T2207/20164—Salient point detection; Corner detection
Abstract
The invention belongs to the technical field of photoelectric measurement, and discloses a rapid monocular vision pose measurement method and a rapid monocular vision pose measurement system based on a cooperative target, which are used for calibrating and recording camera internal parameters; acquiring an image of a cooperative target object in real time; zooming the image according to the real-time requirement; performing target segmentation based on a color extraction method; positioning color frame angular points, determining the initial positions of the checkerboard angular points by using two-dimensional linear interpolation, and amplifying the checkerboard angular points to the original size; carrying out checkerboard corner sub-pixel positioning according to the scaling; and (4) realizing coordinate transformation according to the PnP algorithm to obtain the position and pose information of the checkerboard target. The invention uses a specially designed cooperative target containing a positioning color frame and a high-precision checkerboard as a target object, the coordinate precision of the corner point is high, the random error can be reduced by carrying out mean processing on multiple points, the corner point identification is realized by a method of reducing the resolution and combining a quick positioning algorithm realized according to the target characteristics, and then the sub-pixel identification is carried out, so that the precision is improved.
Description
Technical Field
The invention belongs to the technical field of photoelectric measurement, and particularly relates to a rapid monocular vision pose measurement method and system based on a cooperative target. The method is particularly suitable for real-time measurement of the terminal pose of machine tools and industrial mechanical arms.
Background
Currently, the closest prior art:
in the fields of aviation, aerospace, navigation and industry, high-speed and high-precision pose measurement in a small range is an important condition for realizing precision detection and high-precision control of processing equipment, the requirements of low-cost and high-precision real-time pose measurement equipment in modern high-precision equipment are greatly improved, and the end pose needs to be measured in real time for precision detection and full closed-loop control of machine tools and robots.
In the real-time measurement of the end pose of industrial equipment such as machine tools, robots, and the like, a laser measuring device is used in the conventional method. The laser measurement needs to be matched with a high-precision measurement target, the high-precision target is difficult to manufacture and large in installation difficulty, laser measurement equipment is expensive, cannot be popularized along with the equipment on a large scale, can be installed only when being used mostly, and cannot monitor the operation process of mass production equipment in real time.
The monocular pose measurement based on the cooperative target needs a cooperative target with feature points arranged as required, and has low difficulty in feature extraction, high feature point extraction precision and high measurement precision. Compared with the traditional measuring method, the monocular vision measurement based on the cooperative target has the advantages of low cost, simplicity in installation, non-contact measurement, no limitation of the shape of an object and the like. However, the existing monocular vision measurement algorithm using the cooperative target has large calculation amount and long calculation time, and cannot meet the real-time requirement of measurement.
The laser measuring equipment has high precision and high real-time performance, but the equipment is expensive and the pose detection capability is insufficient; the traditional vision measurement target object based on the cooperative target needs special customization, has high manufacturing cost, weak position and posture measurement capability and poor real-time performance, and cannot fully meet the measurement requirement of modern processing equipment.
In summary, the problems of the prior art are as follows:
(1) the laser measuring equipment has weak pose measuring capability and lacks a low-cost position and attitude measuring method.
(2) The traditional vision measuring equipment based on the cooperative target has the defects of few target points, insufficient precision, insufficient pose measuring capability and weak real-time property.
(3) The existing high-precision cooperative target positioning method has large calculation amount and low sampling rate, and cannot realize real-time measurement.
The difficulty of solving the technical problems is as follows:
(1) the method has the advantages that the space three-dimensional position and the posture of the target can be measured at low cost, high speed and high precision, one of the difficulties in the prior art is realized, the laser measuring equipment cannot measure the posture information at the same time, and the measuring speed of the vision measuring equipment is low.
(2) In the vision measuring equipment, high measuring precision and high measuring speed are contradictory, and improving the measuring and sampling frequency on the premise of ensuring the measuring precision is one of the difficulties in the prior art.
(3) How to quickly separate a cooperative target object and quickly identify target object information in a complex industrial environment by the vision measuring equipment has certain difficulty.
The significance of solving the technical problems is as follows:
(1) the three-dimensional position and attitude information of the target is measured at low cost, high speed and high precision, and is an important condition for realizing equipment precision detection and high-precision control in the fields of industrial processing and the like.
(2) By using the visual measurement equipment, under the condition of ensuring the measurement precision measurement range, the measurement sampling frequency can be improved by improving the measurement speed, and sufficient data is provided for dynamic precision detection and high-precision control.
(3) The method has the advantages that the target object is quickly separated and the target object information is quickly identified in the complex industrial environment, the calculated amount is effectively reduced, the calculating speed is improved, and the high-speed measurement is realized.
Disclosure of Invention
Aiming at the problems in the prior art, the invention provides a rapid monocular vision pose measuring method and system based on a cooperative target. The invention uses a specially designed checkerboard target capable of rapidly identifying the angular point, uses an image scaling method according to the color frame mark, rapidly determines the direction of the checkerboard, positions the initial position of the angular point of the checkerboard, then uses a sub-pixel division mode to improve the angular point positioning precision, and combines a PNP algorithm to realize rapid and high-precision position and pose measurement of the checkerboard target.
The invention is realized in such a way that a rapid monocular vision pose measurement method based on a cooperative target comprises the following steps: 1) and calibrating a camera before pose measurement.
2) An image of the cooperative target object is acquired.
3) And determining a scaling ratio for image scaling.
4) Image segmentation based on color extraction.
5) The initial positions of the checkerboard corners are determined using a fast corner location method.
6) And (5) carrying out sub-pixel positioning on corner points of the checkerboard.
7) And (5) pose calculation.
Further, the rapid monocular vision pose measurement method based on the cooperative target comprises the following steps: and S1, acquiring and recording the camera internal parameters. Before vision measurement, calibrating camera internal parameters by using a high-precision checkerboard target, and recording a camera internal parameter matrix K:
wherein: fx and fy are parameters representing focal length, x 0 、y 0 S is a parameter indicating the inclination of the axis, and is a parameter indicating the deviation of the principal point;
and S2, acquiring the image of the cooperation object in real time by using the camera. The cooperative target is a specially designed high-precision checkerboard target, as shown in fig. 4, and comprises a rectangular color frame and an internal high-precision checkerboard, wherein the color frame corner points are marked by directional black and white blocks. The rectangular color frame is used for quick positioning under a complex background, the black and white block markers are oriented to determine the target direction, the symmetry influence of the checkerboard is eliminated, and the high-precision checkerboard is combined with sub-pixel processing to obtain the high-precision sub-pixel coordinates of the corner points.
S3, zooming the image according to the real-time requirement. And determining the scaling according to the real-time requirement and the recognition rate, and reducing the image to a proper size.
And S4, target segmentation based on color extraction. The original picture is an RGB space, and is converted into an HSV space, so that color identification is facilitated, and a checkerboard color frame area is extracted. And converting the RGB image into HSV space, extracting the target color and thresholding, and separating the target object from the background.
And S5, determining the initial position of the corner point of the checkerboard. And extracting the outline of the target color area, performing quadrilateral fitting, and identifying the pixel coordinates of the color frame corner points. And comparing the gray values of the marks of the directional black and white blocks of the four color frame corner points to determine the direction of the checkerboard. And calculating the initial position of the checkerboard angular points by using a two-dimensional linear interpolation method according to the coordinate relation between the color frame angular points and the checkerboard angular points. And restoring the initial positions of the identified checkerboard corner points to the original sizes according to the scaling ratio.
And S6, sub-pixel positioning of corner points of the checkerboard. And determining a sub-pixel search range according to the picture scaling, and accurately identifying the sub-pixels of the corner points to obtain a coordinate matrix Pu under a checkerboard image coordinate system.
And S7, coordinate transformation is carried out to obtain the position and pose information of the checkerboard target. And inputting the physical size of the checkerboard target object and the number of the checkerboard rows and columns to calculate to obtain a world coordinate matrix Po. Using a PnP (peer-n-point) algorithm in vision measurement, taking a coordinate matrix Po under a world coordinate system, a coordinate matrix Pu under an image coordinate system and a camera internal reference matrix K corresponding to a checkerboard as PnP algorithm parameters (formula 1), and acquiring a position and posture relation matrix of the checkerboard relative to a camera coordinate system: and (5) translating the matrix (T) and rotating the matrix (R) to complete the posture determination of the target object. And (3) calculating the coordinates Pc of the corner point of the target object in the camera coordinate system by combining the formula (2) and the world coordinate matrix Po. And processing the Pc mean value to obtain the coordinates of the center point of the target object, and finishing the position determination of the target.
[R,T]＝PnP(Po,Pu,K) (1)
Pc＝(R×Po T +T) T (2)
Further, in S3, an image scaling k is determined according to the real-time requirement (when the scaling k is 1/2, 1/4, 1/8, the scaling operation calculation speed is fast), and the image is subjected to size compression by using a fast reduction algorithm, so that the image resolution is reduced, and the processing speed is increased.
Further, the outer contour is extracted in S5, quadrilateral fitting is performed to obtain coordinates of four corner points of a fitted quadrilateral, the coordinate pose of the directional black-and-white marking color block at this time is determined according to the coordinate relationship, and the direction of the checkerboard is determined by calculating the gray value of the marking color block corresponding to the four corner points.
When the direction of the checkerboard is determined, the coordinate position corresponding to the checkerboard angular point is calculated by using a two-dimensional linear interpolation method according to the relation between the color frame angular point coordinates and the checkerboard angular point coordinates. And finally, restoring the initial position of the identified checkerboard corner point to the original size according to the scaling ratio k of the picture.
Because the calculation is accelerated by adopting a resolution reduction method, the preliminary identification precision of the checkerboard is reduced by the method, and the checkerboard corner points are in a k multiplied by k rectangle with the obtained pixel coordinates as the center, so that the accurate pixel position cannot be determined.
Further, a sub-pixel corner processing algorithm is used in S6, and due to the reduction of the recognition accuracy in S5, the search range in the algorithm parameters needs to be properly expanded in the sub-pixel processing, so as to realize the accurate recognition of the checkerboard corners. The angular point identification precision can reach 0.1 pixel to 0.01 pixel, the vision measurement precision is greatly improved, and the processing precision processing time of the sub-pixel angular point processing algorithm is related to the iteration times of the algorithm. According to the real-time requirement, a proper iteration precision and number control threshold is selected to obtain proper calculation time so as to meet the requirements of the sub-pixel processing algorithm on precision and real-time.
The invention further aims to provide an information data processing terminal for realizing the rapid monocular vision pose measuring method based on the cooperative target.
Another object of the present invention is to provide a computer-readable storage medium, which includes instructions that, when executed on a computer, cause the computer to execute the above fast monocular vision pose measurement method based on cooperative targets.
Another object of the present invention is to provide a rapid monocular vision pose measurement system based on cooperative targets, which implements the rapid monocular vision pose measurement method based on cooperative targets, the rapid monocular vision pose measurement system based on cooperative targets comprising:
the camera calibration module is used for calibrating the camera before the step posture measurement;
the cooperation target object image acquisition module is used for acquiring an image of a cooperation target object;
the image scaling module is used for determining the scaling to carry out image scaling;
an image segmentation module for image segmentation based on color extraction;
the initial position determining module is used for determining the initial positions of the checkerboard angular points by using a rapid angular point positioning method;
the sub-pixel positioning module is used for sub-pixel positioning of the checkerboard angular points;
and the pose calculation module is used for calculating the pose.
In summary, the advantages and positive effects of the invention are:
compared with the traditional method, the method has the following advantages:
according to the rapid monocular vision pose measuring method based on the cooperative target, provided by the invention, before vision measurement, internal parameters of a camera are calibrated and recorded by using a high-precision checkerboard target; acquiring images of the cooperative target object in real time by using a camera; zooming the image according to the real-time requirement; performing target segmentation based on a color extraction method; positioning color frame angular points, determining the direction of the checkerboard according to the orientation marks, determining the initial positions of the checkerboard angular points by using two-dimensional linear interpolation, and amplifying the checkerboard angular points to the original size; carrying out checkerboard corner sub-pixel positioning according to the scaling; and (4) realizing coordinate transformation according to the PnP algorithm to obtain the position and pose information of the checkerboard target. The invention uses a specially designed cooperative target containing a positioning color frame and a high-precision checkerboard as a target object, the coordinate precision of the corner point is high, the random error can be reduced by carrying out mean processing on multiple points, the corner point identification is realized by a method of reducing the resolution and combining a quick positioning algorithm realized according to the target characteristics, and then the sub-pixel identification is carried out, so that the precision is improved. The method improves the processing speed of the angular point coordinates under the condition of ensuring the identification precision and meets the real-time requirement of small-range high-precision rapid vision measurement.
Compared with the prior art, the invention has the advantages that:
the invention has high measurement precision. The position and the posture of the target object under a camera coordinate system can be obtained by using the high-precision checkerboard target object, the high precision is realized in small-range measurement (20cm x 20cm), the measurement precision can reach 0.01mm, the checkerboard target object can be widely applied to measurement of modern industrial equipment (machine tools and industrial mechanical arms) by fixing the checkerboard target object on the measurement target, and the requirement of the modern industrial equipment on small-range high-precision posture measurement is met. Compared with the traditional measuring equipment, the method has the advantages of low equipment cost and high pose measuring precision. Compared with the traditional monocular vision measurement method, the high-precision chessboard pattern target object corner point precision is high, and random errors can be reduced by multi-point mean processing.
The invention has high measuring speed. The method uses a specially designed cooperative target containing a positioning color frame and a high-precision checkerboard as a target object. The method for identifying the corner points and then identifying the sub-pixels by the method for reducing the resolution and the method for identifying the corner points and then identifying the sub-pixels by the quick positioning algorithm realized according to the target characteristics greatly shortens the corner point identification time of the target object on the premise of ensuring the identification precision, realizes the quick processing of high-pixel pictures, has the speed of 50 frames/second, and meets the real-time requirement of measuring equipment under the general condition.
Drawings
Fig. 1 is a flowchart of a method for measuring a quick monocular vision pose based on a cooperative target according to an embodiment of the present invention.
Fig. 2 is a flowchart of a specific implementation of the cooperative-target-based fast monocular vision pose measurement method according to the embodiment of the present invention.
Fig. 3 is a flowchart of fast sub-pixel corner location provided by an embodiment of the present invention.
Fig. 4 is a schematic diagram of a specially designed checkerboard target provided by an embodiment of the present invention.
FIG. 5 is a schematic diagram of a rapid monocular vision pose measurement system based on cooperative targets according to an embodiment of the present invention.
In the figure: 1. a camera calibration module; 2. a cooperative target image acquisition module; 3. an image scaling module; 4. an image segmentation module; 5. an initial position determination module; 6. a sub-pixel positioning module; 7. and a pose calculation module.
Detailed Description
In order to make the objects, technical solutions and advantages of the present invention more apparent, the present invention is further described in detail with reference to the following embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and do not limit the invention.
The traditional vision measuring equipment based on the cooperative target has the defects of few target points, insufficient precision, insufficient pose measuring capability and weak real-time property. And the existing laser measuring equipment is expensive and high in cost.
Aiming at the problems in the prior art, the invention provides a rapid monocular vision pose measurement method and a measurement system based on a cooperative target, and the invention is described in detail below with reference to the attached drawings.
As shown in fig. 1, the method for measuring a pose of a quick monocular vision based on a cooperative target according to the embodiment of the present invention has the following main processes:
and S101, calibrating a camera before pose measurement.
S102, acquiring an image of the cooperation target object.
S103, determining a scaling ratio to zoom the image.
And S104, segmenting the image based on the color extraction.
And S105, determining the initial positions of the checkerboard corners by using a rapid corner positioning method.
And S106, carrying out sub-pixel positioning on corner points of the checkerboard.
And S107, pose calculation.
The invention is further described with reference to specific examples.
Examples
In the example, a checkerboard target with a checkerboard number of 9 × 12 having red frames is used as a target, the checkerboard arrangement is shown in fig. 4, the number of corner points is 8 × 11, and the length of each checkerboard side is 3 mm. An 1100 ten thousand pixel camera is used as an image acquisition device, and the image size is 3840 pixels by 2880 pixels.
As shown in fig. 2, the implementation steps of the rapid monocular vision pose measurement method based on the cooperative target provided by the embodiment of the present invention are as follows:
s1, before pose measurement, calibrating the used camera by using the checkerboard target according to the Zhang camera calibration method to obtain a camera internal reference matrix as follows:
satisfy the requirement of
Form wherein the principal point offset is [ 1738.24272878019; 1502.02983971068]A pixel.
And S2, acquiring an image containing the cooperative target object by using a camera, wherein the included angle between the checkerboard and the camera is less than 30 degrees, the checkerboard and the camera are positioned at a clear imaging position of the camera, and the image occupies more than the whole picture 1/9.
S3, according to the test, the calculation speed and accuracy are more balanced when the picture length is reduced to 500 pixels, and when the obtained scaling k is 500/3880-0.1289 and k is 0.125, the fast zooming can be realized.
S4: based on the object segmentation of color extraction, the RGB picture is converted into HSV space, and this example uses an object having a red color frame, and HSV threshold of red is [0,10], [156,180], and appropriately expanded for ensuring effect, and the recognition threshold is set to [0,15], [140,180 ]. And identifying red and thresholding, separating a color frame from a background, and reducing the operation amount.
S5: and (5) extracting the outer contour in the thresholded image in S4, denoising by a contour area method, removing red interference with a small area, performing quadrilateral fitting on the remaining contour, and positioning the pixel coordinates of the color frame corner point of the cooperative target object.
And determining the coordinate of the directional color block by using two-dimensional linear interpolation according to the coordinates of the corner points of the color frame, taking the color block with proper size to obtain a grayed average value, sequencing and comparing, and reordering the corner points of the color frame according to a sequencing result to ensure that the corner points of the color frame meet the requirement of the checkerboard direction. And calculating the initial position of the checkerboard corner points by using two-dimensional linear interpolation according to the coordinate relation between the color frame corner points and the checkerboard corner points, and reducing to the original size according to the scaling ratio k which is 0.125. And sequentially arranging all the checkerboard corner points into a coordinate matrix under a matrix Pu checkerboard image coordinate system with 88 rows and 2 columns.
In addition, if the checkerboard corner processing of the compressed picture fails, operations such as coordinate scaling and coordinate transformation are not performed according to the flow chart, and error information is directly returned.
S6: the tessellated corner sub-pixel locations (see fig. 3). The obtained corner position precision is reduced by using a compression acceleration algorithm, parameters related to a search range in a sub-pixel processing algorithm are expanded appropriately, the initial search range is set to [3,3], in the example, the scaling k is 0.125, the search range in the example is [3,3]/k, namely [24,24], namely, the sub-pixel corners are searched in a rectangular window with the side length of [24 × 2+1,24 × 2+1] taken as the center point at the corner position shown by the pixel precision.
Because the accuracy requirement of the invention on the corner point identification is higher, the iteration frequency is set to be 200 in the example, and the iteration control accuracy is 0.01.
S7: the side length of a single checkerboard square is 3mm, and an array of a world coordinate system of the corner points is calculated and obtained according to the arrangement sequence of the checkerboards. The number of the angular points of the checkerboards is 11 × 8, the side length of the checkerboards is 3mm, the checkerboards are placed on a plane where zw is 0 under world coordinates, and the checkerboards are sequentially arranged into a matrix Po with 88 rows and 3 columns.
And (3) acquiring a position and posture relation matrix of the checkerboard relative to a camera coordinate system by using a PnP algorithm: translation matrix (T matrix), rotation matrix (R matrix). Substituting the sub-pixel coordinates of the checkerboard angular points, the world coordinates of the checkerboard angular points and the internal reference parameter matrix of the camera into a PnP algorithm to obtain a translation matrix (T matrix) and a rotation matrix (R matrix).
Using equation (2) (Pc ═ R × Po T +T) T ) And calculating coordinates Pc of the corner point of the target object in a camera coordinate system by combining the world coordinate matrix Po of the target object.
The coordinates Pc of the corner point of the target object under the camera coordinate system are subjected to mean processing, the influence of random errors is reduced, the precision of visual identification is improved, and the coordinates of the checkerboard center point under the camera coordinate system are obtained
The R matrix is a rotation matrix of the checkerboard in the camera coordinate system.
As a preferred embodiment, step S7 further implements the steps of: and (5) coordinate transformation is carried out to obtain the position and pose information of the checkerboard target. And inputting the physical size of the checkerboard target object and the number of the checkerboard rows and columns to calculate to obtain a world coordinate matrix Po. Using a PnP (peer-n-point) algorithm in vision measurement, taking a coordinate matrix Po under a world coordinate system, a coordinate matrix Pu under an image coordinate system and a camera internal reference matrix K corresponding to a checkerboard as PnP algorithm parameters (formula 1), and acquiring a position and posture relation matrix of the checkerboard relative to a camera coordinate system: and (5) translating the matrix (T) and rotating the matrix (R) to complete the posture determination of the target object. And (3) calculating the coordinates Pc of the corner point of the target object in the camera coordinate system by combining the formula (2) and the world coordinate matrix Po. And processing the Pc mean value to obtain the coordinates of the center point of the target object, and finishing the position determination of the target.
[R,T]＝PnP(Po,Pu,K) (1)
Pc＝(R×Po T +T) T (2)
The same position is repeatedly measured for multiple times, the result is subjected to precision analysis, when the range of the Z direction is 50-150mm, the measurement precision of the example can reach 0.005mm, compared with the traditional vision measurement equipment, the precision is higher, the calculation time is about 20ms, and the real-time measurement requirement under the common condition can be met.
The monocular vision real-time pose measuring method based on the cooperative target expands the selection of the modern industrial equipment on a small-range high-speed high-precision measuring method. The invention uses a specially designed checkerboard target with a color frame and direction marks as a cooperative target, has high angular point sub-pixel detection precision, performs mean processing on multi-angular points to reduce random errors, can measure the position information and the attitude information of a target object in real time, and provides feedback data for the control of equipment. The invention can be used in the field of pose measurement for setting cooperative targets, such as measurement of positioning accuracy of machine tools, measurement of positioning accuracy of tail ends of industrial mechanical arms, measurement of position relationship among assembly parts, and the like.
As shown in fig. 5, the fast monocular vision pose measurement system based on the cooperative target of the present invention includes:
and the camera calibration module 1 is used for calibrating the camera before step pose measurement.
And the cooperation object image acquisition module 2 is used for acquiring an image of the cooperation object.
And the image scaling module 3 is used for determining the scaling to perform image scaling.
And the image segmentation module 4 is used for image segmentation based on color extraction.
An initial position determining module 5, configured to determine initial positions of the checkerboard corners by using a fast corner positioning method.
And the sub-pixel positioning module 6 is used for sub-pixel positioning of the checkerboard corner points.
And the pose calculation module 7 is used for calculating the pose.
The invention is further described below with reference to specific embodiments.
Examples
1. Software and hardware components
The hardware part mainly comprises camera model, calibration board data and computer related configuration.
TABLE 1 Camera-related parameters
TABLE 2 checkerboard calibration plate related parameters
TABLE 3 image processing calculation configuration parameters
TABLE 4 software versions
2. Camera calibration
Before the pose measurement is carried out, the used camera is calibrated by using a checkerboard target according to a Zhang camera calibration method, and the obtained camera internal reference matrix is as follows:
3. specific measurement procedure
3.1 image acquisition
And (3) acquiring an image containing the cooperative target object by using a camera, wherein the included angle between the checkerboard and the camera is less than 30 degrees, the checkerboard and the camera are positioned at a clear imaging position of the camera, and the image occupies more than the whole picture 1/9.
3.2 image scaling
In this example, when the picture length after the reduction is about 500 pixels, the calculation speed and the accuracy are balanced, and the scaling k is obtained by changing the scaling k to 500/3880-0.1289 and changing the scaling k to 0.125.
3.3 object segmentation extraction
Based on the object segmentation of color extraction, the RGB picture is converted into HSV space, and this example uses an object having a red color frame, and HSV threshold of red is [0,10], [156,180], and appropriately expanded for ensuring effect, and the recognition threshold is set to [0,15], [140,180 ]. And identifying red, thresholding, separating a color frame from a background and reducing the operation amount.
And extracting the outer contour in the thresholded image in the last step, denoising by a contour area method, removing red interference with a smaller area, and carrying out quadrilateral fitting on the residual contour to position the pixel coordinates of the color frame corner points of the cooperative target object.
And determining the coordinates of the directional color block by using two-dimensional linear interpolation according to the coordinates of the color block corner points, calculating the initial position of the checkerboard corner points by using the two-dimensional linear interpolation according to the coordinate relation between the color block corner points and the checkerboard corner points, and reducing the original size according to the scaling ratio k equal to 0.125. And sequentially arranging all the checkerboard corner points into a coordinate matrix under a matrix Pu checkerboard image coordinate system with 88 rows and 2 columns.
3.4 sub-Pixel localization
The sub-pixel processing algorithm initially finds a range of [3,3], in this example the scaling k is 0.125, and this example finds a range of [3,3]/k, i.e. [24,24], i.e. the sub-pixel corner is found within a rectangular window with a side length of [24 × 2+1,24 × 2+1] - [49,49] with the corner position indicated by pixel accuracy as the center point. In this example, the number of iterations of the subpixel calculation is set to 200, and the iteration control precision is 0.01.
3.5 pose calculation
The side length of a single checkerboard square is 3mm, and an array of a world coordinate system of the corner points is calculated and obtained according to the arrangement sequence of the checkerboards. The number of the angular points of the checkerboards is 11 × 8, the side length of the checkerboards is 3mm, the checkerboards are placed on a plane where zw is 0 under world coordinates, and the checkerboards are sequentially arranged into 88 rows and 3 columns of matrixes Po.
And (3) acquiring a position and posture relation matrix of the checkerboard relative to a camera coordinate system by using a PnP algorithm: translation matrix (T matrix), rotation matrix (R matrix). Substituting the sub-pixel coordinates of the checkerboard angular points, the world coordinates of the checkerboard angular points and the internal reference parameter matrix of the camera into a PnP algorithm to obtain a translation matrix (T matrix) and a rotation matrix (R matrix).
And calculating the coordinates Pc of the corner point of the target object in a camera coordinate system by using a formula.
TABLE 4
| X/mm | Y/mm | Z/mm | |
| Pc | -16.1532515 | 2.271247021 | 135.2428603 |
The same position is repeatedly measured for multiple times, the result is subjected to precision analysis, when the distance in the Z direction is about 135mm, the standard deviation of the measurement result of the system is 0.004096676mm, the measurement precision of the system can reach 0.005mm, compared with the traditional vision measurement equipment, the system has higher precision and the calculation time is about 20ms, and the real-time measurement requirement under the common condition can be met.
TABLE 5 data and analysis of multiple measurements
In the above embodiments, all or part of the implementation may be realized by software, hardware, firmware, or any combination thereof. When used in whole or in part, can be implemented in a computer program product that includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the procedures or functions according to the embodiments of the present invention are wholly or partially generated. The computer may be a general purpose computer, a special purpose computer, a network of computers, or other programmable device. The computer instructions may be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another computer readable storage medium, for example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wire (e.g., coaxial cable, fiber optic, Digital Subscriber Line (DSL), or wireless (e.g., infrared, wireless, microwave, etc.)). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device, such as a server, a data center, etc., that includes one or more of the available media. The usable medium may be a magnetic medium (e.g., floppy Disk, hard Disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., Solid State Disk (SSD)), among others.
The above description is only for the purpose of illustrating the preferred embodiments of the present invention and is not to be construed as limiting the invention, and any modifications, equivalents and improvements made within the spirit and principle of the present invention are intended to be included within the scope of the present invention.
Claims (9)
1. A quick monocular vision pose measurement method based on a cooperative target is characterized by comprising the following steps:
firstly, calibrating a camera before pose measurement;
acquiring an image of a cooperative target object;
step three, determining a scaling ratio to zoom the image;
step four, image segmentation based on color extraction;
step five, determining the initial positions of the checkerboard angular points by using a rapid angular point positioning method;
step six, sub-pixel positioning of checkerboard angular points;
step seven, pose calculation;
the fifth step specifically comprises: extracting the outline of the target color area, performing quadrilateral fitting, and identifying the pixel coordinates of the color frame corner points; comparing the gray values of the marks of the directional black and white blocks of the four color frame corner points to determine the direction of the checkerboard; calculating the initial position of the checkerboard angular points by using a two-dimensional linear interpolation method according to the coordinate relation between the color frame angular points and the checkerboard angular points; restoring the initial position of the identified checkerboard corner point to the original size according to the scaling ratio;
extracting an outer contour, performing quadrilateral fitting to obtain coordinates of four corner points of a fitted quadrilateral, determining the coordinate pose of the directional black-white marking color block at the moment according to the coordinate relation, calculating the gray value of the marking color block corresponding to the four corner points, and determining the direction of the checkerboard;
when the direction of the checkerboard is determined, calculating the coordinate position corresponding to the checkerboard angular point by using a two-dimensional linear interpolation method according to the relation between the color frame angular point coordinates and the checkerboard angular point coordinates; and finally, restoring the initial position of the identified checkerboard corner point to the original size according to the scaling ratio k of the picture.
2. The cooperative target-based rapid monocular vision pose measurement method of claim 1, wherein step one specifically comprises: before vision measurement, calibrating camera internal parameters by using a high-precision checkerboard target and recording; and recording a camera internal reference matrix K:
wherein:fx and fy are parameters representing focal length, x 0 、y 0 In order to indicate the principal point shift, s is a parameter indicating the axis tilt.
3. The cooperative target based rapid monocular vision pose measurement method of claim 1, wherein step two specifically comprises: acquiring images of the cooperative target object in real time by using a camera; the cooperation target is a high-precision checkerboard target and comprises a rectangular color frame and an internal high-precision checkerboard, and color frame corner points are provided with directional color block marks with different colors; the rectangular color block is used for quick positioning under a complex background, the target direction is determined by directional color block marks, the symmetry influence of the checkerboards is eliminated, and the high-precision checkerboards are combined with sub-pixel processing to obtain the high-precision sub-pixel coordinates of the angular points.
4. The cooperative target based rapid monocular vision pose measurement method of claim 1, wherein step three specifically comprises: zooming the image according to the real-time requirement, determining a zooming ratio k according to the real-time requirement and the recognition rate, taking 1/2, 1/4 and 1/8, and performing size compression on the image by using a rapid reduction algorithm;
the fourth step specifically comprises: the method comprises the steps of converting an original image into an RGB space, performing color identification after converting the original image into an HSV space, extracting a checkerboard color frame region, converting the RGB image into the HSV space, extracting a target color, thresholding, and separating a target object from a background.
5. The cooperative target-based rapid monocular vision pose measurement method of claim 1, wherein step six specifically comprises: and determining a sub-pixel search range according to the picture scaling, accurately identifying the sub-pixels of the corner points, and obtaining a coordinate matrix Pu under a checkerboard image coordinate system.
6. The cooperative target based rapid monocular vision pose measurement method of claim 1, wherein step seven specifically comprises: inputting the physical size of the checkerboard target object and the number of the checkerboard rows and columns to calculate to obtain a world coordinate matrix Po; using a PnP algorithm in vision measurement, taking a coordinate matrix Po under a world coordinate system, a coordinate matrix Pu under an image coordinate system and a camera internal reference matrix K which correspond to the checkerboard as a PnP algorithm parameter formula (1), and acquiring a position and posture relation matrix of the checkerboard relative to a camera coordinate system: translating the matrix (T) and rotating the matrix (R) to complete the attitude determination of the target object; calculating the coordinate Pc of the corner point of the target object under a camera coordinate system by combining a formula (2) and a world coordinate matrix Po; processing the Pc mean value to obtain the coordinates of the center point of the target object, and finishing the position determination of the target;
[R,T]＝PnP(Po,Pu,K) (1)；
Pc＝(R×Po T +T) T (2)；
7. an information data processing terminal for realizing the rapid monocular vision pose measuring method based on the cooperative target as described in any one of claims 1-6.
8. A computer-readable storage medium comprising instructions that, when executed on a computer, cause the computer to perform the cooperative-target-based fast monocular visual pose measuring method of any one of claims 1-6.
9. A rapid monocular vision pose measurement system based on a cooperative target for realizing the rapid monocular vision pose measurement method based on the cooperative target as described in any one of claims 1 to 6, wherein the rapid monocular vision pose measurement system based on the cooperative target comprises:
the camera calibration module is used for calibrating the camera before the step posture measurement;
the cooperation target object image acquisition module is used for acquiring an image of a cooperation target object;
the image scaling module is used for determining the scaling to carry out image scaling;
an image segmentation module for image segmentation based on color extraction;
the initial position determining module is used for determining the initial positions of the checkerboard angular points by using a rapid angular point positioning method;
the sub-pixel positioning module is used for sub-pixel positioning of the checkerboard angular points;
and the pose calculation module is used for calculating the pose.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910993526.7A CN110689579B (en) | 2019-10-18 | 2019-10-18 | Rapid monocular vision pose measurement method and measurement system based on cooperative target |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910993526.7A CN110689579B (en) | 2019-10-18 | 2019-10-18 | Rapid monocular vision pose measurement method and measurement system based on cooperative target |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110689579A CN110689579A (en) | 2020-01-14 |
| CN110689579B true CN110689579B (en) | 2022-08-30 |
Family
ID=69113287
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910993526.7A Active CN110689579B (en) | 2019-10-18 | 2019-10-18 | Rapid monocular vision pose measurement method and measurement system based on cooperative target |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110689579B (en) |
Families Citing this family (25)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111604598B (en) * | 2020-05-06 | 2021-04-20 | 大连理工大学 | Tool setting method of mechanical arm feeding type laser etching system |
| CN111531547B (en) * | 2020-05-26 | 2021-10-26 | 华中科技大学 | Robot calibration and detection method based on vision measurement |
| CN111784779A (en) * | 2020-06-15 | 2020-10-16 | 南京理工大学 | Checkerboard image recognition and positioning system and method based on convolutional neural network and nested contour recognition |
| CN111833405B (en) * | 2020-07-27 | 2023-12-08 | 北京大华旺达科技有限公司 | Calibration and identification method and device based on machine vision |
| CN111882615B (en) * | 2020-07-30 | 2024-03-12 | 珠海市新德汇信息技术有限公司 | Card direction identification method and device based on characteristic color block and self-service equipment |
| CN111814918B (en) * | 2020-08-31 | 2020-12-29 | 中国计量大学 | Cooperative target and target identification method |
| CN112200778A (en) * | 2020-09-27 | 2021-01-08 | 桂林量具刃具有限责任公司 | Non-contact multi-cable type parameter measurement method and system |
| CN112258455A (en) * | 2020-09-28 | 2021-01-22 | 上海工程技术大学 | Detection method for detecting spatial position of part based on monocular vision |
| CN112229323B (en) * | 2020-09-29 | 2022-07-05 | 华南农业大学 | Six-degree-of-freedom measurement method of checkerboard cooperative target based on monocular vision of mobile phone and application of six-degree-of-freedom measurement method |
| CN112146834B (en) * | 2020-09-30 | 2022-10-14 | 石家庄铁道大学 | Method and device for measuring structural vibration displacement |
| CN112241697B (en) * | 2020-10-10 | 2023-03-24 | 苏州欧菲光科技有限公司 | Corner color determination method and device, terminal device and readable storage medium |
| CN112284330A (en) * | 2020-11-18 | 2021-01-29 | 青岛科美创视智能科技有限公司 | Distance measuring system and method based on fisheye camera |
| CN112465898B (en) * | 2020-11-20 | 2023-01-03 | 上海交通大学 | Object 3D pose tag acquisition method based on checkerboard calibration plate |
| CN112785656B (en) * | 2021-01-29 | 2023-11-10 | 北京罗克维尔斯科技有限公司 | Calibration method and device of dual-stereoscopic camera, electronic equipment and storage medium |
| CN113043334B (en) * | 2021-02-23 | 2022-12-06 | 上海埃奇机器人技术有限公司 | Robot-based photovoltaic cell string positioning method |
| CN113240747B (en) * | 2021-04-21 | 2022-08-26 | 浙江大学 | Outdoor structure vibration displacement automatic monitoring method based on computer vision |
| CN113112552B (en) * | 2021-04-23 | 2022-05-27 | 上汽大通汽车有限公司 | Checkerboard checking method based on deep learning semantic segmentation |
| CN113405528B (en) * | 2021-06-18 | 2023-02-24 | 天津市勘察设计院集团有限公司 | Total station assisted ball machine attitude measurement and leveling method and device |
| US11423576B1 (en) | 2021-07-23 | 2022-08-23 | Ford Global Technologies, Llc | Infrastructure node localization with respect to alternative frame of reference |
| CN113834638A (en) * | 2021-09-03 | 2021-12-24 | 大连中科创达软件有限公司 | Method, device and equipment for determining field angle and storage medium |
| CN114511498A (en) * | 2021-12-27 | 2022-05-17 | 中国航天空气动力技术研究院 | Propeller vibration measuring method based on monocular vision |
| CN114910076B (en) * | 2022-05-20 | 2024-04-05 | 泉州装备制造研究所 | Outdoor camera positioning method and device based on GPS and IMU information |
| CN114926526B (en) * | 2022-05-23 | 2023-05-05 | 南京航空航天大学 | Pose measurement method based on zoom camera |
| CN115147411B (en) * | 2022-08-30 | 2022-11-22 | 启东赢维数据信息科技有限公司 | Labeler intelligent positioning method based on artificial intelligence |
| CN116592899B (en) * | 2023-04-28 | 2024-03-29 | 哈尔滨工业大学 | Pose measurement system based on modularized infrared targets |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104729534A (en) * | 2015-03-12 | 2015-06-24 | 北京空间飞行器总体设计部 | Monocular visual error measurement system for cooperative target and error limit quantification method |
| CN108257184A (en) * | 2017-12-28 | 2018-07-06 | 中国科学院西安光学精密机械研究所 | A kind of camera attitude measurement method based on square dot matrix cooperative target |
| CN108510543A (en) * | 2018-03-23 | 2018-09-07 | 湖南科技大学 | A kind of gridiron pattern characteristics of image center sub-pixel positioning method |
| CN108648240A (en) * | 2018-05-11 | 2018-10-12 | 东南大学 | Based on a non-overlapping visual field camera posture scaling method for cloud characteristics map registration |
| WO2019058582A1 (en) * | 2017-09-21 | 2019-03-28 | 株式会社日立製作所 | Distance estimation device and method |
-
2019
- 2019-10-18 CN CN201910993526.7A patent/CN110689579B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104729534A (en) * | 2015-03-12 | 2015-06-24 | 北京空间飞行器总体设计部 | Monocular visual error measurement system for cooperative target and error limit quantification method |
| WO2019058582A1 (en) * | 2017-09-21 | 2019-03-28 | 株式会社日立製作所 | Distance estimation device and method |
| CN108257184A (en) * | 2017-12-28 | 2018-07-06 | 中国科学院西安光学精密机械研究所 | A kind of camera attitude measurement method based on square dot matrix cooperative target |
| CN108510543A (en) * | 2018-03-23 | 2018-09-07 | 湖南科技大学 | A kind of gridiron pattern characteristics of image center sub-pixel positioning method |
| CN108648240A (en) * | 2018-05-11 | 2018-10-12 | 东南大学 | Based on a non-overlapping visual field camera posture scaling method for cloud characteristics map registration |
Non-Patent Citations (2)
| Title |
|---|
| "一种单目物体六自由度位姿检测方法";李若尘 等;《光学与光电技术》;20190228;第17卷(第1期);全文 * |
| "单目运动目标的位姿实时测量方法";王震 等;《测绘科学》;20190331;第44卷(第3期);全文 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110689579A (en) | 2020-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110689579B (en) | Rapid monocular vision pose measurement method and measurement system based on cooperative target | |
| CN110276808B (en) | Method for measuring unevenness of glass plate by combining single camera with two-dimensional code | |
| JP3735344B2 (en) | Calibration apparatus, calibration method, and calibration program | |
| CN109801333B (en) | Volume measurement method, device and system and computing equipment | |
| Kurka et al. | Applications of image processing in robotics and instrumentation | |
| CN107977996B (en) | Space target positioning method based on target calibration positioning model | |
| CN111667536A (en) | Parameter calibration method based on zoom camera depth estimation | |
| CN112132907B (en) | Camera calibration method and device, electronic equipment and storage medium | |
| CN110223355B (en) | Feature mark point matching method based on dual epipolar constraint | |
| CN109974618B (en) | Global calibration method of multi-sensor vision measurement system | |
| CN109272555B (en) | External parameter obtaining and calibrating method for RGB-D camera | |
| CN113119129A (en) | Monocular distance measurement positioning method based on standard ball | |
| CN111707187A (en) | Measuring method and system for large part | |
| CN109448043A (en) | Standing tree height extracting method under plane restriction | |
| Ding et al. | A robust detection method of control points for calibration and measurement with defocused images | |
| JP2016218815A (en) | Calibration device and method for line sensor camera | |
| CN115187612A (en) | Plane area measuring method, device and system based on machine vision | |
| CN113963067A (en) | Calibration method for calibrating large-view-field visual sensor by using small target | |
| KR102023087B1 (en) | Method for camera calibration | |
| CN114037987A (en) | Intelligent identification method, device, medium and equipment for scrap steel | |
| CN111524193B (en) | Method and device for measuring two-dimensional size of object | |
| CN110458951B (en) | Modeling data acquisition method and related device for power grid pole tower | |
| CN113221953A (en) | Target attitude identification system and method based on example segmentation and binocular depth estimation | |
| CN111145247B (en) | Position degree detection method based on vision, robot and computer storage medium | |
| CN116205993A (en) | Double-telecentric lens high-precision calibration method for 3D AOI |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |