CN115082555A - High-precision displacement real-time measurement system and method of RGBD monocular camera - Google Patents

Abstract

The invention provides a high-precision real-time displacement measurement system and method of an RGBD monocular camera, and solves the problem that self-calibration in the displacement measurement process cannot be directly realized through monocular vision in the prior art. The measuring system disclosed by the invention comprises a target identification module, an RGBD camera measuring module and a displacement real-time calculating module; the specific method comprises the following steps: obtaining pixel coordinates of the central points of the left and right side identification objects by a target identification module by adopting an angular point detection algorithm; measuring by adopting an RGBD camera measuring module to obtain a world coordinate value of the central point of the left recognition object; and calculating a camera matrix according to the pixel coordinate and the world coordinate value of the central point of the left identification object by adopting a displacement real-time calculation module, further completing self-calibration of the camera, and calculating the world coordinate value of the central point of the right identification object by combining the pixel coordinate of the central point of the right identification object and the calculated camera matrix, thereby realizing high-precision real-time measurement of displacement.

Description

A high-precision displacement real-time measurement system and method for RGBD monocular camera

technical field

The invention belongs to the field of non-contact measurement of object displacement in three-dimensional space, and in particular relates to a high-precision displacement real-time measurement system and method of an RGBD monocular camera.

Background technique

With the continuous improvement of deep learning theory and the rapid development of computer vision technology, many new methods of non-contact measurement of object displacement have emerged, such as measurement methods based on monocular vision, measurement methods based on binocular stereo vision, and visual measurement based on RGBD cameras. method etc. In recent years, the non-contact measurement of displacement and other physical quantities based on RGBD cameras has made the most rapid progress, thanks to the rapid development of deep learning algorithms and vision chips.

The non-contact measurement and analysis of the 3D displacement field in the aviation and aerospace fields has always attracted much attention. How to use the existing hardware and vision chips to achieve high-precision measurement of the 3D displacement field is a challenging research topic. At present, there are few algorithms for measuring object displacement combined with RGBD monocular cameras, and the accuracy of measurement results obtained by different measurement algorithms is quite different.

SUMMARY OF THE INVENTION

The purpose of the present invention is to provide a high-precision displacement real-time measurement system and method for an RGBD monocular camera, which mainly solves the problem that the prior art cannot measure the three-dimensional displacement field with high precision and rapid self-calibration.

To achieve the above object, the present invention provides the following solutions:

A high-precision displacement real-time measurement system of an RGBD monocular camera, which is special in that it includes a target recognition module, an RGBD camera measurement module and a displacement real-time calculation module;

The target recognition module is used to identify and obtain the pixel coordinates of the center point of the recognized object; the RGBD camera measurement module is used to measure the pixel coordinates of the center point of the recognized object to obtain the world coordinate value of the center point of the recognized object; the displacement real-time calculation module uses The real-time camera matrix M is obtained by the pixel coordinates of the center point of the recognized object, the world coordinate value and the known internal parameter matrix of the camera, so as to realize the high-precision real-time displacement measurement of the recognized object.

In addition, the present invention also provides a high-precision displacement real-time measurement method of an RGBD monocular camera, which specifically includes the following steps:

S1, mark the left identifier and the right identifier;

S2, using the target recognition module to recognize the pixel coordinates (u ₁ , v ₁ ) of the center point of the left recognition object;

Define the world coordinate system as O _w -X _w Y _w Z _w , where O _w Z _w is the main optical axis direction;

Define the image coordinate system as o'-uv, where u represents the X direction and v represents the Y direction;

S3, using the RGBD camera measurement module to measure the pixel coordinates (u ₁ , v ₁ ) identified by S2, to obtain the world coordinate value P _w1 of the center point of the identification object on the left, the P _w1 =(X _w1 , Y _w1 , Z _w1 );

S4, based on the pixel coordinates (u ₁ , v ₁ ) identified by S2 and the world coordinate value P _w1 measured by S3, the camera matrix M of the RGBD camera is calculated by the displacement real-time calculation module, and then the self-calibration of the camera is completed;

S5, adopt the method of S2 to identify the pixel coordinates (u ₂ , v ₂ ) of the center point of the right identifier, and use the value of the camera matrix M obtained in S4 to obtain the world coordinate value P _w2 of the center point of the right identifier, so Said P _w2 = (X _w2 , Y _w2 , Z _w2 );

S6, define the world coordinate values of the left identifier and the right identifier at time 0 as P _w1 and P _w2 respectively;

Define the world coordinate values of the left identifier and the right identifier at time t as P _w1 ' and P _w2 ' respectively, where P _w1 '=(X _w1 ', Y _w1 ', Z _w1 '), P _w2 '= (X _w2 ', Y _w2 ', Z _w2 ');

Then the real-time displacement of the left and right identifiers on the X-axis from time 0 to time t is: WYx=|X _w1 ' _- X _w1 |+|X _w2 ' _- X _w2 |, from time 0 to t The real-time displacement of the left identifier and the right identifier on the Y-axis at the moment is: WYy=|Y _w1 ' _- Y _w1 |+|Y _w2 ' _- Y _w2 | High-precision displacement real-time measurement of side identifiers.

Further, in S4, the camera matrix M is calculated in real time according to the movement of the camera position, and the calculation formula is:

Among them, K is the camera's internal parameter matrix, λ is the coefficient constant, R is the camera's external parameter rotation matrix, and t is the camera's external parameter translation matrix.

Further, in S2, the target identification module uses a corner detection algorithm to identify.

Compared with the prior art, the beneficial effects of the present invention are:

1. A high-precision displacement real-time measurement system for an RGBD monocular camera provided by the present invention includes a target recognition module, an RGBD camera measurement module and a displacement real-time calculation module. On the premise of no calibration module, it can realize the rapid automatic self-measurement of the RGBD camera. The calibration measurement solves the tedious problem of the calibration process in the displacement measurement, and saves the measurement time and labor cost.

2. The high-precision displacement real-time measurement method of an RGBD monocular camera provided by the present invention realizes the high-precision measurement of the three-dimensional displacement field by using the high-precision ranging function of the RGBD monocular camera, and provides a practical solution for the actual displacement measurement. Strong data support.

3. The high-precision displacement real-time measurement method of an RGBD monocular camera provided by the present invention can realize real-time measurement of the displacement of the center point of any marker in three-dimensional space by identifying the central pixel coordinates of the point to be measured. The realization of in-plane two-dimensional displacement measurement has better performance.

Description of drawings

FIG. 1 is a schematic diagram of the framework of a high-precision displacement real-time measurement system of an RGBD monocular camera of the present invention.

FIG. 2 is a flowchart of a high-precision displacement real-time measurement method of an RGBD monocular camera of the present invention.

3 is a camera imaging model diagram of an RGBD measurement module of a high-precision displacement real-time measurement method of an RGBD monocular camera of the present invention;

4 is an original diagram of the marking of the left identifier and the right identifier in the embodiment of the present invention;

Fig. 5 is the grayscale image of Fig. 4;

6 is an extraction diagram of a left identifier and a right identifier in an embodiment of the present invention;

Fig. 7 is the measurement result detection diagram of the left identification object and the right identification object in the embodiment of the present invention;

8 is a real-time displacement change diagram of a high-precision displacement real-time measurement method of an RGBD monocular camera in an embodiment of the present invention;

Fig. 9 is a displacement change diagram measured by a rod-type displacement sensor collected by a PXI data acquisition system.

In the attached picture:

1-High-precision displacement real-time measurement system of RGBD monocular camera, 2-target recognition module, 3-RGBD camera measurement module, 4-displacement real-time calculation module.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, not all of the embodiments. The embodiments of the present invention, and all other embodiments obtained by those of ordinary skill in the art without creative work, fall within the protection scope of the present invention.

As shown in Figure 1, a high-precision displacement real-time measurement system 1 of an RGBD monocular camera includes a target recognition module 2, an RGBD camera measurement module 3 and a displacement real-time calculation module 4; the target recognition module 2 is used to identify the center of the recognized object The pixel coordinates of the point; the RGBD camera measurement module 3 is used to obtain the world coordinate value of the center point of the recognized object by measuring the pixel coordinates of the center point of the recognized object; The camera matrix M is calculated from the world coordinate value of the center point of the side identification object and the known internal parameter matrix K of the camera, and then the center of the right identification object is calculated by measuring the pixel coordinates of the center point of the right identification object and the calculated camera matrix M. The world coordinate value of the point is obtained, and then the change value of the real-time measurement displacement of the left and right identification objects is obtained, and the high-precision real-time displacement measurement of the RGBD monocular camera is realized.

In addition, as shown in FIG. 2 , the present invention also provides a high-precision real-time displacement measurement method of an RGBD monocular camera, comprising the following steps:

S1, mark the left identifier and the right identifier, and the left identifier and the right identifier refer to any two positions in the plane.

S2, using the target recognition module 2 to identify the pixel coordinates (u ₁ , v ₁ ) of the center point of the identification object on the left, in this embodiment, the target recognition module 2 uses a corner detection algorithm to identify, and the corner detection algorithm It can quickly and accurately identify the required information and provide accurate basic data for subsequent measurements and calculations.

Define the coordinate system of the camera in the target recognition module as O _c -X _c Y _c Z _c , where O _c is the optical center, and O _c Z _c is the main optical axis direction;

Define the world coordinate system as O _w -X _w Y _w Z _w , where O _w Z _w is the main optical axis direction;

Define the image coordinate system as o'-uv, where u represents the X direction and v represents the Y direction.

S3, the RGBD camera measurement module 3 is used to measure the pixel coordinates (u ₁ , v ₁ ) of S2 to obtain the world coordinate value P _w1 of the center point of the identification object on the left, where P _w1 =(X _w1 , Y _w1 , Z _w1 ), in this step, the RGBD camera measurement module 3 uses an RGBD camera to measure.

As shown in Figure 3, it is the camera imaging model diagram of the RGBD camera measurement module.

S4, based on the pixel coordinates (u ₁ , v ₁ ) of S2 and the world coordinate value P _w1 obtained by S3, the camera matrix M of the RGBD camera is calculated by the displacement real-time calculation module 4, and then the self-calibration of the camera is completed.

In this step, the camera matrix M is calculated in real time according to the movement of the camera position, and the specific calculation formula is:

which is:

Among them, since the target recognition module 2 selects a fixed-focus camera, K is a known value, λ is a coefficient constant, R is a rotation matrix of camera extrinsic parameters, and t is a translation matrix of camera extrinsic parameters.

S5, adopt the method of S2 to identify the pixel coordinates (u ₂ , v ₂ ) of the center point of the identification object on the right, and use the value of the camera matrix M obtained in S4 to obtain the accurate center point of the identification object on the right using the calculation formula of S4 The world coordinate value P _w2 of , where P _w2 =(X _w2 , Y _w2 , Z _w2 ).

S6, define the world coordinate value of the left identifier at time 0 as P _w1 , then P _w1 =(X _w1 , Y _w1 , Z _w1 );

Define the world coordinate value of the right identifier at time 0 as P _w2 , then P _w2 =(X _w2 , Y _w2 , Z _w2 );

Define the world coordinate value of the left identifier at time t as P _w1 ', then P _w1 '=(X _w1 ', Y _w1 ', Z _w1 ');

Define the world coordinate value of the right identifier at time t as P _w2 ', then P _w2 '=(X _w2 ', Y _w2 ', Z _w2 ').

Then the real-time displacement of the left and right identifiers on the X-axis from time 0 to time t is: WYx=|X _w1 ' _- X _w1 |+|X _w2 ' _- X _w2 |, from time 0 to t The real-time displacement of the left identifier and the right identifier on the Y-axis at the moment is: WYy=|Y _w1 ' _- Y _w1 |+|Y _w2 '-Y _w2 |, and then the left identifier and the right identifier are completed. The high-precision displacement of the side marker is measured.

In this embodiment, black square identification objects are pasted on both sides of the sliding seal section of the test bed to measure the displacement of the heater in real time. Algorithm verification adopts offline verification. The computer configuration environment for algorithm verification is i5-6200U processor, 8G memory, and RGBD camera is used to shoot 1080P resolution video at 30 frames/s. Heater displacement video data for 60s.

The specific method is: fix the RGBD camera on the tripod, adjust the level adjuster to make the RGBD camera close to the horizontal state; align the tripod with the center of the square marker, adjust the distance between the RGBD camera and the recognition object, so that the two recognition The object is located in the center of the viewing frame of the RGBD camera; the marked point image video of the heater thermal test is processed by algorithm to obtain the original image shown in Figure 4, the grayscale image shown in Figure 5, and the extraction of the recognized object shown in Figure 6. Fig. and Fig. 7 is a graph showing the detection result of the recognized object.

It can be seen from Figure 4 to Figure 7 that it is feasible to stick black square identifiers on both sides of the sliding seal section of the test bench to measure the displacement of the heater.

As shown in Figure 7, during the whole time period of the heater thermal test, the recognized objects in the video move stably with the displacement of the sliding seal section, and the recognized objects are clearly extracted from the background image, which can verify the target recognition module 2. Identify the effectiveness of the algorithm. It takes about 0.17s to process the output horizontal axis displacement data into a text file. It takes about 0.17s to process the 60s video data. It can be verified that the algorithm meets the real-time requirements of the heater displacement measurement. Figure 8 is the displacement data obtained based on machine vision acquisition and processing. There are discontinuities in the data because the camera is not recognized. The figure shows that the displacement from 0 frames to 400 frames is not obvious, and the displacement (WY) fluctuates around 0mm, obtained from 400 frames to 1400 frames of images. The displacement (WY) increases parabolically, and the overall growth is relatively fast.

As shown in Figure 8, the maximum displacement measured by the real-time displacement measurement method based on the RGBD monocular camera of the present invention is 15.7324mm, and the displacement measurement is accurate to 0.0001mm, while the traditional displacement measurement method based on monocular vision can measure Compared with the traditional monocular vision measurement method, the measurement accuracy of the present invention is obviously improved. In addition, the monocular camera can only realize the two-dimensional displacement measurement in the plane, and the present invention adopts the high-precision ranging function of the RGBD monocular camera, and can realize the high-precision measurement of the three-dimensional displacement field by identifying the center pixel coordinates of the point to be measured, It provides a strong data support for the actual displacement measurement.

As shown in Figure 9, compare the displacement data measured by the rod-type displacement sensor collected by the PXI data acquisition system, and select the same time interval for comparison. s, converted to 60s time data.

Contrast Fig. 8 and Fig. 9, namely the measurement results of the measuring method of the present invention and the measuring method of the pull rod type displacement sensor are compared: the maximum displacement measured by the measuring method of the present invention is 15.7324mm, and the maximum displacement measured by the pull rod type displacement sensor is 15.7330mm, the difference between the two is only 0.0006mm, but the displacement measurement accuracy reaches 0.001mm, so it is verified that the accuracy of the measurement result of the present invention is wirelessly close to the measurement accuracy of the pull rod type displacement sensor, but the pull rod type displacement sensor can only realize the contact type. High-precision displacement measurement, and the displacement measurement system based on the RGBD monocular camera of the present invention fully meets the requirements of non-contact high-precision displacement measurement, and the final measurement result is also the non-contact measurement of the three-dimensional displacement field in the aviation and aerospace fields. Provides strong data support.

Camera calibration is the most basic and important link in systems such as size measurement, pose tracking, defect detection, etc. The high-precision displacement real-time measurement system and method for an RGBD monocular camera of the present invention is also applicable to the research of the camera calibration process. , which are all within the protection scope of the present invention.

Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, and substitutions can be made in these embodiments without departing from the principle and spirit of the invention and modifications, the scope of the present invention is defined by the appended claims and their equivalents. The purpose of the present invention is to provide a super-resolution real-time displacement measurement system and method to solve the problems raised in the above background.

Claims (4)

1. a high-precision displacement real-time measurement system of RGBD monocular camera, is characterized in that: comprise target recognition module (2), RGBD camera measurement module (3) and displacement real-time calculation module (4); The target recognition module (2) is used for identifying the pixel coordinates of the center point of the recognized object; The RGBD camera measurement module (3) is used to measure the pixel coordinates of the center point of the identifier to obtain the world coordinate value of the center point of the identifier; The displacement real-time calculation module (4) is used to obtain a real-time camera matrix M by identifying the pixel coordinates of the center point of the object, the world coordinate value and the known internal parameter matrix of the camera, thereby realizing high-precision real-time displacement measurement of the identified object. 2. a high-precision displacement real-time measurement method of RGBD monocular camera, is characterized in that, comprises the following steps: S1, mark the left identifier and the right identifier; S2, using the target recognition module (2) to recognize the pixel coordinates (u ₁ , v ₁ ) of the center point of the left recognition object; Define the world coordinate system as O _w -X _w Y _w Z _w , where O _w Z _w is the main optical axis direction; Define the image coordinate system as o'-uv, where u represents the X direction and v represents the Y direction; S3, the RGBD camera measurement module (3) is used to measure the pixel coordinates (u ₁ , v ₁ ) identified by S2 to obtain the world coordinate value P _w1 of the center point of the identification object on the left, where the P _w1 =(X _w1 , Y _w1 , Z _w1 ); S4, based on the pixel coordinates (u ₁ , v ₁ ) identified by S2 and the world coordinate value P _w1 measured by S3, the camera matrix M of the RGBD camera is calculated by the displacement real-time calculation module (4), and then the self-calibration of the camera is completed; S5, adopt the method of S2 to identify the pixel coordinates (u ₂ , v ₂ ) of the center point of the right identifier, and use the value of the camera matrix M obtained in S4 to obtain the world coordinate value P _w2 of the center point of the right identifier, so Said P _w2 = (X _w2 , Y _w2 , Z _w2 ); S6, define the world coordinate values of the left identifier and the right identifier at time 0 as P _w1 and P _w2 respectively; Define the world coordinate values of the left identifier and the right identifier at time t as P _w1 ' and P _w2 ' respectively, where P _w1 '=(X _w1 ', Y _w1 ', Z _w1 '), P _w2 '= (X _w2 ', Y _w2 ', Z _w2 '); Then the real-time displacement of the left and right identifiers on the X-axis from time 0 to time t is: WYx=|X _w1 '-X _w1 |+|X _w2 '-X _w2 |, from time 0 to t The real-time displacement of the left identifier and the right identifier on the Y-axis at the moment is: WYy=|Y _w1 '-Y _w1 |+|Y _w2 '-Y _w2 |, so far, the left identifier and right identifier are completed. High-precision displacement real-time measurement of side identifiers. 3. the high-precision displacement real-time measuring method of a kind of RGBD monocular camera according to claim 2, is characterized in that: In S4, the camera matrix M is calculated in real time according to the movement of the camera position, and the calculation formula is:

Among them, K is the camera's internal parameter matrix, λ is the coefficient constant, R is the camera's external parameter rotation matrix, and t is the camera's external parameter translation matrix. 4. The high-precision displacement real-time measurement method of an RGBD monocular camera according to claim 3, characterized in that: in S2, the target identification module (2) uses a corner detection algorithm to identify.

Images