WO2024109846A1

WO2024109846A1 - Optical target three-dimensional measurement system and method, electronic device, and storage medium

Info

Publication number: WO2024109846A1
Application number: PCT/CN2023/133443
Authority: WO
Inventors: 谷飞飞; 段俊凯; 宋展
Original assignee: 中国科学院深圳先进技术研究院
Priority date: 2022-11-25
Filing date: 2023-11-22
Publication date: 2024-05-30
Also published as: CN115854866A

Abstract

The present application provides an optical target three-dimensional measurement system and method, an electronic device, and a storage medium. The system comprises: a reference camera; a measurement camera, an optical axis center line of the measurement camera intersecting with an optical axis center line of the reference camera; a reference plate, the reference plate being fixed to the surface of the measurement camera facing the reference camera, and the reference camera tracking and locating a transformation pose of the measurement camera by capturing a change posture of the reference plate; a motion mechanism, the motion mechanism driving the measurement camera to realize rotation and pitching motion; and an optical target, the optical target being used for contacting an object to be measured, information of the optical target being captured by means of the measurement camera. By means of a new vision system structure of "reference camera + measurement camera", the solution greatly increases a measurement field of view and can realize measurement of a large-size target.

Description

Optical target three-dimensional measurement system, method, electronic device and storage medium

Technical Field

The present invention belongs to the field of visual measurement technology, and in particular relates to an optical target three-dimensional measurement system, method, electronic equipment and storage medium.

Background technique

Portable optical target 3D measurement technology is widely used in the fields of mechanical manufacturing, aerospace, industrial inspection and clinical medicine due to its advantages of being easy to carry, install, operate and meet the requirements of on-site online measurement. For example, it can detect the geometric information of the inner cavity and outer shape of large special-shaped workpieces in industry, and track and locate the position information of joints and spine in medical surgery.

The portable optical target measurement system is a contact optical measurement system, which generally includes a visual system, an optical target, a computer and related software. The surface of the optical target contains several marking points, and a contact probe is fixed at the bottom. When in use, the optical target is held and its position is adjusted so that the probe contacts a certain point to be measured on the surface of the object to be measured, and the surface marking point can be observed and captured by the visual system; then the three-dimensional coordinate information of the marking point is obtained through image processing methods and visual measurement technology, and then the position of the measuring point is inferred and calculated. Among them, in order to achieve three-dimensional positioning, the visual system is generally composed of two or more cameras. In these systems, the field of view of the visual system is relatively limited, and in order to achieve measurement, the optical target must be fully imaged in two or more cameras, which greatly limits the application of portable optical target three-dimensional measurement technology and equipment in the measurement of large-scale objects.

Summary of the invention

The purpose of the embodiments of this specification is to provide an optical target three-dimensional measurement system, method, electronic device and storage medium.

To solve the above technical problems, the embodiments of the present application are implemented in the following ways:

In a first aspect, the present application provides an optical target three-dimensional measurement system, the system comprising:

Reference camera;

Measuring camera; the center line of the optical axis of the measuring camera intersects the center line of the optical axis of the reference camera;

The reference plate is fixed on the side of the measurement camera facing the reference camera. Track and locate the changed posture of the test board to measure the changed posture of the camera;

Motion mechanism: the motion mechanism drives the measuring camera to realize rotation and pitch motion;

Optical target: The optical target is used to contact the object to be measured, and the information of the optical target is captured by the measuring camera.

In one embodiment, the motion mechanism includes a pitch mechanism and a rotation mechanism. The pitch mechanism is an angular displacement platform, which realizes the pitch motion of the measuring camera through angular displacement; and the rotation mechanism is a horizontal rotation mechanism, which realizes the rotation motion of the measuring camera.

In one embodiment, the optical target includes a target body and a probe, and a plurality of marking points are arranged on the surface of the target body.

In a second aspect, the present application provides a method for three-dimensional measurement of an optical target, the method comprising:

The optical target contacts the measuring point on the surface of the object to be measured, and the motion mechanism adjusts the position of the measuring camera to put the optical target in the best imaging position;

The measuring camera captures the marking point information on the target surface of the optical target;

According to the marking point information and the calibrated marking point information, the three-dimensional coordinates of each marking point are obtained;

According to the three-dimensional coordinates of each marking point, calculate the three-dimensional coordinates of the measuring point;

The three-dimensional information of the measured object is determined based on the three-dimensional coordinates of multiple measuring points.

In one embodiment, the method further includes: determining a motion conversion relationship between a reference camera coordinate system and a current measurement camera coordinate system, including:

According to the installation relationship between the reference plate and the measuring camera, the motion conversion relationship between the reference plate coordinate system and the measuring camera coordinate system is determined;

Before the optical target three-dimensional measurement system works, the three-dimensional coordinates of each feature point on the reference plate in the reference plate coordinate system and the pixel coordinates of the imaging corner points corresponding to each feature point in the reference coordinate system are obtained;

According to the homogeneous correspondence between the three-dimensional coordinates of each feature point and the pixel coordinates of the imaging corner point, the motion transformation relationship between the reference camera coordinate system and the reference board coordinate system is estimated;

Determine the motion conversion relationship between the reference camera coordinate system and the measurement camera coordinate system according to the motion conversion relationship between the reference plate coordinate system and the measurement camera coordinate system and the motion conversion relationship between the reference camera coordinate system and the reference plate coordinate system;

After the optical target 3D measurement system works, the position and posture of the measuring camera changes, and the image on the reference plate also changes, so the motion conversion relationship between the current reference plate coordinate system and the reference camera is determined;

According to the motion conversion relationship between the reference plate coordinate system and the measurement camera coordinate system and the current reference plate coordinate system The motion transformation relationship between the reference camera coordinate system and the current measurement camera coordinate system is determined.

In one embodiment, calculating the three-dimensional coordinates of the measuring point according to the three-dimensional coordinates of each marking point includes:

Get the distance from each marking point to the measuring point;

According to the three-dimensional coordinates of each marking point and the corresponding distance, a set of equations is established;

Solve the system of equations to obtain the three-dimensional coordinates of the measuring points.

In one embodiment, the three-dimensional coordinates of each marking point are determined by the following steps:

Acquire an optical target image captured by a measuring camera during measurement, wherein the optical target image includes various marking points;

According to the pinhole imaging model, the relationship between the spatial coordinates of each marking point and the corresponding pixel coordinates in the optical target image is determined;

The three-dimensional coordinates of each marking point are determined according to the geometric relationship between each marking point and the relationship between the spatial coordinates of each marking point and the corresponding pixel coordinates in the optical target image.

In one embodiment, after obtaining the three-dimensional coordinates of the measuring point, the method further includes:

The position and posture information of the current measuring camera is read, and according to the motion conversion relationship between the reference camera coordinate system and the current measuring camera coordinate system, the three-dimensional coordinates of the measuring point in the measuring coordinate system are converted into the coordinates in the reference coordinate system.

In a third aspect, the present application provides an electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the optical target three-dimensional measurement method according to the second aspect is implemented.

In a fourth aspect, the present application provides a readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the optical target three-dimensional measurement method as in the second aspect.

It can be seen from the technical solution provided in the above embodiments of this specification that:

1) The new visual system structure of "reference camera + measurement camera" greatly increases the measurement field of view, enabling the measurement of large-size targets;

2) Affected by factors such as lens distortion, the imaging error gradually increases from the center to the edge of the image. Through the new measurement method of this application, the measurement camera posture can be adjusted in real time during the measurement so that the target is imaged at the center of the image as much as possible, thereby improving the accuracy of 3D reconstruction;

3) Compared with dual-camera or multi-camera measurement systems, the optical target in this application only needs to be imaged in one camera, which reduces the difficulty of placing the handheld target and is more flexible in use.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the embodiments of this specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

FIG1 is a schematic diagram of the structure of an optical target three-dimensional measurement system provided by the present application;

FIG2 is a schematic diagram of the structure of the visual module provided by the present application;

FIG3 is a schematic diagram of a pattern of a reference plate provided in the present application;

FIG4 is a schematic diagram of the distribution of marking points on the target surface provided by the present application;

FIG5 is a schematic diagram of the geometric relationship of the reference plate provided by the present application in the image plane of the reference camera;

FIG6 is a schematic diagram of a reference plate image captured by a reference camera when the measurement camera provided in the present application is in different postures;

FIG7 is a schematic diagram of the process of the optical target three-dimensional measurement method provided by the present application;

FIG8 is a schematic diagram of monocular imaging of four-point collinear optical targets provided by the present application;

FIG. 9 is a schematic diagram of the structure of an electronic device provided in this application.

Detailed ways

In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on the embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

In the following description, specific details such as specific system structures and technologies are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, systems, circuits, and methods are omitted to avoid unnecessary details that hinder the present application. Please provide a description.

It will be apparent to those skilled in the art that various modifications and variations may be made to the specific embodiments of the present application description without departing from the scope or spirit of the present application. Other embodiments derived from the present application description will be apparent to those skilled in the art. The present application description and examples are merely exemplary.

The words “include,” “including,” “have,” “contain,” etc. used in this document are open-ended terms, meaning including but not limited to.

Unless otherwise specified, "parts" in this application are calculated by mass.

In the related art, in order to achieve three-dimensional positioning, the visual part of the portable optical target three-dimensional measurement system generally adopts a stereoscopic vision system composed of two or more cameras, such as a handheld optical target and its measurement method for visual coordinate measurement, which realizes a spatial measurement method based on a binocular stereoscopic vision system by designing a plane target with several LED optical marking points installed on the surface. Another example is a handheld spherical target and measurement method used in binocular stereoscopic vision measurement, which designs a handheld spherical target with regular feature points arranged on the surface, and realizes three-dimensional measurement based on a binocular stereoscopic vision system. However, since a single camera cannot restore the depth information of the marking point when measuring, there are few products and research based on monocular optical target positioning systems. There are also some single-camera solutions, such as a portable spherical target for rapid calibration and measurement of a monocular vision system, which uses a spherical target imaging model to realize monocular spatial positioning of the center of the ball. However, although this method can use a single camera to realize the center of the ball positioning, the positioning accuracy is low and the performance is unstable, so it cannot be applied in practice. In addition, all of the above systems have the problem of limited measurement field of view. Once the visual system is fixed, the measurement field of view is also fixed. This is especially true for dual-camera or multi-camera systems, which require handheld targets to be imaged in each camera. This severely limits the measurement range of the target measurement system and cannot be applied to the measurement of large-size workpieces.

Based on the above, the current portable optical target 3D measurement system mostly uses two or more cameras to collect images based on fixed postures. There are the following disadvantages:

1) Small field of view. In the existing portable optical target 3D measurement system, the measurement range is limited because two or more cameras use a fixed viewing angle. In addition, all the marking points on the optical target are required to be imaged in all cameras, which further reduces the effective measurement range. Therefore, it is not suitable for the spatial measurement of large-sized objects.

2) Low measurement accuracy. Due to the limited viewing angle, it is difficult for the optical target to be imaged in the center of the image. It is well known that the central area of the image has the smallest distortion and the highest imaging accuracy. The closer the imaging is to the edge of the image, the greater the distortion, which will affect the imaging and extraction accuracy of the target marker point, and further affect the 3D measurement accuracy of the measured point.

3) Inflexible use. In the existing portable optical target 3D measurement system, the operator holds the optical target and needs to carefully adjust its position to make it in the effective imaging area of the visual system. If the position is not appropriate, measurement cannot be achieved. For some invisible surfaces, it may even be necessary to adjust the position of the visual system multiple times or even move the position of the visual system to observe them. However, if the visual system is moved, additional calibration objects are required to calibrate its position relationship with the world coordinate system. Therefore, it is less flexible to use.

In order to achieve the measurement of large-size targets, this application proposes an optical target three-dimensional measurement method and system. In this application, the visual system adopts a combination mode of "reference camera + measurement camera". The measurement camera realizes rotation and pitch movement through a motion mechanism, and a high-precision reference plate is pasted or installed on the surface. The reference camera is fixed and the position information of the measurement camera is determined in real time by capturing the real-time status of the reference plate.

The present invention is further described in detail below with reference to the accompanying drawings and embodiments.

1 , which shows a schematic structural diagram of an optical target three-dimensional measurement system applicable to an embodiment of the present application.

As shown in FIG1 , the optical target three-dimensional measurement system may include

Reference camera 10;

Measuring camera 20; the optical axis center line of the measuring camera intersects the optical axis center line of the reference camera;

A reference plate 30 is fixed to a side of the measuring camera facing the reference camera; the reference camera tracks and locates the transformed posture of the measuring camera by capturing the changed posture of the reference plate;

A motion mechanism 40, which drives the measuring camera 20 to realize rotation and pitch motion;

The optical target 50 contacts the object to be measured, and the information of the optical target 50 is captured by the measuring camera 20 .

The motion mechanism 40 includes a pitch mechanism 410 and a rotation mechanism 420 . The pitch mechanism 410 is an angular displacement platform, which realizes the pitch motion of the measuring camera through angular displacement; the rotation mechanism 420 is a horizontal rotation mechanism, which realizes the rotation motion of the measuring camera.

It can be understood that the reference camera 10 , the measuring camera 20 , the reference plate 30 and the motion mechanism (or motion control mechanism) 40 can be collectively referred to as a vision module, as shown in FIG. 2 .

Specifically, the reference camera 10 and the measuring camera 20 are installed vertically. The intersection point is as close as possible to the imaging optical center of the measuring camera, as shown in Figure 2.

The motion mechanism 40 is composed of a pitch mechanism 410 and a rotation mechanism 420. The pitch mechanism 410 is an angular displacement platform, which can achieve ±30° pitch motion of the measuring camera through angular displacement; the rotation mechanism 420 is a horizontal rotation mechanism, which can achieve 360° rotation motion of the measuring camera. The measuring camera 20 is fixedly mounted on the horizontal mounting surface above the motion mechanism 40, and a wide range of observation angles can be achieved through the designed motion mechanism.

The angular displacement centerline of the pitch mechanism 410 coincides with the rotation centerline of the rotation mechanism 420 and intersects with the optical axis centerline of the measuring camera 20 , and the intersection point is as close as possible to the imaging optical center position of the measuring camera 20 ; as shown in FIG. 2 .

It can be understood that the positional relationship of the reference camera being at the top and the measuring camera being at the bottom in FIG. 1 and FIG. 2 can also be reversed, that is, the measuring camera being at the top and the reference camera being at the bottom, and this is not limited here.

It can also be understood that the mode in which the rotating mechanism is on top and the pitching mechanism is on the bottom in the motion mechanism in Figures 1 and 2 can also be a mode in which the rotating mechanism is on the bottom and the pitching mechanism is on top, and this is not limited here.

The reference plate 30 can be installed or pasted on the upper surface of the measuring camera 20 (i.e., the side facing the reference camera 10). The reference plate 30 uses a high-precision reference plate and is used to track the position of the measuring camera 20. As shown in FIG3 , the patterns that can be used as the reference plate 30 include a checkerboard pattern (i.e., black and white squares in both horizontal and vertical directions, as shown in FIG3 (a)), a dot pattern (i.e., dots arranged at equal intervals in both horizontal and vertical directions, as shown in FIG3 (b)), etc. In this application, the checkerboard pattern shown in FIG3 (a) is used as an example for explanation, and the checkerboard intersections are used as feature points.

The optical target (or simply referred to as target) includes a target body and a probe, and a plurality of marking points are arranged on the surface of the target body.

It can be understood that the distribution of the marking points on the target surface is shown in Figure 1. It can be four points in a collinear manner, or it can be any other geometric distribution relationship, including multiple points in a collinear manner, multiple points in a coplanar manner, multiple points in multiple planes, etc. As long as the constraint relationship between the points is known, the three-dimensional coordinates of the marking points can be calculated, as shown in Figure 4 (including Figure 4 (a), Figure 4 (b), and Figure 4 (c)).

It can also be understood that the marking points on the target surface can be passive physical marking points such as corner points, intersection points, reflective balls, etc., or optical marking points that actively emit light such as infrared LED lights, and no limitation is made here.

This embodiment greatly increases the measurement accuracy through the new visual system structure of "reference camera + measurement camera". The measurement field can realize the measurement of large-sized targets.

In addition, compared with a dual-camera or multi-camera measurement system, the optical target in the present application only needs to be imaged in one camera (ie, the measurement camera), which reduces the difficulty of placing the handheld target and is more flexible in use.

It can be understood that the motion transformation relationship between the reference camera coordinate system and the current measurement camera coordinate system is first determined, specifically including:

According to the motion conversion relationship between the reference plate coordinate system and the measuring camera coordinate system and the motion conversion relationship between the current reference plate coordinate system and the reference camera, the conversion relationship between the reference camera coordinate system and the current measuring camera coordinate system is determined.

Specifically, the checkerboard pattern in Figure 3(a) is used as an example. The reference camera and the measurement camera have been calibrated using the Zhang method in advance. Assume that the calibrated internal parameter matrices of the two are K _ref and K _mea , both of which are 3×3 matrices; the calibrated lens distortion coefficients of the reference camera and the measurement camera are k _ref and k _cam , respectively. The reference plate and the measurement camera are fixedly installed. Assume that the reference plate coordinate system is O _plane -X _plane Y _plane Z _plane (abbreviated as ), the measurement camera coordinate system is O _meacam -X _meacam Y _meacam Z _meacam (abbreviated as ), according to the installation relationship, the motion transformation relationship between the two coordinate systems can be known, which is recorded as Then we have:

Before the optical target 3D measurement system (also referred to as the system) starts working, the reference camera is calculated The pose between the reference camera coordinate system and the reference board coordinate system (i.e., the conversion relationship between the reference camera coordinate system and the reference board coordinate system), and save the result in the memory for later use. The calculation method is as follows: Assume that the characteristic corner point on the chessboard is in the chessboard coordinate system The three-dimensional coordinates under the reference board are _Pi ^plane (i＝1,2,....,m), where m is the total number of corner points on the chessboard. Because the size and arrangement of each square on the chessboard are known, _Pi ^plane (i＝1,2,....,m) is the known three-dimensional coordinate data of Z _plane ＝0. The reference camera captures the chessboard image on the reference board And extract all the corner points on the chessboard as reference feature points, so as to obtain the pixel coordinates of all imaging corner points corresponding to _Pi ^plane (i＝1,2,....,m) under the reference camera Referring to FIG5 , there is shown the geometric relationship of the reference plate imaging on the reference camera image plane.

Assume the reference camera coordinate system O _refcam -X _refcam Y _refcam Z _refcam (abbreviated as ) and the reference plate coordinate system The motion conversion relationship between them is according to The one-way correspondence of Specific steps are as follows:

1) Normalization of chessboard pixel corners: Transform to the reference camera coordinate system

Where (u ₀ ,v ₀ ) is the pixel coordinate of the center point of the reference camera image, (f ₁ ,f ₂ ) is the focal length of the lens, both of which are pre-calibrated internal parameters of the reference camera. Represents image pixels The pixel coordinates along the horizontal and vertical direction of the image.

2) Corner point coordinates to remove lens distortion: the normalized coordinates obtained Contains lens distortion, so distortion correction is required. The distortion correction method is relatively mature and will not be repeated here. The Γ function represents the correction process function. Suppose the coordinates of the corner point after correction are

Where _kd represents the distortion coefficient of the reference camera lens calibrated in advance.

3) Calculation First, calculate Homography between and _Pi ^plane (i＝1,2,....,m) The property matrix H is a 3×3 matrix, and then it can be calculated from H and the reference plate coordinate system The motion conversion relationship between Calculated as follows:
u ₁ ＝H(:,1)/H(:,1)
u ₂ =H(:,2)-dot(u ₁ ,H(:,2))·u ₁
u ₃ =u ₁ ×u ₂ (4)

Where u ₁ , u ₂ , u ₃ represent vectors along the x-axis, y-axis, and z-axis of the space respectively. They represent the rotation matrix and translation matrix from the reference plate coordinate system to the measurement camera coordinate system. dot() represents the dot product operation.

From this we can get:

4) Calculate the reference camera coordinate system and measuring camera coordinate system The motion conversion relationship between From formulas (1) and (5), we can deduce

After the system is working, the measurement camera produces a posture change. When the measurement camera posture changes, the chessboard image on the reference board changes accordingly. Figure 6 shows examples of reference board images captured by the reference camera when the measurement camera is in different postures. When the measurement camera is in the original position, the reference board image captured by the reference camera is shown in Figure 6(a). If the measurement camera only has horizontal rotation movement, the reference board image captured by the reference camera is shown in Figure 6(b), where the chessboard squares only rotate without deformation; if the measurement camera has both horizontal rotation and pitch movement, the reference board image captured by the reference camera is shown in Figure 6(c), where the chessboard squares rotate and deform.

Assume that after the measurement camera pose changes, the reference camera captures the checkerboard image on the reference board as The result after extracting the changed chessboard corner point information and sorting is recorded as Assume the current reference plate coordinate system is According to the above steps (1)-(3), the reference camera coordinate system can be obtained and reference plate coordinate system The motion conversion relationship between satisfy,

Combining formulas (1) and (7), the reference camera coordinate system can be calculated And the current new measurement camera coordinate system The motion conversion relationship between

Through this method, the motion transformation relationship between the measurement camera and the reference camera coordinate system can be obtained when the measurement camera is in any position, thereby realizing the unification of the three-dimensional reconstruction data collected at different positions of the measurement camera.

In this embodiment, before the system starts working, the reference camera captures the checkerboard image on the reference board and records the position of the feature points as the reference corner point information; when the measurement camera posture changes, the checkerboard image on the reference board changes accordingly. The reference camera captures the checkerboard image on the reference board, extracts the changed checkerboard corner point information, and calculates the posture change of the current measurement camera in combination with the reference corner point information.

7 , which shows a schematic flow chart of the optical target three-dimensional measurement method applicable to the embodiment of the present application.

As shown in FIG7 , a method for three-dimensional measurement of an optical target may include:

S710, the optical target contacts the measuring point on the surface of the measured object, and the motion mechanism adjusts the position of the measuring camera to place the optical target in an optimal imaging position.

S720, the measuring camera captures the marking point information on the target surface of the optical target;

S730, obtaining the three-dimensional coordinates of each marking point according to the marking point information and the calibrated marking point information;

S740, calculating the three-dimensional coordinates of the measuring point according to the three-dimensional coordinates of each marking point, may include:

Get the distance from each marking point to the measuring point;

According to the geometric relationship between each marking point and the spatial coordinates of each marking point and the optical target image The relationship between the corresponding pixel coordinates in is used to determine the three-dimensional coordinates of each marking point.

Specifically, taking the four-point collinear target shown in Figure 8 as an example, before measurement, a three-coordinate measuring machine or other methods are used to calibrate the positional relationship between each marking point on the optical target and the probe probe, that is, the distance between points A, B, C, and D, and their distances to the probe are calibrated in advance.

During measurement, the target is held and fine-tuned so that the probe Q of the target contacts the position of the measured point, and the measuring camera captures the target image. The four marking points are imaged on the image. Based on the image, the imaging marking points a, b, c, and d corresponding to the marking points A, B, C, and D are reconstructed.

First, normalize the marker points:

In the formula is the pixel coordinate of the center point of the camera image, (f ₁ ^mea , f ₂ ^mea ) is the focal length of the lens, and both are pre-calibrated internal parameters of the measurement camera.

Then the corner point coordinates are dedistorted by lens: the normalized coordinates _χd obtained contain lens distortion, so distortion correction is required. The distortion correction method is relatively mature and will not be described here. The Γ′ function represents the correction process function. Let the corrected corner point coordinates be _χn :
χ _n =Γ′(χ _n ,k _d ′),(χ＝a,b,c,d) (10)

Where _k′d represents the pre-calibrated measurement camera lens distortion coefficient.

In the pinhole imaging model, there is the following relationship between the spatial coordinates P _δ (δ＝A,B,C,D) of the four points A, B, C, and D in space and the pixel coordinates χ _n (χ＝a,b,c,d) of the four points a,b,c,d imaged on the image plane of the measurement camera:

In the formula and are the augmented matrix forms of χ _n and P _δ respectively. s _δ is the unknown depth factor of point δ, which is generally the Z-axis coordinate of the point: s _δ = Z _δ . Assuming that the world coordinate system is located in the measurement camera coordinate system, we have Formula (11) can be transformed into:

In addition, taking the target marking points A, B, and C in Figure 8 as an example, since the three are collinear, the following relationship exists:

Where λ _A and λ _C are known length coefficients. If B is the midpoint of AC, then λ _A = λ _C = 0.5. Combining formulas (12)-(13), we get:

Formula (14) cross-multiplies both sides simultaneously get:

The derivation can be obtained:

In addition, since the position distribution of each marking point has been calibrated in advance during the processing of the optical target, if the length between AC is known to be L _AC , then:

Combining formula (16) and (17), we can get:

After obtaining the depth factor Z _A , the three-dimensional coordinate _PA of point A can be obtained by formula (12), and the three-dimensional coordinates _PB and _PC of points B and C can be obtained by formulas (16) and (13). Other points on the optical target can also be calculated in the same way.

After obtaining the three-dimensional coordinates of all the marked points on the optical target through the above, the three-dimensional coordinates of the measuring point can be calculated. Assume that the three-dimensional coordinates of the current measuring point Q are P _Q (X _Q , Y _Q , Z _Q ), and the distances from the four points A, B, C, and D on the target to the measuring point Q are calibrated in advance. Assume that the distances are L _AQ , L _BQ , L _CQ , and L _DQ , respectively, then:

There are only three unknowns in the formula: (X _Q , Y _Q , Z _Q ), which can be solved by the least squares method.

Specifically, after obtaining the current measuring point position, read the pose information of the current measuring camera, and use the reference camera coordinate system in formula (8) And the current new measurement camera coordinate system The motion conversion relationship between The three-dimensional coordinates of the measuring points in the measurement coordinate system can be converted to the reference coordinate system to achieve data unification.

S750: Determine three-dimensional information of the measured object according to the three-dimensional coordinates of the multiple measuring points.

Specifically, according to the measured positions of the multiple measuring points, information such as the inner cavity shape, shape and position tolerance, etc. of the object to be measured, that is, the three-dimensional information of the object to be measured can be calculated according to requirements.

The optical target three-dimensional measurement method provided in the embodiment of the present application can adjust the measurement camera posture in real time during measurement so that the target is imaged at the center of the image as much as possible, thereby improving the three-dimensional reconstruction accuracy.

In the embodiment of the present application, the reference camera is fixed, and the position and posture information of the measuring camera can be determined in real time by capturing the real-time status of the reference plate, which is simple and flexible to use.

Fig. 9 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. As shown in Fig. 9, a schematic diagram of the structure of an electronic device 900 suitable for implementing the embodiment of the present application is shown.

As shown in FIG9 , the electronic device 900 includes a central processing unit (CPU) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage part 908 into a random access memory (RAM) 903. In the RAM 903, various programs and data required for the operation of the device 900 are also stored. The CPU 901, the ROM 902, and the RAM 903 are connected to each other via a bus 904. An input/output (I/O) interface 905 is also connected to the bus 904.

The following components are connected to the I/O interface 905: an input section 906 including a keyboard, a mouse, etc.; An output section 907 such as a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I/O interface 906 as needed. A removable medium 911, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 910 as needed, so that a computer program read therefrom is installed into the storage section 908 as needed.

In particular, according to an embodiment of the present disclosure, the process described above with reference to FIG. 1 can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program tangibly contained on a machine-readable medium, and the computer program includes a program code for executing the above-mentioned optical target three-dimensional measurement method. In such an embodiment, the computer program can be downloaded and installed from a network through the communication part 909, and/or installed from the removable medium 911.

The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of the code, and the aforementioned module, program segment, or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and/or flow chart, and the combination of the boxes in the block diagram and/or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

The units or modules involved in the embodiments described in the present application may be implemented by software or hardware. The units or modules described may also be arranged in a processor. The names of these units or modules do not constitute limitations on the units or modules themselves in certain circumstances.

The systems, systems, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop, a mobile phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

As another aspect, the present application further provides a storage medium, which may be a storage medium included in the aforementioned system in the above embodiment; or a storage medium that exists independently and is not assembled into a device. The storage medium stores one or more programs, and the aforementioned programs are used by one or more processors to execute the optical target three-dimensional measurement method described in the present application.

Storage media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined in this article, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

It should be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of further restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

An optical target three-dimensional measurement system, characterized in that the system comprises:

Reference camera;

A measuring camera; the optical axis center line of the measuring camera intersects with the optical axis center line of the reference camera;

A reference plate, wherein the reference plate is fixed to a side of the measuring camera facing the reference camera; the reference camera tracks and locates the transformed posture of the measuring camera by capturing the changed posture of the reference plate;

A motion mechanism, wherein the motion mechanism drives the measuring camera to realize rotation and pitch motion;

An optical target is used to contact the object to be measured, and information of the optical target is captured by the measuring camera.
The system according to claim 1 is characterized in that the motion mechanism includes a pitch mechanism and a rotation mechanism, the pitch mechanism is an angular displacement platform, and the pitch motion of the measuring camera is achieved through angular displacement; the rotation mechanism is a horizontal rotation mechanism, and the rotation motion of the measuring camera is achieved.
The system according to claim 1 is characterized in that the optical target includes a target body and a probe, and a plurality of marking points are arranged on the surface of the target body.
A three-dimensional measurement method for an optical target, characterized in that the method comprises:

The optical target contacts a measuring point on the surface of the object to be measured, and the motion mechanism adjusts the position of the measuring camera so that the optical target is in an optimal imaging position;

The measuring camera captures the marking point information on the target surface of the optical target;

According to the marking point information and the calibrated marking point information, the three-dimensional coordinates of each marking point are obtained;

Calculating the three-dimensional coordinates of the measuring point according to the three-dimensional coordinates of each marking point;

The three-dimensional information of the measured object is determined according to the three-dimensional coordinates of the multiple measuring points.
The method according to claim 4, characterized in that the method further comprises: determining a motion conversion relationship between a reference camera coordinate system and a current measurement camera coordinate system, comprising:

Determining a motion conversion relationship between the reference plate coordinate system and the measurement camera coordinate system according to an installation relationship between the reference plate and the measurement camera;

Before the optical target 3D measurement system works, obtain the coordinates of each feature point on the reference plate in the reference plate coordinate system. The three-dimensional coordinates under the reference coordinate system and the pixel coordinates of the imaging corner points corresponding to each feature point;

According to the homogeneous correspondence between the three-dimensional coordinates of each feature point and the pixel coordinates of the imaging corner point, estimating the motion transformation relationship between the reference camera coordinate system and the reference board coordinate system;

Determine the motion conversion relationship between the reference camera coordinate system and the measuring camera coordinate system according to the motion conversion relationship between the reference plate coordinate system and the measuring camera coordinate system and the motion conversion relationship between the reference camera coordinate system and the reference plate coordinate system;

After the optical target three-dimensional measurement system is working, the position and posture of the measuring camera changes, and the image on the reference plate also changes, so as to determine the motion conversion relationship between the current reference plate coordinate system and the reference camera;

The conversion relationship between the reference camera coordinate system and the current measuring camera coordinate system is determined according to the motion conversion relationship between the reference plate coordinate system and the measuring camera coordinate system and the motion conversion relationship between the current reference plate coordinate system and the reference camera.
The method according to claim 4, characterized in that the step of calculating the three-dimensional coordinates of the measuring point according to the three-dimensional coordinates of each of the marking points comprises:

Obtaining the distance from each marking point to the measuring point;

Establishing a set of equations according to the three-dimensional coordinates of each marking point and the corresponding distances;

Solve the equations to obtain the three-dimensional coordinates of the measuring points.
The method according to claim 6, characterized in that the three-dimensional coordinates of each marking point are determined by the following steps:

Acquire an optical target image captured by a measuring camera during measurement, wherein the optical target image includes various marking points;

Determine, according to the pinhole imaging model, the relationship between the spatial coordinates of each of the marking points and the corresponding pixel coordinates in the optical target image;

The three-dimensional coordinates of each marking point are determined according to the geometric relationship between the marking points and the relationship between the spatial coordinates of each marking point and the corresponding pixel coordinates in the optical target image.
The method according to claim 4, characterized in that after obtaining the three-dimensional coordinates of the measuring point, the method further comprises:

The position and posture information of the current measuring camera is read, and according to the motion conversion relationship between the reference camera coordinate system and the current measuring camera coordinate system, the three-dimensional coordinates of the measuring point in the measuring coordinate system are converted into the coordinates in the reference coordinate system.
An electronic device comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the optical target three-dimensional measurement method as described in any one of claims 4 to 8 is implemented.
A readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the optical target three-dimensional measurement method as described in any one of claims 4 to 8 is implemented.