WO2021063127A1 - 多相机环境中主动式刚体的位姿定位方法及相关设备 - Google Patents
多相机环境中主动式刚体的位姿定位方法及相关设备 Download PDFInfo
- Publication number
- WO2021063127A1 WO2021063127A1 PCT/CN2020/110248 CN2020110248W WO2021063127A1 WO 2021063127 A1 WO2021063127 A1 WO 2021063127A1 CN 2020110248 W CN2020110248 W CN 2020110248W WO 2021063127 A1 WO2021063127 A1 WO 2021063127A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- rigid body
- dimensional space
- coordinates
- space point
- pose
- Prior art date
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Definitions
- the invention relates to the technical field of computer vision, in particular to a method for positioning an active rigid body in a multi-camera environment and related equipment.
- the traditional optical motion capture method uses the ultra-high-power near-infrared light source in the motion capture camera to emit infrared light and irradiate it on the passive marking point; the marking point coated with high-reflective material reflects the irradiated infrared light, and this part of the infrared light
- the ambient light with background information will pass through the low-distortion lens and reach the infrared narrow band pass filter unit of the camera. Since the light band of the infrared narrow band pass filter unit is the same as that of the infrared light source, the ambient light with redundant background information will be filtered out, leaving only the infrared light with the marking point information to pass through and be taken by the camera. Photosensitive element recording.
- the photosensitive element then converts the light signal into an image signal and outputs it to the control circuit, and the image processing unit in the control circuit uses a Field Programmable Gate Array (FPGA) to preprocess the image signal in the form of hardware, and finally to The tracking software flows out the 2D coordinate information of the marked points.
- the tracking and positioning software adopts the principle of computer multi-eye vision, and calculates the coordinates and direction of the point cloud in the three-dimensional capture space according to the matching relationship between the two-dimensional point cloud of the image and the relative position and orientation of the camera. Based on the three-dimensional coordinates of the point cloud, the tracking and positioning software resolves the position and orientation of each rigid body in the capture space by identifying different rigid body structures.
- the motion capture camera is required to have more complex image processing devices, and the camera cost is relatively high;
- the marking points are required to be coated with highly reflective materials, which are likely to cause wear during use and affect the normal operation of the system;
- the main purpose of the present invention is to provide an active rigid body pose positioning method and related equipment in a multi-camera environment, aiming to solve the current passive motion capture methods that require high requirements for motion capture cameras, and tracking and positioning rely on rigid body structures and other technologies. problem.
- the present invention provides a method for positioning an active rigid body in a multi-camera environment.
- the method includes the following steps:
- Acquire camera parameters of multiple cameras calculate the reprojection coordinates of the second frame of the marker point according to the camera parameters, determine the reprojection error according to the rigid body coordinates of the second frame of the marker point, and use gradient descent
- the cost function is constructed by using the reprojection error, and the cost function is minimized to obtain the motion pose of the rigid body, and the rigid body is tracked and positioned according to the initial pose and the motion pose.
- the multiple cameras are matched in pairs, and each of the marked points is obtained according to the spatial position data of the two cameras and the coordinates of the multiple two-dimensional spatial points in the same frame.
- the three-dimensional space point code and the three-dimensional space point coordinates of the frame including:
- Singular value decomposition method of solving least squares method solving a three-dimensional space point, after traversing all pairwise matching cameras, a set of three-dimensional space points is obtained, a set of three-dimensional space points is the three-dimensional space points of the marked points coordinate;
- the conversion of all three-dimensional space point codes and three-dimensional space point coordinates of the same frame into rigid body coordinates in a rigid body coordinate system to obtain the rigid body codes and rigid body coordinates of each frame of each of the marked points includes:
- the determining the initial pose of the rigid body by solving the pose estimation from the coordinates of the three-dimensional space points in the first frame to the coordinates of the rigid body includes:
- P1 is the coordinate of the three-dimensional space point in the first frame
- P1' is the coordinate of the rigid body in the first frame
- R is the Euclidean transformation rotation matrix of the rigid body
- T is the translation matrix
- the initial pose of the rigid body is obtained according to the Euclidean transformation rotation matrix and the translation matrix.
- the present invention also provides an active rigid body pose positioning device in a multi-camera environment, including:
- the acquiring data module is used to acquire the two-dimensional point coordinates of two adjacent frames captured by multiple cameras, the two-dimensional point code corresponding to the two-dimensional point coordinates, and the spatial position data of multiple cameras, The coordinates of the multiple two-dimensional space points with the same two-dimensional space point codes are classified into the same type and are marked under the same marking point;
- the three-dimensional spatial data calculation module is used to match a plurality of the cameras in pairs, and obtain each of the marks according to the spatial position data of the two cameras and the coordinates of a plurality of the two-dimensional spatial points of the same frame in the same frame 3D space point code and 3D space point coordinates of each frame of point;
- the calculation rigid body data module is used to convert all 3D space point codes and 3D space point coordinates in the same frame into rigid body coordinates in a rigid body coordinate system to obtain the rigid body codes and rigid body coordinates of each frame of each of the marked points;
- the module for determining the initial pose of the rigid body is used to determine the initial pose of the rigid body by solving the pose estimation from the three-dimensional space point coordinates to the coordinates of the rigid body in the first frame;
- the rigid body positioning module is used to obtain camera parameters of multiple cameras, calculate the reprojection coordinates of the second frame of the marker point according to the camera parameters, and determine the reprojection coordinates according to the rigid body coordinates of the second frame of the marker point
- the gradient descent method is used to construct a cost function using reprojection error, and the cost function is minimized to obtain the motion pose of the rigid body, and the rigid body is tracked and positioned according to the initial pose and the motion pose.
- the three-dimensional spatial data calculation module includes:
- the three-dimensional space point coordinate group unit is used for pairwise matching of all cameras that have captured the same marked point.
- the triangulation principle in multi-view geometry is solved by singular value decomposition to solve the least square method to obtain a set of three-dimensional space point coordinates;
- the rejection unit is used to determine whether the three-dimensional space point coordinates are within a preset threshold value range, and if the three-dimensional space point coordinates exceed the threshold value range, the three-dimensional space point coordinates are eliminated to obtain a group of the three-dimensional space point coordinates after elimination;
- a three-dimensional space point coding unit is determined, which is used to assign the two-dimensional space point code of the marker point to the code corresponding to the three-dimensional space point coordinates to obtain the three-dimensional space point code of the marker point.
- the calculating rigid body data module further includes:
- the origin calculation unit is used to calculate the coordinate average value of the three-dimensional space point coordinates corresponding to the multiple mark points in the same frame, and record the coordinate average value as the origin in the rigid body coordinate system;
- Determining the rigid body coordinate unit which is used to calculate the difference between the origin and the three-dimensional space point coordinates corresponding to each of the mark points in the same frame, to obtain the rigid body coordinates of each of the mark points in each frame;
- the rigid body coding unit is determined, which is used to assign the three-dimensional space point code of the marking point to the code corresponding to the rigid body coordinate to obtain the rigid body code of the marking point.
- the module for determining the initial pose of the rigid body includes:
- the solving matrix unit is used for solving the pose estimation from the three-dimensional space point coordinates to the rigid body coordinates in the first frame, substituting the three-dimensional space point coordinates and the rigid body coordinates into the equation, and solving the Euclidean equation by iterating the nearest point To transform the rotation matrix and the translation matrix, the equation is as follows:
- P1 is the coordinate of the three-dimensional space point in the first frame
- P1' is the coordinate of the rigid body in the first frame
- R is the Euclidean transformation rotation matrix of the rigid body
- T is the translation matrix
- the initial pose of the rigid body is obtained according to the Euclidean transformation rotation matrix and the translation matrix.
- the present invention also provides a device for positioning an active rigid body in a multi-camera environment.
- the device includes a memory, a processor, and a device that is stored in the memory and can run on the processor.
- a pose positioning program for an active rigid body in a multi-camera environment where the pose positioning program for an active rigid body in the multi-camera environment is executed by the processor to realize the above-mentioned pose positioning of an active rigid body in a multi-camera environment Method steps.
- the present invention also provides a computer-readable storage medium that stores a program for positioning the pose of an active rigid body in a multi-camera environment.
- the pose positioning program is executed by the processor, the steps of the active rigid body pose positioning method in the multi-camera environment as described above are realized.
- the process of positioning the rigid body pose in the process of positioning the rigid body pose, first calculate the three-dimensional spatial point data according to the captured two-dimensional spatial point data, wherein the spatial point data includes the spatial point Encoding and coordinates, forming a rigid body based on multiple three-dimensional space point data, transforming the spatial coordinates of these three-dimensional space point data into rigid body coordinates in the rigid body coordinate system, and calculating the initial pose of the rigid body according to the three-dimensional space point coordinates and the rigid body coordinates,
- the motion pose of the rigid body is obtained by combining the camera parameters, and finally the rigid body is tracked and positioned according to the initial pose and the motion pose.
- the present invention not only simplifies the complex device structure of the traditional optical motion capture camera, reduces the cost of the camera, but also the rigid body is not easy to wear and destroy, and the sustainability of use is greatly improved.
- the most important thing is that the tracking and positioning of rigid bodies is based on spatial point data, instead of restricting rigid body structures. This not only makes the rigid body structure unified and greatly optimized in terms of aesthetics, but also the diversity of coding states doubles the number of identifiable rigid bodies. increase.
- FIG. 1 is a schematic structural diagram of the operating environment of the active rigid body pose positioning device in the multi-camera environment related to the solution of the embodiment of the present invention
- FIG. 2 is a flowchart of a method for positioning an active rigid body in a multi-camera environment in an embodiment of the present invention
- FIG. 3 is a detailed flowchart of step S2 in an embodiment of the present invention.
- FIG. 4 is a detailed flowchart of step S3 in an embodiment of the present invention.
- Fig. 5 is a structural diagram of an active rigid body pose positioning device in a multi-camera environment in an embodiment of the present invention.
- the active rigid body pose positioning device in the multi-camera environment includes: a processor 1001, such as a CPU, a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005.
- the communication bus 1002 is used to implement connection and communication between these components.
- the user interface 1003 may include a display screen (Display) and an input unit such as a keyboard (Keyboard), and the network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a WI-FI interface).
- the memory 1005 may be a high-speed RAM memory, or a non-volatile memory (non-volatile memory), such as a magnetic disk memory.
- the memory 1005 may also be a storage device independent of the aforementioned processor 1001.
- the hardware structure of the active rigid body pose positioning device in the multi-camera environment shown in FIG. 1 does not constitute a limitation on the active rigid body pose positioning device in the multi-camera environment, and may include ratios More or fewer parts are shown, or some parts are combined, or different parts are arranged.
- the memory 1005 which is a computer-readable storage medium, may include an operating system, a network communication module, a user interface module, and a pose positioning program for an active rigid body in a multi-camera environment.
- the operating system is a program that manages and controls the pose positioning equipment and software resources of the active rigid body in the multi-camera environment, and supports the operation of the pose positioning program of the active rigid body in the multi-camera environment and other software and/or programs.
- the network interface 1004 is mainly used to access the network; the user interface 1003 is mainly used to detect confirmation commands and edit commands, etc., and process
- the device 1001 can be used to call the pose positioning program of the active rigid body in the multi-camera environment stored in the memory 1005, and execute the operations of the following embodiments of the pose positioning method of the active rigid body in the multi-camera environment.
- FIG. 2 is a flowchart of a method for positioning an active rigid body in a multi-camera environment in an embodiment of the present invention. As shown in FIG. 2, a method for positioning an active rigid body in a multi-camera environment includes The following steps:
- Step S1 acquiring data: acquiring two-dimensional point coordinates in two adjacent frames captured by multiple cameras, two-dimensional point codes corresponding to the two-dimensional point coordinates, and spatial position data of multiple cameras, and encoding the two-dimensional points
- the coordinates of the same multiple two-dimensional space points are classified into the same kind, and are marked under the same marking point.
- the marker points in this step are generally set at different positions of the rigid body.
- the two-dimensional space coordinate information of the marker points is captured by multiple cameras, and the spatial point data is determined through the preset rigid body encoding technology.
- the spatial point data includes two-dimensional spatial points. Coordinates and corresponding two-dimensional space point codes.
- the spatial position data is obtained by calibrating and calculating the spatial position relationship of each camera.
- there are eight marking points on the rigid body and the marking points can be eight luminous LED lights. Therefore, a rigid body usually contains eight spatial point data.
- each frame of data for a single camera contains the spatial point data of eight marker points.
- the encoding of the same marker point in different frames is the same.
- the spatial point data with the same spatial point code in all cameras can be divided together as the same type, and these spatial point data are considered to be projections of the same marker point in space on different cameras.
- Step S2 Calculate the three-dimensional space data: match multiple cameras in pairs, and obtain the three-dimensional space point code of each marker point and frame according to the spatial position data of the two cameras and the multiple two-dimensional space point coordinates of the same frame And three-dimensional space point coordinates.
- the processing of this step is performed on each frame of data of each marker point.
- multiple cameras that capture the marker point are matched in pairs.
- singular value decomposition singular Value Decomposition
- Decomposition, SVD solve the least square method to obtain a set of three-dimensional space point data.
- the rigid body includes eight marker points
- eight three-dimensional space point codes and three-dimensional space point coordinates of the eight marker points are obtained through this step.
- step S2 further includes:
- Step S201 Solve the least squares method: perform pairwise matching on all cameras that capture the same marked point, and use the two-dimensional space point coordinates captured by the two matched cameras in the same frame using the multi-view geometry
- the principle of triangulation through the singular value decomposition (SVD) method of solving the least squares method, the solution to obtain a three-dimensional space point, after traversing all pairwise matching cameras, a set of three-dimensional space points are obtained, and a set of three-dimensional space points is the mark The three-dimensional space point coordinates of the point.
- SSD singular value decomposition
- the two cameras are camera 1 and camera 2
- the coordinates of two two-dimensional space points captured in the same frame are A(a1, a2), B(b1, b2)
- the rotation matrix of camera 1 is R1( R11, R12, R13)
- R1 is a 3*3 matrix
- the translation matrix is T1 (T11, T12, T13)
- T1 is a 3*1 matrix
- the rotation matrix of camera 2 is R2 (R21, R22, R23)
- the translation matrix is T2 (T21, T22, T23).
- R2 is a 3*3 matrix
- the translation matrix T2 is a 3*1 matrix.
- the coordinates of the two two-dimensional space points captured by all pairwise matching cameras are finally calculated in the above-mentioned manner to obtain a set of three-dimensional space points, which are the three-dimensional space point coordinates of the marker points.
- Step S202 excluding out-of-threshold coordinates: determine whether the three-dimensional space point coordinates are within a preset threshold range, and if the three-dimensional space point coordinates are beyond the threshold range, the three-dimensional space point coordinates are eliminated to obtain a set of eliminated three-dimensional space point coordinates.
- this threshold range is a coordinate parameter preset in advance. If the three-dimensional space point coordinates are found to deviate from the threshold range, the three-dimensional space point coordinates are considered to be wrong data and eliminated.
- Step S203 Calculate the average value: Calculate the average value of a set of three-dimensional space point coordinates, and optimize it by Gauss-Newton method to obtain the three-dimensional space point coordinates of the marker point.
- Step S204 assignment: assign the two-dimensional space point code of the marker point to the code corresponding to the three-dimensional space point coordinates to obtain the three-dimensional space point code of the marker point.
- any marker point whether it is a two-dimensional point code or a three-dimensional point code, it needs to be consistent. Therefore, in this step, directly assign the two-dimensional point code corresponding to the marker point to the three-dimensional point code to get Contains three-dimensional space point coordinates and three-dimensional space point coded three-dimensional space point data.
- a set of three-dimensional point data is analyzed through a specific solution algorithm based on the known two-dimensional point data, and after operations such as integration, averaging and optimization of multiple three-dimensional point data are performed, a more accurate one is finally obtained.
- Three-dimensional space point data provides accurate data for subsequent analysis of rigid body data.
- Step S3 Calculate rigid body coordinates: convert all three-dimensional space point codes and three-dimensional space point coordinates of the same frame into rigid body coordinates in a rigid body coordinate system, and obtain the rigid body code and rigid body coordinates of each marked point and each frame.
- the three-dimensional space point data corresponding to each marking point can be obtained, and the multiple three-dimensional space point data corresponding to multiple marking points can be formed into a rigid body. If the rigid body currently in use has eight luminous LED lights, the rigid body Contains eight three-dimensional space point data. Through multiple three-dimensional space point data, such as three-dimensional space point coordinates in eight three-dimensional space point data, it can be transformed into rigid body coordinates in a rigid body coordinate system.
- step S3 further includes:
- Step S301 Calculate the average value: Calculate the coordinate average value of the three-dimensional space point coordinates corresponding to the multiple mark points in the same frame, and record the coordinate average value as the origin in the rigid body coordinate system.
- the average value is calculated for each dimension of the three-dimensional space point coordinates corresponding to all the mark points in the same frame to obtain the coordinate average value, and this coordinate average value is recorded as the origin in the rigid coordinate system as all the mark points
- the reference data of the corresponding three-dimensional space point coordinates is obtained.
- step S2 obtains eight three-dimensional space point coordinate data, and calculates the average value of each dimension of the eight three-dimensional space point coordinate data to obtain the coordinate average value.
- Step S302 Calculate the difference: calculate the difference between the origin and the three-dimensional space point coordinates corresponding to each mark point in the same frame, and obtain the rigid body coordinates of each mark point in each frame.
- the average value of the coordinates is taken as the origin in the rigid body coordinate system, and the difference between the coordinates of each three-dimensional space point and the origin is calculated, and the difference obtained is the rigid body coordinate of each marked point.
- the three-dimensional space point coordinates corresponding to the eight mark points are calculated as the difference from the origin.
- the difference between the coordinates of each dimension and the dimension coordinates corresponding to the origin is calculated, and finally Get eight rigid body coordinates.
- Step S303 assignment: assign the three-dimensional space point code of the mark point to the code corresponding to the rigid body coordinate to obtain the rigid body code of the mark point.
- this step directly assigns the three-dimensional space point codes corresponding to the marked points to the rigid body codes, and the coordinate data including the rigid body coordinates and the rigid body codes in the rigid body coordinate system can be obtained.
- the three-dimensional space point data is converted into rigid body coordinate data in a rigid body coordinate system, so as to provide definite and accurate data for the subsequent estimation of the pose.
- Step S4 Determine the initial pose: Determine the initial pose of the rigid body by solving the pose estimation from the coordinates of the three-dimensional space points in the first frame to the coordinates of the rigid body.
- the three-dimensional space point data and rigid body coordinate data of each of the two adjacent frames can be obtained, and the initial pose is determined by performing pose estimation on the three-dimensional space point coordinates and rigid body coordinates of the first frame. If the first frame of two adjacent frames is the initial data captured by multiple cameras, then this initial pose is the initial pose of the rigid body. If the first frame of the two adjacent frames is not the initial data captured by multiple cameras, such as the third frame and the fourth frame, the initial pose of the third frame is relative to the fourth frame. Yu Rigid body is the movement posture in the movement process.
- step S3 further includes:
- the three-dimensional space point coordinates and the rigid body coordinates are substituted into the equation, and the Euclidean transformation rotation matrix and translation matrix are solved by iterating the nearest point.
- the equation is as follows:
- P1 is the coordinate of the three-dimensional space point in the first frame
- P1' is the coordinate of the rigid body in the first frame
- R is the Euclidean transformation rotation matrix of the rigid body
- T is the translation matrix
- the initial pose of the rigid body is obtained.
- the three-dimensional space point coordinates P1 ⁇ P11, P12..., P18 ⁇ corresponding to the eight marker points
- the eight rigid body coordinates P1' ⁇ P11', P12'..., P18' ⁇
- P11', P12'..., P18' remain unchanged
- the initial pose problem of the rigid body is solved by solving the pose estimation from the three-dimensional point coordinates of the space coordinate system to the rigid body coordinate of the rigid body coordinate system.
- ICP iterative Closest Point
- R and T the SVD decomposition method
- R and T the Euclidean transformation matrix
- R and translation of the rigid body can be obtained.
- Matrix T Calculate R and T to get the pose data of the rigid body, which is defined as the initial pose of the rigid body.
- Step S5 rigid body tracking and positioning: obtain camera parameters of multiple cameras, calculate the reprojection coordinates of the second frame of the marker point according to the camera parameters, determine the reprojection error according to the rigid body coordinates of the second frame of the marker point, and use the gradient descent method to use
- the reprojection error constructs a cost function and minimizes the cost function to obtain the motion pose of the rigid body. According to the initial pose and the motion pose, the rigid body is tracked and positioned.
- the camera model data that is, the calibrated camera parameters
- Both the rigid body space point and the camera image point have codes corresponding to each mark point. Through the coding, a set of paired camera image points and rigid body space points can be easily obtained. Because each rigid body space point matches a different camera The camera image points, so when calculating the pose, you need to add the camera model data on the basis of step S3.
- the reprojection error is used to construct the cost function and make the cost The function is minimized, so that the pose information of the rigid body including the Euclidean transformation rotation matrix R and the translation matrix T can be calculated. According to the pose information, the tracking and positioning of the active light rigid body can be realized.
- A′(a1′, a2′) (p1′′/p3′′, p2′′/p3′′);
- the Euclidean transformation rotation matrix and translation matrix of the rigid body are calculated, and the Euclidean transformation rotation matrix and translation matrix can be compared with the initial pose Euclidean transformation rotation matrix and translation matrix respectively.
- two adjacent frames can be compared in pairs, which helps to obtain more accurate rigid body pose data.
- the active optical rigid body has coding information so that the motion capture tracking positioning no longer depends on the rigid body structure, but can directly obtain the two-dimensional space coordinates and the three-dimensional space according to the coding information
- the coordinate matching relationship makes the posture calculation of the rigid body faster and more accurate.
- a device for positioning an active rigid body in a multi-camera environment is proposed. As shown in FIG. 5, the device includes:
- the data acquisition module is used to acquire the two-dimensional point coordinates of two adjacent frames captured by multiple cameras, the two-dimensional point codes corresponding to the two-dimensional point coordinates, and the spatial position data of multiple cameras to encode the two-dimensional points
- the coordinates of the same multiple two-dimensional space points are classified into the same kind and marked under the same marking point;
- the module of calculating 3D space data is used to match multiple cameras in pairs. According to the space position data of the two cameras and the coordinates of multiple 2D space points in the same frame of the same frame, the 3D space point code of each marker point per frame is obtained And three-dimensional space point coordinates;
- the calculation rigid body data module is used to convert all 3D space point codes and 3D space point coordinates in the same frame into rigid body coordinates in the rigid body coordinate system, and obtain the rigid body codes and rigid body coordinates of each mark point and each frame;
- the module for determining the initial pose of the rigid body is used to determine the initial pose of the rigid body by solving the pose estimation from the point coordinates in the three-dimensional space of the first frame to the coordinates of the rigid body;
- the rigid body positioning module is used to obtain the camera parameters of multiple cameras, calculate the reprojection coordinates of the second frame of the marker point according to the camera parameters, and determine the reprojection error according to the rigid body coordinates of the second frame of the marker point.
- the gradient descent method is used to determine the reprojection error.
- the projection error constructs a cost function and minimizes the cost function to obtain the motion pose of the rigid body, and track and locate the rigid body according to the initial pose and the motion pose.
- this embodiment does not describe the content of the embodiment of an active rigid body positioning device in a multi-camera environment. Too much repeat.
- the module for calculating three-dimensional spatial data includes:
- the rejection unit is used to judge whether the three-dimensional space point coordinates are within the preset threshold range, and if it exceeds the threshold range, the three-dimensional space point coordinates are eliminated to obtain a set of eliminated three-dimensional space point coordinates;
- the three-dimensional space point coding unit is determined, which is used to assign the two-dimensional space point code of the marker point to the code corresponding to the three-dimensional space point coordinates to obtain the three-dimensional space point code of the marker point.
- this embodiment does not describe the content of the embodiment of an active rigid body positioning device in a multi-camera environment. Too much repeat.
- the calculating rigid body data module further includes:
- the calculation origin unit is used to calculate the coordinate average value of the three-dimensional space point coordinates corresponding to multiple mark points in the same frame, and the coordinate average value is recorded as the origin in the rigid body coordinate system;
- the rigid body coding unit is determined, which is used to assign the three-dimensional space point code of the mark point to the code corresponding to the rigid body coordinate to obtain the rigid body code of the mark point.
- this embodiment does not describe the content of the embodiment of an active rigid body positioning device in a multi-camera environment. Too much repeat.
- the module for determining the initial pose of a rigid body includes:
- the solution matrix unit is used to solve the pose estimation from the three-dimensional space point coordinates to the rigid body coordinates in the first frame, substituting the three-dimensional space point coordinates and the rigid body coordinates into the equation, and solving the Euclidean transformation rotation matrix and translation matrix by iterating the nearest point, etc.
- the formula is as follows:
- P1 is the coordinate of the three-dimensional space point in the first frame
- P1' is the coordinate of the rigid body in the first frame
- R is the Euclidean transformation rotation matrix of the rigid body
- T is the translation matrix
- the initial pose of the rigid body is obtained according to the Euclidean transformation rotation matrix and the translation matrix.
- this embodiment does not describe the content of the embodiment of an active rigid body positioning device in a multi-camera environment. Too much repeat.
- a device for positioning an active rigid body in a multi-camera environment includes a memory, a processor, and an active rigid body in a multi-camera environment that is stored in the memory and can run on the processor.
- the pose positioning program of the active rigid body in the multi-camera environment is executed by the processor to implement the steps in the pose positioning method of the active rigid body in the multi-camera environment of the above embodiments.
- a computer-readable storage medium stores a pose positioning program for an active rigid body in a multi-camera environment, and the pose positioning program for an active rigid body in a multi-camera environment is processed by the processor The steps in the active rigid body pose positioning method in the multi-camera environment of the foregoing embodiments are implemented during execution.
- the storage medium may be a non-volatile storage medium.
- the program can be stored in a computer-readable storage medium, and the storage medium can include: Read only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or optical disk, etc.
Abstract
本发明涉及计算机视觉技术领域，尤其涉及一种多相机环境中主动式刚体的位姿定位方法及相关设备。该方法包括：获取相邻两帧的二维空间点坐标、二维空间点编码，计算三维空间点编码和三维空间点坐标；将所有三维空间点编码和三维空间点坐标转化为刚体坐标；通过求解位姿估计，确定刚体的初始位姿；利用梯度下降法，使用重投影误差构造代价函数，并使代价函数最小化，得到刚体的运动位姿，根据初始位姿和运动位姿对刚体跟踪定位。本发明采用上述定位方式后，不但精简了传统光学动捕相机复杂器件结构，降低了相机成本，而且刚体不易磨损和毁坏，使用可持续性大大提升。
Description
本发明涉及计算机视觉技术领域，尤其涉及一种多相机环境中主动式刚体的位姿定位方法及相关设备。
传统的光学动捕方法是通过动捕相机内的超大功率近红外光源发出红外光，照射在被动式标记点上；涂有高反光材料的标记点反射被照射到的红外光，而这部分红外光和带有背景信息的环境光会经过低畸变镜头，到达摄像机红外窄带通滤光单元。由于红外窄带通滤光单元的通光波段跟红外光源的波段一致，因此，带有冗余背景信息的环境光会被过滤掉，只剩下带有标记点信息的红外光通过，并被摄像机感光元件记录。感光元件再将光信号转化为图像信号输出到控制电路，而控制电路中的图像处理单元使用现场可编程门阵列(Field Programmable Gate Array，FPGA)，以硬件形式对图像信号进行预处理，最后向跟踪软件流出标记点的2D坐标信息。跟踪定位软件采用计算机多目视觉原理，根据图像二维点云间的匹配关系及相机的相对位置和朝向，计算点云在三维捕捉空间内的坐标及方向。以点云的三维坐标为基础，跟踪定位软件通过识别不同的刚体结构，解算出每个刚体在捕捉空间内的位置及朝向。
以上被动式动捕方法存在以下缺点：
第一，要求动捕相机具有比较复杂的图像处理器件，相机造价相对较高；
第二，要求标记点涂有高反光材料，在使用过程中易造成磨损，影响系统正常运行；
第三，跟踪定位依赖于刚体的结构，刚体的设计使得刚体数量很局限，而且刚体的识别和跟踪需要相机捕捉到刚体上全部标记点，使用环境非常严苛。
发明内容
本发明的主要目的在于提供一种多相机环境中主动式刚体的位姿定位方法及相关设备，旨在解决目前被动式动捕方法中对动捕相机要求较高、跟踪定位依赖于刚体结构等技术问题。
为实现上述目的，本发明提供一种多相机环境中主动式刚体的位姿定位方法，所述方法包括以下步骤：
获取多个相机捕捉的相邻两帧的二维空间点坐标、所述二维空间点坐标对应的二维空间点编码和多个所述相机的空间位置数据，将所述二维空间点编码相同的多个所述二维空间点坐标分为同类，且标记于同一个标记点下；
将多个所述相机两两进行匹配，根据两个所述相机的空间位置数据及同类同帧的多个所述二维空间点坐标，得到每个所述标记点每帧的三维空间点编码和三维空间点坐标；
将同帧的所有三维空间点编码和三维空间点坐标，转化为刚体坐标系下的刚体坐标，得到每个所述标记点每帧的刚体编码和刚体坐标；
通过求解第一帧所述三维空间点坐标到所述刚体坐标的位姿估计，确定刚体的初始位姿；
获取多个所述相机的相机参数，根据所述相机参数计算所述标记点第二帧的重投影坐标，根据所述标记点第二帧的所述刚体坐标，确定重投影误差，利用梯度下降法，使用重投影误差构造代价函数，并使所述代价函数最小化，得到刚体的运动位姿，根据初始位姿和运动位姿对所述刚体跟踪定位。
可选地，所述将多个所述相机两两进行匹配，根据两个所述相机的空间位置数据及同类同帧的多个所述二维空间点坐标，得到每个所述标记点每帧的三维空间点编码和三维空间点坐标，包括：
将捕捉到的同一个标记点的所有相机进行两两匹配，对匹配的两个相机在同帧中捕捉到的两个所述二维空间点坐标，利用多视几何中的三角测量原理， 通过奇异值分解求解最小二乘法方法，解算得到一个三维空间点，遍历所有两两匹配的相机后，得到一组三维空间点，一组所述三维空间点即为所述标记点的三维空间点坐标；
判断所述三维空间点坐标是否处于预设的阈值范围内，若超过所述阈值范围，则剔除所述三维空间点坐标，得到剔除后的一组所述三维空间点坐标；
计算一组所述三维空间点坐标的平均值，通过高斯牛顿法优化，得到所述标记点的三维空间点坐标；
将所述标记点的二维空间点编码赋值给所述三维空间点坐标对应的编码，得到所述标记点的三维空间点编码。
可选地，所述将同帧的所有三维空间点编码和三维空间点坐标，转化为刚体坐标系下的刚体坐标，得到每个所述标记点每帧的刚体编码和刚体坐标，包括：
计算同帧的多个所述标记点对应的所述三维空间点坐标的坐标平均值，将所述坐标平均值记为刚体坐标系下的原点；
分别计算原点与同帧的每个所述标记点对应的所述三维空间点坐标之间的差值，得到每个所述标记点每帧的刚体坐标；
将所述标记点的三维空间点编码赋值给所述刚体坐标对应的编码，得到所述标记点的刚体编码。
可选地，所述通过求解第一帧所述三维空间点坐标到所述刚体坐标的位姿估计，确定刚体的初始位姿，包括：
求解第一帧所述三维空间点坐标到所述刚体坐标的位姿估计时，将所述三维空间点坐标和所述刚体坐标代入等式中，通过迭代最近点求解欧式变换旋转矩阵和平移矩阵，所述等式如下公式：
P1＝RP1′+T
其中，P1为第一帧所述三维空间点坐标，P1′为第一帧所述刚体坐标，R为刚体的欧式变换旋转矩阵，T为平移矩阵；
根据所述欧式变换旋转矩阵和所述平移矩阵得到刚体的初始位姿。
进一步地，为实现上述目的，本发明还提供一种多相机环境中主动式刚体的位姿定位装置，包括：
获取数据模块，用于获取多个相机捕捉的相邻两帧的二维空间点坐标、所述二维空间点坐标对应的二维空间点编码和多个所述相机的空间位置数据，将所述二维空间点编码相同的多个所述二维空间点坐标分为同类，且标记于同一个标记点下；
计算三维空间数据模块，用于将多个所述相机两两进行匹配，根据两个所述相机的空间位置数据及同类同帧的多个所述二维空间点坐标，得到每个所述标记点每帧的三维空间点编码和三维空间点坐标；
计算刚体数据模块，用于将同帧的所有三维空间点编码和三维空间点坐标，转化为刚体坐标系下的刚体坐标，得到每个所述标记点每帧的刚体编码和刚体坐标；
确定刚体初始位姿模块，用于通过求解第一帧所述三维空间点坐标到所述刚体坐标的位姿估计，确定刚体的初始位姿；
刚体定位模块，用于获取多个所述相机的相机参数，根据所述相机参数计算所述标记点第二帧的重投影坐标，根据所述标记点第二帧的所述刚体坐标，确定重投影误差，利用梯度下降法，使用重投影误差构造代价函数，并使所述代价函数最小化，得到刚体的运动位姿，根据初始位姿和运动位姿对所述刚体跟踪定位。
可选地，所述计算三维空间数据模块，包括：
计算三维空间点坐标组单元，用于将捕捉到的同一个标记点的所有相机进行两两匹配，对匹配的两个相机在同帧中捕捉到的两个所述二维空间点坐标， 利用多视几何中的三角测量原理，通过奇异值分解求解最小二乘法方法，解算得到一组三维空间点坐标；
剔除单元，用于判断所述三维空间点坐标是否处于预设的阈值范围内，若超过所述阈值范围，则剔除所述三维空间点坐标，得到剔除后的一组所述三维空间点坐标；
确定三维空间点坐标单元，用于计算一组所述三维空间点坐标的平均值，通过高斯牛顿法优化，得到所述标记点的三维空间点坐标；
确定三维空间点编码单元，用于将所述标记点的二维空间点编码赋值给所述三维空间点坐标对应的编码，得到所述标记点的三维空间点编码。
可选地，所述计算刚体数据模块，还包括：
计算原点单元，用于计算同帧的多个所述标记点对应的所述三维空间点坐标的坐标平均值，将所述坐标平均值记为刚体坐标系下的原点；
确定刚体坐标单元，用于分别计算原点与同帧的每个所述标记点对应的所述三维空间点坐标之间的差值，得到每个所述标记点每帧的刚体坐标；
确定刚体编码单元，用于将所述标记点的三维空间点编码赋值给所述刚体坐标对应的编码，得到所述标记点的刚体编码。
可选地，所述确定刚体初始位姿模块，包括：
求解矩阵单元，用于求解第一帧所述三维空间点坐标到所述刚体坐标的位姿估计时，将所述三维空间点坐标和所述刚体坐标代入等式中，通过迭代最近点求解欧式变换旋转矩阵和平移矩阵，所述等式如下公式：
P1＝RP1′+T
其中，P1为第一帧所述三维空间点坐标，P1′为第一帧所述刚体坐标，R为刚体的欧式变换旋转矩阵，T为平移矩阵；
根据所述欧式变换旋转矩阵和所述平移矩阵得到刚体的初始位姿。
为实现上述目的，本发明还提供一种多相机环境中主动式刚体的位姿定位设备，所述设备包括：存储器、处理器以及存储在所述存储器上并可在所述处理器上运行的多相机环境中主动式刚体的位姿定位程序，所述多相机环境中主动式刚体的位姿定位程序被所述处理器执行时实现如上所述的多相机环境中主动式刚体的位姿定位方法的步骤。
为实现上述目的，本发明还提供一种计算机可读存储介质，所述计算机可读存储介质上存储有多相机环境中主动式刚体的位姿定位程序，所述多相机环境中主动式刚体的位姿定位程序被处理器执行时实现如上所述的多相机环境中主动式刚体的位姿定位方法的步骤。
本发明提供的多相机环境中主动式刚体的位姿定位方法，在对刚体位姿进行定位过程中，首先根据捕获的二维空间点数据，计算三维空间点数据，其中空间点数据包括空间点编码和坐标，根据多个三维空间点数据组成一个刚体，并将这些三维空间点数据的空间坐标转化为刚体坐标系下的刚体坐标，根据三维空间点坐标和刚体坐标计算刚体的初始位姿，结合相机参数求出刚体的运动位姿，根据初始位姿和运动位姿最后对刚体进行跟踪定位。本发明采用上述定位方式后，不但精简了传统光学动捕相机复杂器件结构，降低了相机成本，而且刚体不易磨损和毁坏，使用可持续性大大提升。最为重要的是，对刚体的跟踪定位依据空间点数据，而不再对刚体结构进行约束，这不仅使得刚体结构统一化，美观方面大大优化，而且编码状态的多样性促使可识别刚体数量成倍增加。
通过阅读下文优选实施方式的详细描述，各种其他的优点和益处对于本领域普通技术人员将变得清楚明了。附图仅用于示出优选实施方式的目的，而并不认为是对本发明的限制。
图1为本发明实施例方案涉及的多相机环境中主动式刚体的位姿定位设备 的运行环境的结构示意图；
图2为本发明一个实施例中多相机环境中主动式刚体的位姿定位方法的流程图；
图3为本发明一个实施例中步骤S2的细化流程图；
图4为本发明一个实施例中步骤S3的细化流程图；
图5为本发明一个实施例中多相机环境中主动式刚体的位姿定位装置的结构图。
为了使本发明的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本发明进行进一步详细说明。应当理解，此处所描述的具体实施例仅仅用以解释本发明，并不用于限定本发明。
本技术领域技术人员可以理解，除非特意声明，这里使用的单数形式“一”、“一个”、“所述”和“该”也可包括复数形式。应该进一步理解的是，本发明的说明书中使用的措辞“包括”是指存在所述特征、整数、步骤、操作、元件和/或组件，但是并不排除存在或添加一个或多个其他特征、整数、步骤、操作、元件、组件和/或它们的组。
参照图1，为本发明实施例方案涉及的多相机环境中主动式刚体的位姿定位设备运行环境的结构示意图。
如图1所示，该多相机环境中主动式刚体的位姿定位设备包括：处理器1001，例如CPU，通信总线1002、用户接口1003，网络接口1004，存储器1005。其中，通信总线1002用于实现这些组件之间的连接通信。用户接口1003可以包括显示屏(Display)、输入单元比如键盘(Keyboard)，网络接口1004可选的可以包括标准的有线接口、无线接口(如WI-FI接口)。存储器1005可以是高速 RAM存储器，也可以是稳定的存储器(non-volatile memory)，例如磁盘存储器。存储器1005可选的还可以是独立于前述处理器1001的存储装置。
本领域技术人员可以理解，图1中示出的多相机环境中主动式刚体的位姿定位设备的硬件结构并不构成对多相机环境中主动式刚体的位姿定位设备的限定，可以包括比图示更多或更少的部件，或者组合某些部件，或者不同的部件布置。
如图1所示，作为一种计算机可读存储介质的存储器1005中可以包括操作系统、网络通信模块、用户接口模块以及多相机环境中主动式刚体的位姿定位程序。其中，操作系统是管理和控制多相机环境中主动式刚体的位姿定位设备和软件资源的程序，支持多相机环境中主动式刚体的位姿定位程序以及其它软件和/或程序的运行。
在图1所示的多相机环境中主动式刚体的位姿定位设备的硬件结构中，网络接口1004主要用于接入网络；用户接口1003主要用于侦测确认指令和编辑指令等，而处理器1001可以用于调用存储器1005中存储的多相机环境中主动式刚体的位姿定位程序，并执行以下多相机环境中主动式刚体的位姿定位方法的各实施例的操作。
参照图2，为本发明一个实施例中的多相机环境中主动式刚体的位姿定位方法的流程图，如图2所示，一种多相机环境中主动式刚体的位姿定位方法，包括以下步骤：
步骤S1，获取数据：获取多个相机捕捉的相邻两帧的二维空间点坐标、二维空间点坐标对应的二维空间点编码和多个相机的空间位置数据，将二维空间点编码相同的多个二维空间点坐标分为同类，且标记于同一个标记点下。
本步骤的标记点一般都设置在刚体的不同位置，通过多个相机捕捉标记点的二维空间坐标信息，通过预设的刚体编码技术，确定出空间点数据，空间点数据包括二维空间点坐标及对应的二维空间点编码。空间位置数据是由通过标定计算得到各相机的空间位置关系得到的。通常，在刚体上设有八个标记点， 标记点可以是八个发光的LED灯。因此刚体通常包含八个空间点数据，在多个相机在捕捉到的信息中，单个相机每帧数据中都包含八个标记点的空间点数据，同一个标记点在不同帧时的编码是相同的，不同的标记点在同帧时的编码是不同的。基于此可以把所有相机中带有相同空间点编码的空间点数据划分在一起作为同类，并认为这些空间点数据是空间中同一个标记点在不同相机上的投影。
步骤S2，计算三维空间数据：将多个相机两两进行匹配，根据两个相机的空间位置数据及同类同帧的多个二维空间点坐标，得到每个标记点每帧的三维空间点编码和三维空间点坐标。
对每个标记点的每帧数据分别进行本步骤的处理，处理时将捕捉到此标记点的多个相机两两进行匹配，利用多视几何中的三角测量原理，通过奇异值分解(Singular Value Decomposition，SVD)求解最小二乘法解算得到一组三维空间点数据。
例如，刚体包括八个标记点时，通过本步骤得到八个标记点的八个三维空间点编码和三维空间点坐标。
在一个实施例中，如图3所示，步骤S2进一步包括：
步骤S201，求解最小二乘法：将捕捉到的同一个标记点的所有相机进行两两匹配，对匹配的两个相机在同帧中捕捉到的两个二维空间点坐标，利用多视几何中的三角测量原理，通过奇异值分解(SVD)求解最小二乘法方法，解算得到一个三维空间点，遍历所有两两匹配的相机后，得到一组三维空间点，一组三维空间点即为标记点的三维空间点坐标。
设两个相机分别是相机1和相机2，在同帧中捕捉到的两个二维空间点坐标分别为A(a1，a2)，B(b1，b2)，相机1的旋转矩阵为R1(R11，R12，R13)，R1是3*3的矩阵，平移矩阵为T1(T11，T12，T13)，T1是3*1的矩阵，相机2的旋转矩阵为R2(R21，R22，R23)，平移矩阵为T2(T21，T22，T23)，同样地，R2是3*3的矩阵，平移矩阵T2是3*1的矩阵，通过下述公式得到同帧中的一个三维空间点坐标C(c1，c2，c3)：
1)根据两个相机的内参和畸变参数，将像素坐标A(a1，a2)，B(b1，b2)转化为相机坐标A′(a1′，a2′)，B′(b1′，b2′)；
2)构造最小二乘法矩阵X和Y，其中X是4*3的矩阵，Y是4*1的矩阵；X矩阵第一行为a1′*R13-R11，X矩阵第二行为a2′*R13-R12，X矩阵第三行为b1′*R23-R21，X矩阵第四行为b2′*R23-R22；Y矩阵第一行为T11-a1′*T13，Y矩阵第二行为T12-a2′*T13，Y矩阵第三行为T21-b1′*T23，Y矩阵第四行为T22-b2′*T23。
3)根据等式X*C＝Y和已经构造好的矩阵X、Y，利用SVD分解即可以求得一个三维空间点坐标C。
本步骤最终将所有两两匹配的相机捕捉到的两个二维空间点坐标均通过上述方式进行解算，得到一组三维空间点，该组三维空间点即为标记点的三维空间点坐标。
步骤S202，剔除阈值外坐标：判断三维空间点坐标是否处于预设的阈值范围内，若超过阈值范围，则剔除三维空间点坐标，得到剔除后的一组三维空间点坐标。
在得到多个三维空间点坐标后，需要检查这些三维空间点坐标是否处于预设的阈值范围内，即较小的阈值距离，此阈值范围是提前预设的坐标参数。若发现三维空间点坐标偏离阈值范围，则认为此三维空间点坐标是错误数据，进行剔除。
步骤S203，计算平均值：计算一组三维空间点坐标的平均值，通过高斯牛顿法优化，得到标记点的三维空间点坐标。
将剔除错误数据后的所有三维空间点坐标计算其平均值，计算时将三维空间点坐标的每个维度分别计算平均值，得到三维空间点坐标C′(c1′，c2′，c3′)，通过如下过程，即高斯牛顿法(Gauss-Newton)对得到的三维空间点坐标进行优化，最终得到某一标记点的三维空间点坐标C(c1，c2，c3)：
1)根据每台相机的R和T，为C′计算下列值并求总和g0、H0；
计算三维空间点坐标C′在每台相机的投影坐标，匹配实际图像坐标最近点并计算与最近点的图像坐标的残差；
根据每台相机的R和T计算C′在相机坐标系内的3D坐标q，定义：
给定相机I坐标系里面的1个3D点p(x，y，z)及其在相机上的成像坐标(u，v)，则
相应的Jacobian矩阵
以世界坐标系中的3D点位变量，则有
根据Gauss-Newton算法，计算梯度
2)计算
3)最终得到优化后的三维空间点坐标C(c1，c2，c3)。
步骤S204，赋值：将标记点的二维空间点编码赋值给三维空间点坐标对应的编码，得到标记点的三维空间点编码。
由于任一标记点的编码，无论是二维空间点编码，还是三维空间点编码，都需要一致，因此本步骤直接将标记点对应的二维空间点编码赋值给三维空间点编码，即可得到包含三维空间点坐标和三维空间点编码的三维空间点数据。
本实施例根据已知的二维空间点数据通过具体的求解算法，解析出一组三维空间点数据，并对多个三维空间点数据进行整合、平均及优化等操作后，最终得到较为准确的三维空间点数据，为后续解析刚体数据提供精确数据。
步骤S3，计算刚体坐标：将同帧的所有三维空间点编码和三维空间点坐标，转化为刚体坐标系下的刚体坐标，得到每个标记点每帧的刚体编码和刚体坐标。
通过步骤S2可以得到每个标记点对应的三维空间点数据，将多个标记点对应得到的多个三维空间点数据组成一个刚体，若当前使用的刚体具有八个发光的LED灯，则此刚体包含八个三维空间点数据。通过多个三维空间点数据，如八个三维空间点数据中的三维空间点坐标，可以转化为刚体坐标系下的刚体坐标。
在一个实施例中，如图4所示，步骤S3进一步包括：
步骤S301，计算平均值：计算同帧的多个标记点对应的三维空间点坐标的坐标平均值，将坐标平均值记为刚体坐标系下的原点。
在确定刚体坐标时，首先确定刚体坐标系下的原点。本步骤通过对同一帧中的所有标记点对应的三维空间点坐标的每一维度分别计算平均值，得到坐标平均值，并将此坐标平均值记为刚体坐标系下的原点，作为所有标记点对应的三维空间点坐标的参考数据。
例如，刚体包含八个标记点时，则步骤S2得到八个三维空间点坐标数据，将这八个三维空间点坐标数据的每一维度计算平均值，得到坐标平均值。
步骤S302，计算差值：分别计算原点与同帧的每个标记点对应的三维空间点坐标之间的差值，得到每个标记点每帧的刚体坐标。
以坐标平均值作为刚体坐标系下的原点，将每个三维空间点坐标分别与原点进行差值计算，得到的差值即为每个标记点的刚体坐标。
例如，刚体包含八个标记点时，八个标记点对应的三维空间点坐标分别与原点进行差值计算，计算时，对每一维度的坐标分别与原点对应的维度坐标进行差值计算，最终得到八个刚体坐标。
步骤S303，赋值：将标记点的三维空间点编码赋值给刚体坐标对应的编码，得到标记点的刚体编码。
与步骤S204相似的，本步骤直接将标记点对应的三维空间点编码赋值给刚体编码，即可得到在刚体坐标系下包含刚体坐标和刚体编码的坐标数据。
本实施例根据三维空间点数据转化为刚体坐标系下的刚体坐标数据，为后续估计位姿提供确定且精确的数据。
步骤S4，确定初始位姿：通过求解第一帧三维空间点坐标到刚体坐标的位姿估计，确定刚体的初始位姿。
通过步骤S1到步骤S3可以得到相邻两帧中每一帧的三维空间点数据和刚体坐标数据，通过对第一帧的三维空间点坐标和刚体坐标进行位姿估计，确定出初始位姿，若相邻两帧中的第一帧是多个相机捕获的初始数据，则此初始位姿即为刚体的初始位姿。若相邻两帧中的第一帧并不是多个相机捕获的初始数据，例如是第三帧和第四帧，则第三帧的初始位姿是相对于第四帧是初始位姿，相对于刚体是运动过程中的运动位姿。
在一个实施例中，步骤S3进一步包括：
求解第一帧三维空间点坐标到刚体坐标的位姿估计时，将三维空间点坐标和刚体坐标代入等式中，通过迭代最近点求解欧式变换旋转矩阵和平移矩阵，等式如下公式：
P1＝RP1′+T
其中，P1为第一帧三维空间点坐标，P1′为第一帧刚体坐标，R为刚体的欧式变换旋转矩阵，T为平移矩阵；
根据欧式变换旋转矩阵和平移矩阵得到刚体的初始位姿。
例如，刚体包含八个标记点时，八个标记点对应的三维空间点坐标P1＝{P11，P12…，P18}，八个刚体坐标P1′＝{P11′，P12′…，P18′}，其中P11′，P12′…，P18′保持不变，通过求解空间坐标系三维空间点坐标到刚体坐标系刚体坐标的位姿估计来解决刚体初始位姿问题。
本实施例在将数据代入等式中时，可以用迭代最近点(Iterative Closest Point，ICP)求解R和T，采用SVD分解的方法进行ICP求解，可求得刚体的欧式变换旋转矩阵R和平移矩阵T。求出R和T便得到刚体的位姿数据，该位姿定义为刚体的初始位姿。
步骤S5，刚体跟踪定位：获取多个相机的相机参数，根据相机参数计算标记点第二帧的重投影坐标，根据标记点第二帧的刚体坐标，确定重投影误差，利用梯度下降法，使用重投影误差构造代价函数，并使代价函数最小化，得到刚体的运动位姿，根据初始位姿和运动位姿对刚体跟踪定位。
在计算初始位姿时，仅考虑两组空间点之间的变化，和相机参数无关。但是在刚体运动跟踪时，不考虑相机参数误差会比较大，达不到动作捕捉的精度要求。因此为了提高精度，本步骤将相机模型数据，即标定后的相机参数也加入了计算。无论是刚体空间点还是相机图像点都带有各个标记点对应的编码，通过编码可以很容易的得到一组配对好的相机图像点和刚体空间点，由于每个刚体空间点匹配不同相机下的相机图像点，因此在计算位姿时需要在步骤S3的基础上加入相机模型数据，通过加入标定后的相机参数，利用Gauss-Newton梯度下降法，使用重投影误差来构造代价函数并使该代价函数最小化，由此就可以计算得到刚体的包括欧式变换旋转矩阵R和平移矩阵T的位姿信息，根据该位姿信息便可以实现主动光刚体的跟踪定位。
假设第二帧的二维图像点坐标为A(a1，a2)，三维空间点坐标为P2＝{P21，P22…，P28}，经过下述公式得到第二帧的重投影坐标：
假设三维空间点为C(c1，c2，c3)，相机a的旋转矩阵为Rcam，平移矩阵为Tcam，那么通过计算C′＝C*Rcam+Tcam，C′(c1′，c2′，c3′)是一个三维坐标，将C′归一化即可得到三维空间点C在相机a的重投影坐标A′(a1′，a2′)＝(c1′/c3′，c2′/c3′)。
将第二帧的相机图像坐标A(a1，a2)与重投影坐标A′(a1′，a2′)计算差值，得到重投影误差Error：
Error＝A-A′＝(a1-a1′，a2-a2′)
将上述误差函数扩展到相邻两帧之间，则最小化代价函数Error的计算方法为：
P′＝(P*R+T)；
P″＝P′*Rcam+Tcam；
则A′(a1′，a2′)＝(p1″/p3″，p2″/p3″)；
根据Error＝A-A′＝(a1-a1′，a2-a2′)将上述参数代入其中，通过非线性优化算法可以得到误差最小化时相应的位姿变换(R，T)。
本步骤使代价函数最小化后，计算得到刚体的欧式变换旋转矩阵和平移矩阵后，还可将欧式变换旋转矩阵和平移矩阵分别与初始位姿的欧式变换旋转矩阵和平移矩阵分别进行比较，以此类推，相邻两帧可两两对比，从而有利于求出更精确的刚体位姿数据。
本实施例多相机环境中主动式刚体的位姿定位方法，主动光刚体带有编码信息使得动捕跟踪定位不再依赖于刚体结构，而是可以直接根据编码信息得到二维空间坐标与三维空间坐标的匹配关系，刚体的姿态运算更加快速的同时，也更加精确，上述主动式光学动捕相对于传统的光学动捕具有非常明显的优势。
在一个实施例中，提出了一种多相机环境中主动式刚体的位姿定位装置，如图5所示，该装置包括：
获取数据模块，用于获取多个相机捕捉的相邻两帧的二维空间点坐标、二维空间点坐标对应的二维空间点编码和多个相机的空间位置数据，将二维空间点编码相同的多个二维空间点坐标分为同类，且标记于同一个标记点下；
计算三维空间数据模块，用于将多个相机两两进行匹配，根据两个相机的空间位置数据及同类同帧的多个二维空间点坐标，得到每个标记点每帧的三维空间点编码和三维空间点坐标；
计算刚体数据模块，用于将同帧的所有三维空间点编码和三维空间点坐标， 转化为刚体坐标系下的刚体坐标，得到每个标记点每帧的刚体编码和刚体坐标；
确定刚体初始位姿模块，用于通过求解第一帧三维空间点坐标到刚体坐标的位姿估计，确定刚体的初始位姿；
刚体定位模块，用于获取多个相机的相机参数，根据相机参数计算标记点第二帧的重投影坐标，根据标记点第二帧的刚体坐标，确定重投影误差，利用梯度下降法，使用重投影误差构造代价函数，并使代价函数最小化，得到刚体的运动位姿，根据初始位姿和运动位姿对刚体跟踪定位。
基于与上述本发明实施例的多相机环境中主动式刚体的位姿定位方法相同的实施例说明内容，因此本实施例对多相机环境中主动式刚体的位姿定位装置的实施例内容不做过多赘述。
在一个实施例中，计算三维空间数据模块，包括：
计算三维空间点坐标组单元，用于将捕捉到的同一个标记点的所有相机进行两两匹配，对匹配的两个相机在同帧中捕捉到的两个二维空间点坐标，利用多视几何中的三角测量原理，通过奇异值分解求解最小二乘法方法，解算得到一组三维空间点坐标；
剔除单元，用于判断三维空间点坐标是否处于预设的阈值范围内，若超过阈值范围，则剔除三维空间点坐标，得到剔除后的一组三维空间点坐标；
确定三维空间点坐标单元，用于计算一组三维空间点坐标的平均值，通过高斯牛顿法优化，得到标记点的三维空间点坐标；
确定三维空间点编码单元，用于将标记点的二维空间点编码赋值给三维空间点坐标对应的编码，得到标记点的三维空间点编码。
基于与上述本发明实施例的多相机环境中主动式刚体的位姿定位方法相同的实施例说明内容，因此本实施例对多相机环境中主动式刚体的位姿定位装置的实施例内容不做过多赘述。
在一个实施例中，计算刚体数据模块，还包括：
计算原点单元，用于计算同帧的多个标记点对应的三维空间点坐标的坐标平均值，将坐标平均值记为刚体坐标系下的原点；
确定刚体坐标单元，用于分别计算原点与同帧的每个标记点对应的三维空间点坐标之间的差值，得到每个标记点每帧的刚体坐标；
确定刚体编码单元，用于将标记点的三维空间点编码赋值给刚体坐标对应的编码，得到标记点的刚体编码。
基于与上述本发明实施例的多相机环境中主动式刚体的位姿定位方法相同的实施例说明内容，因此本实施例对多相机环境中主动式刚体的位姿定位装置的实施例内容不做过多赘述。
在一个实施例中，确定刚体初始位姿模块，包括：
求解矩阵单元，用于求解第一帧三维空间点坐标到刚体坐标的位姿估计时，将三维空间点坐标和刚体坐标代入等式中，通过迭代最近点求解欧式变换旋转矩阵和平移矩阵，等式如下公式：
P1＝RP1′+T
其中，P1为第一帧三维空间点坐标，P1′为第一帧刚体坐标，R为刚体的欧式变换旋转矩阵，T为平移矩阵；根据欧式变换旋转矩阵和平移矩阵得到刚体的初始位姿。
基于与上述本发明实施例的多相机环境中主动式刚体的位姿定位方法相同的实施例说明内容，因此本实施例对多相机环境中主动式刚体的位姿定位装置的实施例内容不做过多赘述。
在一个实施例中，提出了一种多相机环境中主动式刚体的位姿定位设备，设备包括：存储器、处理器以及存储在存储器上并可在处理器上运行的多相机环境中主动式刚体的位姿定位程序，多相机环境中主动式刚体的位姿定位程序被处理器执行时实现上述各实施例的多相机环境中主动式刚体的位姿定位方法中的步骤。
在一个实施例中，一种计算机可读存储介质，计算机可读存储介质上存储有多相机环境中主动式刚体的位姿定位程序，多相机环境中主动式刚体的位姿定位程序被处理器执行时实现上述各实施例的多相机环境中主动式刚体的位姿定位方法中的步骤。其中，存储介质可以为非易失性存储介质。
本领域普通技术人员可以理解上述实施例的各种方法中的全部或部分步骤是可以通过程序来指令相关的硬件来完成，该程序可以存储于一计算机可读存储介质中，存储介质可以包括：只读存储器(ROM，Read Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁盘或光盘等。
以上所述实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。
以上所述实施例仅表达了本发明一些示例性实施例，其描述较为具体和详细，但并不能因此而理解为对本发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本发明构思的前提下，还可以做出若干变形和改进，这些都属于本发明的保护范围。因此，本发明专利的保护范围应以所附权利要求为准。
Claims (10)
- 一种多相机环境中主动式刚体的位姿定位方法，其特征在于，所述方法包括以下步骤：获取多个相机捕捉的相邻两帧的二维空间点坐标、所述二维空间点坐标对应的二维空间点编码和多个所述相机的空间位置数据，将所述二维空间点编码相同的多个所述二维空间点坐标分为同类，且标记于同一个标记点下；将多个所述相机两两进行匹配，根据两个所述相机的空间位置数据及同类同帧的多个所述二维空间点坐标，得到每个所述标记点每帧的三维空间点编码和三维空间点坐标；将同帧的所有三维空间点编码和三维空间点坐标，转化为刚体坐标系下的刚体坐标，得到每个所述标记点每帧的刚体编码和刚体坐标；通过求解第一帧所述三维空间点坐标到所述刚体坐标的位姿估计，确定刚体的初始位姿；获取多个所述相机的相机参数，根据所述相机参数计算所述标记点第二帧的重投影坐标，根据所述标记点第二帧的所述刚体坐标，确定重投影误差，利用梯度下降法，使用重投影误差构造代价函数，并使所述代价函数最小化，得到刚体的运动位姿，根据初始位姿和运动位姿对所述刚体跟踪定位。
- 根据权利要求1所述的多相机环境中主动式刚体的位姿定位方法，其特征在于，所述将多个所述相机两两进行匹配，根据两个所述相机的空间位置数据及同类同帧的多个所述二维空间点坐标，得到每个所述标记点每帧的三维空间点编码和三维空间点坐标，包括：将捕捉到的同一个标记点的所有相机进行两两匹配，对匹配的两个相机在同帧中捕捉到的两个所述二维空间点坐标，通过奇异值分解求解最小二乘法方法，解算得到一个三维空间点，遍历所有两两匹配的相机后，得到一组三维空间点，一组所述三维空间点即为所述标记点的三维空间点坐标；判断所述三维空间点坐标是否处于预设的阈值范围内，若超过所述阈值范 围，则剔除所述三维空间点坐标，得到剔除后的一组所述三维空间点坐标；计算一组所述三维空间点坐标的平均值，通过高斯牛顿法优化，得到所述标记点的三维空间点坐标；将所述标记点的二维空间点编码赋值给所述三维空间点坐标对应的编码，得到所述标记点的三维空间点编码。
- 根据权利要求1所述的多相机环境中主动式刚体的位姿定位方法，其特征在于，所述将同帧的所有三维空间点编码和三维空间点坐标，转化为刚体坐标系下的刚体坐标，得到每个所述标记点每帧的刚体编码和刚体坐标，包括：计算同帧的多个所述标记点对应的所述三维空间点坐标的坐标平均值，将所述坐标平均值记为刚体坐标系下的原点；分别计算原点与同帧的每个所述标记点对应的所述三维空间点坐标之间的差值，得到每个所述标记点每帧的刚体坐标；将所述标记点的三维空间点编码赋值给所述刚体坐标对应的编码，得到所述标记点的刚体编码。
- 根据权利要求1所述的多相机环境中主动式刚体的位姿定位方法，其特征在于，所述通过求解第一帧所述三维空间点坐标到所述刚体坐标的位姿估计，确定刚体的初始位姿，包括：求解第一帧所述三维空间点坐标到所述刚体坐标的位姿估计时，将所述三维空间点坐标和所述刚体坐标代入等式中，通过迭代最近点求解欧式变换旋转矩阵和平移矩阵，所述等式如下公式：P1＝RP1′+T其中，P1为第一帧所述三维空间点坐标，P1′为第一帧所述刚体坐标，R为刚体的欧式变换旋转矩阵，T为平移矩阵；根据所述欧式变换旋转矩阵和所述平移矩阵得到刚体的初始位姿。
- 一种多相机环境中主动式刚体的位姿定位装置，其特征在于，所述装置包括：获取数据模块，用于获取多个相机捕捉的相邻两帧的二维空间点坐标、所述二维空间点坐标对应的二维空间点编码和多个所述相机的空间位置数据，将所述二维空间点编码相同的多个所述二维空间点坐标分为同类，且标记于同一个标记点下；计算三维空间数据模块，用于将多个所述相机两两进行匹配，根据两个所述相机的空间位置数据及同类同帧的多个所述二维空间点坐标，得到每个所述标记点每帧的三维空间点编码和三维空间点坐标；计算刚体数据模块，用于将同帧的所有三维空间点编码和三维空间点坐标，转化为刚体坐标系下的刚体坐标，得到每个所述标记点每帧的刚体编码和刚体坐标；确定刚体初始位姿模块，用于通过求解第一帧所述三维空间点坐标到所述刚体坐标的位姿估计，确定刚体的初始位姿；刚体定位模块，用于获取多个所述相机的相机参数，根据所述相机参数计算所述标记点第二帧的重投影坐标，根据所述标记点第二帧的所述刚体坐标，确定重投影误差，利用梯度下降法，使用重投影误差构造代价函数，并使所述代价函数最小化，得到刚体的运动位姿，根据初始位姿和运动位姿对所述刚体跟踪定位。
- 根据权利要求5所述的多相机环境中主动式刚体的位姿定位装置，其特征在于，所述计算三维空间数据模块，包括：计算三维空间点坐标组单元，用于将捕捉到的同一个标记点的所有相机进行两两匹配，对匹配的两个相机在同帧中捕捉到的两个所述二维空间点坐标，通过奇异值分解求解最小二乘法方法，解算得到一组三维空间点坐标；剔除单元，用于判断所述三维空间点坐标是否处于预设的阈值范围内，若 超过所述阈值范围，则剔除所述三维空间点坐标，得到剔除后的一组所述三维空间点坐标；确定三维空间点坐标单元，用于计算一组所述三维空间点坐标的平均值，通过高斯牛顿法优化，得到所述标记点的三维空间点坐标；确定三维空间点编码单元，用于将所述标记点的二维空间点编码赋值给所述三维空间点坐标对应的编码，得到所述标记点的三维空间点编码。
- 根据权利要求5所述的多相机环境中主动式刚体的位姿定位装置，其特征在于，所述计算刚体数据模块，还包括：计算原点单元，用于计算同帧的多个所述标记点对应的所述三维空间点坐标的坐标平均值，将所述坐标平均值记为刚体坐标系下的原点；确定刚体坐标单元，用于分别计算原点与同帧的每个所述标记点对应的所述三维空间点坐标之间的差值，得到每个所述标记点每帧的刚体坐标；确定刚体编码单元，用于将所述标记点的三维空间点编码赋值给所述刚体坐标对应的编码，得到所述标记点的刚体编码。
- 根据权利要求5所述的多相机环境中主动式刚体的位姿定位装置，其特征在于，所述确定刚体初始位姿模块，包括：求解矩阵单元，用于求解第一帧所述三维空间点坐标到所述刚体坐标的位姿估计时，将所述三维空间点坐标和所述刚体坐标代入等式中，通过迭代最近点求解欧式变换旋转矩阵和平移矩阵，所述等式如下公式：P1＝RP1′+T其中，P1为第一帧所述三维空间点坐标，P1′为第一帧所述刚体坐标，R为刚体的欧式变换旋转矩阵，T为平移矩阵；根据所述欧式变换旋转矩阵和所述平移矩阵得到刚体的初始位姿。
- 一种多相机环境中主动式刚体的位姿定位设备，其特征在于，所述设备 包括：存储器、处理器以及存储在所述存储器上并可在所述处理器上运行的多相机环境中主动式刚体的位姿定位程序，所述多相机环境中主动式刚体的位姿定位程序被所述处理器执行时实现如权利要求1至4中任一项所述的多相机环境中主动式刚体的位姿定位方法的步骤。
- 一种计算机可读存储介质，其特征在于，所述计算机可读存储介质上存储有多相机环境中主动式刚体的位姿定位程序，所述多相机环境中主动式刚体的位姿定位程序被处理器执行时实现如权利要求1至4中任一项所述的多相机环境中主动式刚体的位姿定位方法的步骤。
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910937826.3A CN110689584B (zh) | 2019-09-30 | 2019-09-30 | 多相机环境中主动式刚体的位姿定位方法及相关设备 |
| CN201910937826.3 | 2019-09-30 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2021063127A1 true WO2021063127A1 (zh) | 2021-04-08 |
Family
ID=69111054
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2020/110248 WO2021063127A1 (zh) | 2019-09-30 | 2020-08-20 | 多相机环境中主动式刚体的位姿定位方法及相关设备 |
Country Status (2)
| Country | Link |
|---|---|
| CN (2) | CN110689584B (zh) |
| WO (1) | WO2021063127A1 (zh) |
Cited By (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113223050A (zh) * | 2021-05-12 | 2021-08-06 | 之江实验室 | 一种基于ArUco码的机器人运动轨迹实时采集方法 |
| CN113450417A (zh) * | 2021-05-12 | 2021-09-28 | 深圳市格灵精睿视觉有限公司 | 标定参数优化方法、装置、设备及存储介质 |
| CN114004901A (zh) * | 2022-01-04 | 2022-02-01 | 南昌虚拟现实研究院股份有限公司 | 多相机标定方法、装置、终端设备及可读存储介质 |
| CN114092561A (zh) * | 2022-01-24 | 2022-02-25 | 之江实验室 | 一种基于多目相机的无纹理三维物体跟踪方法及装置 |
| CN114092564A (zh) * | 2021-10-29 | 2022-02-25 | 上海科技大学 | 无重叠视域多相机系统的外参数标定方法、系统、终端及介质 |
| CN114789446A (zh) * | 2022-05-27 | 2022-07-26 | 平安普惠企业管理有限公司 | 机器人位姿估计方法、装置、电子设备及存储介质 |
| CN114808754A (zh) * | 2022-05-18 | 2022-07-29 | 中铁二十四局集团有限公司 | 一种大型转体桥梁空间位置高精度实时预测方法 |
| CN115388911A (zh) * | 2022-08-24 | 2022-11-25 | Oppo广东移动通信有限公司 | 光学动作捕捉系统的精度测量方法、装置、电子设备 |
| CN117197241A (zh) * | 2023-09-14 | 2023-12-08 | 上海智能制造功能平台有限公司 | 一种基于多目视觉的机器人末端绝对位姿高精度跟踪方法 |
| CN117523678A (zh) * | 2024-01-04 | 2024-02-06 | 广东茉莉数字科技集团股份有限公司 | 一种基于光学动作数据的虚拟主播区分方法及系统 |
Families Citing this family (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN114170307A (zh) * | 2019-09-30 | 2022-03-11 | 深圳市瑞立视多媒体科技有限公司 | 单相机环境中主动式刚体的位姿定位方法及相关设备 |
| CN110689584B (zh) * | 2019-09-30 | 2021-09-03 | 深圳市瑞立视多媒体科技有限公司 | 多相机环境中主动式刚体的位姿定位方法及相关设备 |
| WO2021196108A1 (zh) * | 2020-04-02 | 2021-10-07 | 深圳市瑞立视多媒体科技有限公司 | 大空间环境下边扫场边标定方法、装置、设备及存储介质 |
| CN112985411A (zh) * | 2021-03-02 | 2021-06-18 | 南京航空航天大学 | 一种气浮台靶标布局以及姿态解算方法 |
| CN113358325A (zh) * | 2021-07-02 | 2021-09-07 | 中国空气动力研究与发展中心低速空气动力研究所 | 一种风洞投放物位姿测量方法、装置、设备及存储介质 |
| CN116580083B (zh) * | 2023-07-13 | 2023-09-22 | 深圳创维智慧科技有限公司 | 摄像设备的位姿估计方法、装置、电子设备及存储介质 |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106097300A (zh) * | 2016-05-27 | 2016-11-09 | 西安交通大学 | 一种基于高精度运动平台的多相机标定方法 |
| EP3166074A1 (en) * | 2015-04-22 | 2017-05-10 | Thomson Licensing | Method of camera calibration for a multi-camera system and apparatus performing the same |
| CN110244282A (zh) * | 2019-06-10 | 2019-09-17 | 于兴虎 | 一种多相机系统和激光雷达联合系统及其联合标定方法 |
| CN110689584A (zh) * | 2019-09-30 | 2020-01-14 | 深圳市瑞立视多媒体科技有限公司 | 多相机环境中主动式刚体的位姿定位方法及相关设备 |
Family Cites Families (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101226638B (zh) * | 2007-01-18 | 2010-05-19 | 中国科学院自动化研究所 | 一种对多相机系统的标定方法及装置 |
| US9959625B2 (en) * | 2015-12-29 | 2018-05-01 | The United States Of America As Represented By The Secretary Of The Air Force | Method for fast camera pose refinement for wide area motion imagery |
| CN108269286B (zh) * | 2016-12-30 | 2020-06-19 | 中国空气动力研究与发展中心超高速空气动力研究所 | 基于组合立体标志的多相机位姿关联方法 |
| CN106875451B (zh) * | 2017-02-27 | 2020-09-08 | 安徽华米智能科技有限公司 | 相机标定方法、装置及电子设备 |
| CN109712172A (zh) * | 2018-12-28 | 2019-05-03 | 哈尔滨工业大学 | 一种初始位姿测量结合目标追踪的位姿测量方法 |
-
2019
- 2019-09-30 CN CN201910937826.3A patent/CN110689584B/zh active Active
- 2019-09-30 CN CN202110890276.1A patent/CN113643378B/zh active Active
-
2020
- 2020-08-20 WO PCT/CN2020/110248 patent/WO2021063127A1/zh active Application Filing
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP3166074A1 (en) * | 2015-04-22 | 2017-05-10 | Thomson Licensing | Method of camera calibration for a multi-camera system and apparatus performing the same |
| CN106097300A (zh) * | 2016-05-27 | 2016-11-09 | 西安交通大学 | 一种基于高精度运动平台的多相机标定方法 |
| CN110244282A (zh) * | 2019-06-10 | 2019-09-17 | 于兴虎 | 一种多相机系统和激光雷达联合系统及其联合标定方法 |
| CN110689584A (zh) * | 2019-09-30 | 2020-01-14 | 深圳市瑞立视多媒体科技有限公司 | 多相机环境中主动式刚体的位姿定位方法及相关设备 |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113450417A (zh) * | 2021-05-12 | 2021-09-28 | 深圳市格灵精睿视觉有限公司 | 标定参数优化方法、装置、设备及存储介质 |
| CN113223050A (zh) * | 2021-05-12 | 2021-08-06 | 之江实验室 | 一种基于ArUco码的机器人运动轨迹实时采集方法 |
| CN114092564B (zh) * | 2021-10-29 | 2024-04-09 | 上海科技大学 | 无重叠视域多相机系统的外参数标定方法、系统、终端及介质 |
| CN114092564A (zh) * | 2021-10-29 | 2022-02-25 | 上海科技大学 | 无重叠视域多相机系统的外参数标定方法、系统、终端及介质 |
| CN114004901A (zh) * | 2022-01-04 | 2022-02-01 | 南昌虚拟现实研究院股份有限公司 | 多相机标定方法、装置、终端设备及可读存储介质 |
| CN114004901B (zh) * | 2022-01-04 | 2022-03-18 | 南昌虚拟现实研究院股份有限公司 | 多相机标定方法、装置、终端设备及可读存储介质 |
| CN114092561A (zh) * | 2022-01-24 | 2022-02-25 | 之江实验室 | 一种基于多目相机的无纹理三维物体跟踪方法及装置 |
| CN114808754A (zh) * | 2022-05-18 | 2022-07-29 | 中铁二十四局集团有限公司 | 一种大型转体桥梁空间位置高精度实时预测方法 |
| CN114789446A (zh) * | 2022-05-27 | 2022-07-26 | 平安普惠企业管理有限公司 | 机器人位姿估计方法、装置、电子设备及存储介质 |
| CN115388911A (zh) * | 2022-08-24 | 2022-11-25 | Oppo广东移动通信有限公司 | 光学动作捕捉系统的精度测量方法、装置、电子设备 |
| CN117197241A (zh) * | 2023-09-14 | 2023-12-08 | 上海智能制造功能平台有限公司 | 一种基于多目视觉的机器人末端绝对位姿高精度跟踪方法 |
| CN117523678A (zh) * | 2024-01-04 | 2024-02-06 | 广东茉莉数字科技集团股份有限公司 | 一种基于光学动作数据的虚拟主播区分方法及系统 |
| CN117523678B (zh) * | 2024-01-04 | 2024-04-05 | 广东茉莉数字科技集团股份有限公司 | 一种基于光学动作数据的虚拟主播区分方法及系统 |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110689584B (zh) | 2021-09-03 |
| CN113643378A (zh) | 2021-11-12 |
| CN113643378B (zh) | 2023-06-09 |
| CN110689584A (zh) | 2020-01-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2021063127A1 (zh) | 多相机环境中主动式刚体的位姿定位方法及相关设备 | |
| WO2021063128A1 (zh) | 单相机环境中主动式刚体的位姿定位方法及相关设备 | |
| US20220254105A1 (en) | Systems and Methods for 3D Facial Modeling | |
| TWI569229B (zh) | 資料之登錄方法 | |
| JP6862584B2 (ja) | 画像処理システム及び画像処理方法 | |
| TWI624170B (zh) | 影像掃描系統及其方法 | |
| JP6079333B2 (ja) | 校正装置、方法及びプログラム | |
| JP2018028899A (ja) | 画像レジストレーションの方法及びシステム | |
| US20200334842A1 (en) | Methods, devices and computer program products for global bundle adjustment of 3d images | |
| CN112200057B (zh) | 人脸活体检测方法、装置、电子设备及存储介质 | |
| CN107808398B (zh) | 摄像头参数算出装置以及算出方法、程序、记录介质 | |
| JPH10124658A (ja) | 神経回路網を利用したカメラの映像歪補正方法 | |
| CN112200157A (zh) | 一种降低图像背景干扰的人体3d姿态识别方法及其系统 | |
| KR20200142391A (ko) | 광학식 위치 추적 시스템의 3차원 마커 좌표 추정 방법 | |
| Jacob et al. | Depth information enhancement using block matching and image pyramiding stereo vision enabled RGB-D sensor | |
| KR102229861B1 (ko) | 저채널 라이다와 스테레오 카메라를 이용한 깊이 추정 장치 및 방법 | |
| CN110428461B (zh) | 结合深度学习的单目slam方法及装置 | |
| JP7166446B2 (ja) | ロボットの姿勢を推定するシステムおよび方法、ロボット、並びに記憶媒体 | |
| CN114266823A (zh) | 一种结合SuperPoint网络特征提取的单目SLAM方法 | |
| KR20150097251A (ko) | 다중 영상간 대응점을 이용한 카메라 정렬 방법 | |
| CN113628334A (zh) | 视觉slam方法、装置、终端设备及存储介质 | |
| JP6065670B2 (ja) | ３次元計測システム、プログラム及び方法。 | |
| CN110827394B (zh) | 脸部表情建构方法、装置及非暂态电脑可读取记录媒体 | |
| CN113643328B (zh) | 标定物的重建方法、装置、电子设备及计算机可读介质 | |
| JP2016102755A (ja) | 情報処理装置、情報処理方法及びプログラム |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 20870865 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 20870865 Country of ref document: EP Kind code of ref document: A1 |