CN113048985A - Camera relative motion estimation method under known relative rotation angle condition - Google Patents

Camera relative motion estimation method under known relative rotation angle condition Download PDF

Info

Publication number
CN113048985A
CN113048985A CN202110596663.4A CN202110596663A CN113048985A CN 113048985 A CN113048985 A CN 113048985A CN 202110596663 A CN202110596663 A CN 202110596663A CN 113048985 A CN113048985 A CN 113048985A
Authority
CN
China
Prior art keywords
camera
view
affine
matrix
translation vector
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110596663.4A
Other languages
Chinese (zh)
Other versions
CN113048985B (en
Inventor
关棒磊
谭泽
李璋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
National University of Defense Technology
Original Assignee
National University of Defense Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by National University of Defense Technology filed Critical National University of Defense Technology
Priority to CN202110596663.4A priority Critical patent/CN113048985B/en
Publication of CN113048985A publication Critical patent/CN113048985A/en
Application granted granted Critical
Publication of CN113048985B publication Critical patent/CN113048985B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/20—Instruments for performing navigational calculations
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01C—MEASURING DISTANCES, LEVELS OR BEARINGS; SURVEYING; NAVIGATION; GYROSCOPIC INSTRUMENTS; PHOTOGRAMMETRY OR VIDEOGRAMMETRY
    • G01C21/00—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00
    • G01C21/10—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration
    • G01C21/12—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning
    • G01C21/16—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation
    • G01C21/165—Navigation; Navigational instruments not provided for in groups G01C1/00 - G01C19/00 by using measurements of speed or acceleration executed aboard the object being navigated; Dead reckoning by integrating acceleration or speed, i.e. inertial navigation combined with non-inertial navigation instruments
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G06T7/262—Analysis of motion using transform domain methods, e.g. Fourier domain methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G06T7/75—Determining position or orientation of objects or cameras using feature-based methods involving models

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Remote Sensing (AREA)
  • Radar, Positioning & Navigation (AREA)
  • General Physics & Mathematics (AREA)
  • Mathematical Physics (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Optimization (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Mathematical Analysis (AREA)
  • Computational Mathematics (AREA)
  • Pure & Applied Mathematics (AREA)
  • Automation & Control Theory (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Computing Systems (AREA)
  • Multimedia (AREA)
  • Algebra (AREA)
  • Image Analysis (AREA)
  • Length Measuring Devices By Optical Means (AREA)

Abstract

The application relates to a camera relative motion estimation method based on affine matching point pairs under the condition of known relative rotation angle. When the inertial measurement unit is fixedly connected with the camera under the installation condition, the relative rotation angles of the inertial measurement unit and the camera in the motion process are the same, and the relative rotation angle output by the inertial measurement unit can be directly used as the relative rotation angle of the camera or a plurality of cameras, so that the method can be applied to wider scenes such as unknown installation relation or changed installation relation between the inertial measurement unit and the camera. Meanwhile, the situation that the relative rotation angle of the camera or the multi-camera system is known is introduced, and the relative motion of the single camera and the multi-camera system can be estimated by adopting the image coordinates of the two affine matching point pairs and the local affine transformation matrix in the field. Therefore, the calculation accuracy and efficiency are improved, and the method is suitable for equipment with limited calculation capacity, such as unmanned aerial vehicle autonomous navigation, automatic driving automobiles and augmented reality equipment.

Description

已知相对旋转角度条件下的像机相对运动估计方法Estimation method of camera relative motion under the condition of known relative rotation angle

技术领域technical field

本申请涉及定位技术领域，特别是涉及一种已知相对旋转角度条件下的像机相对运动估计方法。The present application relates to the field of positioning technology, and in particular, to a method for estimating relative motion of a camera under the condition of a known relative rotation angle.

背景技术Background technique

单像机或多像机的相对运动估计是三维视觉中的一个基本问题，例如机器人定位和地图绘制，增强现实和自动驾驶。单像机系统采用中心透视投影模型建模，单像机的相对运动估计通常利用5个同名点对的本质矩阵算法，或者利用4个同名点对的单应性矩阵算法。多像机系统由固定在单一刚体上的多个单像机组成。它采用通用像机模型建模，且没有单一的投影中心。多像机的相对运动估计通常利用6个同名点对的最小配置解，或者17个同名点对的线性求解方法。Relative motion estimation of single or multi-cameras is a fundamental problem in 3D vision, such as robot localization and mapping, augmented reality and autonomous driving. The single camera system is modeled by the central perspective projection model, and the relative motion estimation of the single camera usually uses the essential matrix algorithm of 5 point pairs with the same name, or the homography matrix algorithm of 4 point pairs with the same name. A multi-camera system consists of multiple single cameras fixed on a single rigid body. It is modeled with a generic camera model and does not have a single center of projection. The relative motion estimation of the multi-camera usually utilizes the minimum configuration solution of 6 identical point pairs, or the linear solution method of 17 identical dot pairs.

当前，手机、无人机等电子设备中通常都有惯性测量单元和像机等传感器，而且惯性测量单元和像机固联安装，因此可以利用惯性测量单元为像机相对运动估计提供旋转角度信息，主要可分为以下两种情况：（1）若惯性测量单元与像机之间的安装关系事先精确标定已知，根据惯性测量单元输出的旋转角度能够直接为像机提供姿态角，从而减少像机相对运动估计中待求解的位姿参数。例如，在惯性测量单元为像机提供统一重力方向的条件下，利用3个同名点对能够估计单像机系统的相对运动，4个同名点对能够估计多像机系统的相对运动。（2）若惯性测量单元与像机之间的安装关系未知，可根据两者固联安装条件下，惯性测量单元和像机在运动过程中各自的相对旋转角相同这一性质，惯性测量单元输出的相对旋转角可以直接用作像机的相对旋转角，同样能减少像机相对运动估计中待求解的位姿参数。利用4个同名点对和5个同名点对分别能估计单像机系统和多像机系统的相对运动。由于不需要对惯性测量单元与像机进行标定，使得惯性测量单元和像机的融合更加灵活方便，在三维重建、视觉里程计等领域具有广泛的应用场景。At present, there are usually sensors such as inertial measurement units and cameras in electronic devices such as mobile phones and drones, and the inertial measurement unit and the camera are fixedly installed. Therefore, the inertial measurement unit can be used to provide rotation angle information for the relative motion estimation of the camera. , which can be mainly divided into the following two cases: (1) If the installation relationship between the inertial measurement unit and the camera is accurately calibrated and known in advance, the rotation angle output by the inertial measurement unit can directly provide the camera with the attitude angle, thereby reducing the The pose parameters to be solved in the camera relative motion estimation. For example, under the condition that the inertial measurement unit provides the camera with a uniform gravity direction, the relative motion of the single-camera system can be estimated by using 3 point pairs with the same name, and the relative motion of the multi-camera system can be estimated with 4 point pairs with the same name. (2) If the installation relationship between the inertial measurement unit and the camera is unknown, according to the property that the inertial measurement unit and the camera have the same relative rotation angle during the movement process under the fixed installation condition of the two, the inertial measurement unit The output relative rotation angle can be directly used as the relative rotation angle of the camera, which can also reduce the pose parameters to be solved in the relative motion estimation of the camera. The relative motion of the single-camera system and the multi-camera system can be estimated by using 4 identical point pairs and 5 identical name point pairs, respectively. Since there is no need to calibrate the inertial measurement unit and the camera, the integration of the inertial measurement unit and the camera is more flexible and convenient, and it has a wide range of application scenarios in the fields of 3D reconstruction and visual odometry.

传统的相对运动估计算法通常采用SIFT和SURF等特征算法获得同名点对。目前多视图几何估计中，通过ASIFT和MODS等特征算法提取的仿射匹配对点，因为包含更多的图像点对信息而受到了越来越多的关注。仿射匹配对点不仅包括同名点对的图像坐标，而且包括同名点对之间领域信息的局部仿射矩阵。每个仿射匹配点对在对极几何关系上给出三个约束方程，能够有效减少相对运动估计方法所需点对的数量。Traditional relative motion estimation algorithms usually use feature algorithms such as SIFT and SURF to obtain point pairs with the same name. In the current multi-view geometric estimation, the affine matching points extracted by feature algorithms such as ASIFT and MODS have received more and more attention because they contain more image point pair information. Affine matching pairs of points include not only the image coordinates of the same-named point pairs, but also the local affine matrix of domain information between the same-named point-pairs. Each affine matching point pair gives three constraint equations on the epipolar geometric relationship, which can effectively reduce the number of point pairs required by the relative motion estimation method.

发明内容SUMMARY OF THE INVENTION

基于此，有必要针对上述技术问题，提供一种能够利用仿射匹配点对解决已知相对旋转角度条件下的像机相对运动估计方法。Based on this, it is necessary to provide a method for estimating the relative motion of a camera under the condition of known relative rotation angles by using affine matching point pairs to solve the above technical problems.

一种已知相对旋转角度条件下的像机相对运动估计方法，所述方法包括：A method for estimating relative motion of a camera under the condition of a known relative rotation angle, the method comprising:

获取像机拍摄的第一视图和第二视图中至少两个仿射匹配点对，选择第j个所述仿射匹配点对建立世界参考系；所述世界参考系的原点为第j个所述仿射匹配点对在三维空间中的位置，所述世界参考系的坐标轴方向与所述第一视图方向一致；Obtain at least two affine matching point pairs in the first view and the second view captured by the camera, and select the jth affine matching point pair to establish a world reference system; the origin of the world reference system is the jth affine matching point pair. the position of the affine matching point pair in the three-dimensional space, and the coordinate axis direction of the world reference system is consistent with the first view direction;

获取所述第一视图和所述第二视图之间的第一位姿关系，获取所述第一视图和所述世界参考系的第二位姿关系，以及获取第二视图和所述世界参考系的第三位姿关系；所述第一位姿关系、所述第二位姿关系以及所述第三位姿关系中均包括旋转矩阵和平移向量；obtaining a first pose relationship between the first view and the second view, obtaining a second pose relationship between the first view and the world reference frame, and obtaining a second view and the world reference The third pose relationship of the system; the first pose relationship, the second pose relationship and the third pose relationship all include a rotation matrix and a translation vector;

对所述旋转矩阵和所述平移向量进行参数化，以及根据所述第一视图和所述第二视图之间的相对旋转角度，确定所述旋转矩阵对应未知数的旋转参数约束；Parameterizing the rotation matrix and the translation vector, and determining the rotation parameter constraint of the unknown corresponding to the rotation matrix according to the relative rotation angle between the first view and the second view;

采用参数化后的所述旋转矩阵和所述平移向量表示所述第一位姿关系以及获取对应的本质矩阵；Using the parameterized rotation matrix and the translation vector to represent the first attitude relationship and obtain the corresponding essential matrix;

获取由第j个所述仿射匹配点确定的所述本质矩阵与所述仿射匹配点中仿射匹配矩阵对应的两个仿射变换约束，获取其他所述仿射匹配点确定的所述第一视图和所述第二视图的一个对极几何约束以及所述本质矩阵与所述仿射匹配点中仿射匹配矩阵对应的两个仿射变换约束；Obtain the essential matrix determined by the jth affine matching point and two affine transformation constraints corresponding to the affine matching matrix in the affine matching point, and obtain the One epipolar geometric constraint of the first view and the second view and two affine transformation constraints corresponding to the essential matrix and the affine matching matrix in the affine matching point;

根据第j个所述仿射匹配点对应的两个仿射变换约束，以及其他所述仿射匹配点确定的一个对极几何约束和两个仿射变换约束，求解得到所述旋转矩阵和所述平移向量，根据所述旋转矩阵和所述平移向量确定像机相对运动关系。According to the two affine transformation constraints corresponding to the jth affine matching point, as well as one epipolar geometric constraint and two affine transformation constraints determined by the other affine matching points, the rotation matrix and all the The translation vector is used to determine the relative motion relationship of the camera according to the rotation matrix and the translation vector.

上述已知相对旋转角度条件下的像机相对运动估计方法，利用惯性测量单元提供的相对旋转角度，并根据仿射匹配对点和像机运动模型之间的约束。提出了基于两个仿射匹配点对的相对运动估计最小解，分别估计单个像机和多像机系统的运动，从而极大减少求解单像机和多像机系统相对运动估计问题所需要的点对数量，明显提高了算法的精度及鲁棒性，并且适用于惯性测量单元与像机之间安装关系未知或存在变化的情况。The above method for estimating the relative motion of the camera under the condition of the known relative rotation angle uses the relative rotation angle provided by the inertial measurement unit, and matches the constraints between the point and the camera motion model according to affine matching. A minimum solution for relative motion estimation based on two affine matched point pairs is proposed, which estimates the motion of a single camera and a multi-camera system respectively, thereby greatly reducing the relative motion estimation problem of single-camera and multi-camera systems. The number of point pairs significantly improves the accuracy and robustness of the algorithm, and is suitable for situations where the installation relationship between the inertial measurement unit and the camera is unknown or changes.

附图说明Description of drawings

图1为一个实施例中已知相对旋转角度条件下的像机相对运动估计方法；1 is a method for estimating relative motion of a camera under the condition of a known relative rotation angle in one embodiment;

图2为一个实施例中单像机参数分布示意图；Fig. 2 is a schematic diagram of parameter distribution of a single camera in one embodiment;

图3为一个实施例中多像机参数分布示意图。FIG. 3 is a schematic diagram of parameter distribution of a multi-camera in an embodiment.

具体实施方式Detailed ways

为了使本申请的目的、技术方案及优点更加清楚明白，以下结合附图及实施例，对本申请进行进一步详细说明。应当理解，此处描述的具体实施例仅仅用以解释本申请，并不用于限定本申请。In order to make the purpose, technical solutions and advantages of the present application more clearly understood, the present application will be described in further detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, but not to limit the present application.

在一个实施例中，如图1所示，提供了一种已知相对旋转角度条件下的像机相对运动估计方法，包括以下步骤：In one embodiment, as shown in FIG. 1 , a method for estimating the relative motion of a camera under the condition of a known relative rotation angle is provided, including the following steps:

步骤102，获取像机拍摄的第一视图和第二视图中至少两个仿射匹配点对，选择第j个仿射匹配点对建立世界参考系。Step 102: Acquire at least two affine matching point pairs in the first view and the second view captured by the camera, and select the jth affine matching point pair to establish a world reference system.

世界参考系的原点为第j个仿射匹配点对在三维空间中的位置，世界参考系的坐标轴方向与第一视图方向一致。The origin of the world reference system is the position of the jth affine matching point pair in the three-dimensional space, and the coordinate axis direction of the world reference system is consistent with the direction of the first view.

步骤104，获取第一视图和第二视图之间的第一位姿关系，获取第一视图和世界参考系的第二位姿关系，以及获取第二视图和世界参考系的第三位姿关系。Step 104: Obtain the first pose relationship between the first view and the second view, obtain the second pose relationship between the first view and the world reference system, and obtain the third pose relationship between the second view and the world reference system .

第一位姿关系、第二位姿关系以及第三位姿关系中均包括旋转矩阵和平移向量。The first pose relationship, the second pose relationship, and the third pose relationship all include a rotation matrix and a translation vector.

步骤106，对旋转矩阵和平移向量进行参数化，以及根据第一视图和第二视图之间的相对旋转角度，确定旋转矩阵对应未知数的旋转参数约束，采用参数化后的旋转矩阵和平移向量表示第一位姿关系以及获取对应的本质矩阵。Step 106: Parameterize the rotation matrix and the translation vector, and determine the rotation parameter constraint of the unknown number corresponding to the rotation matrix according to the relative rotation angle between the first view and the second view, and use the parameterized rotation matrix and translation vector to represent The first pose relationship and the corresponding essential matrix are obtained.

步骤108，获取由第j个仿射匹配点确定的本质矩阵与仿射匹配点中仿射匹配矩阵对应的两个仿射变换约束，获取其他仿射匹配点确定的第一视图和第二视图的一个对极几何约束以及本质矩阵与仿射匹配点中仿射匹配矩阵对应的两个仿射变换约束。Step 108: Obtain the essential matrix determined by the jth affine matching point and two affine transformation constraints corresponding to the affine matching matrix in the affine matching point, and obtain the first view and the second view determined by other affine matching points. An epipolar geometric constraint of , and two affine transformation constraints corresponding to the essential matrix and the affine matching matrix in the affine matching point.

步骤110，根据第j个仿射匹配点对应的两个仿射变换约束，以及其他仿射匹配点确定的一个对极几何约束和两个仿射变换约束，得到旋转矩阵和平移向量，根据旋转矩阵和平移向量确定像机相对运动关系。Step 110: Obtain a rotation matrix and a translation vector according to the two affine transformation constraints corresponding to the jth affine matching point, as well as one epipolar geometric constraint and two affine transformation constraints determined by other affine matching points, and according to the rotation The matrix and translation vector determine the relative motion relationship of the camera.

上述已知相对旋转角度条件下的像机相对运动估计方法，利用惯性测量单元提供的相对旋转角度，并根据仿射匹配对点和像机运动模型之间的约束。提出了基于两个仿射匹配点对的相对运动估计最小解，分别估计单个像机和多像机系统的运动，从而极大减少求解单像机和多像机系统相对运动估计问题所需要的点对数量，明显提高了算法的精度及鲁棒性，并且适用于惯性测量单元与像机之间安装关系未知或存在变化的情况。The above method for estimating the relative motion of the camera under the condition of the known relative rotation angle uses the relative rotation angle provided by the inertial measurement unit, and matches the constraints between the point and the camera motion model according to affine matching. A minimum solution for relative motion estimation based on two affine matched point pairs is proposed, which estimates the motion of a single camera and a multi-camera system respectively, thereby greatly reducing the relative motion estimation problem of single-camera and multi-camera systems. The number of point pairs significantly improves the accuracy and robustness of the algorithm, and is suitable for situations where the installation relationship between the inertial measurement unit and the camera is unknown or changes.

对于单像机相对运动估计，假定第j个仿射匹配点对表示为

Figure 514165DEST_PATH_IMAGE001
，其中
Figure 710660DEST_PATH_IMAGE002
和
Figure 885289DEST_PATH_IMAGE003
分别是第一视图和第二视图中同名点对的归一化齐次图像坐标，
Figure 892560DEST_PATH_IMAGE004
是2×2的局部仿射变换矩阵，
Figure 355902DEST_PATH_IMAGE005
表征着
Figure 231454DEST_PATH_IMAGE006
和
Figure 463852DEST_PATH_IMAGE007
周围无穷小邻域内的仿射变换关系。同名点对相应的单位方向向量可以通过如下等式计算：
Figure 625712DEST_PATH_IMAGE008
，
Figure 638668DEST_PATH_IMAGE009
。本发明实施例输入条件是两个仿射匹配点对（最少一个仿射匹配点对和一个同名点对）和惯性测量单元提供的单像机相对旋转角度。For single-camera relative motion estimation, it is assumed that the j-th affine matching point pair is expressed as
Figure 514165DEST_PATH_IMAGE001
,in
Figure 710660DEST_PATH_IMAGE002
and
Figure 885289DEST_PATH_IMAGE003
are the normalized homogeneous image coordinates of point pairs with the same name in the first and second views, respectively,
Figure 892560DEST_PATH_IMAGE004
is a 2×2 local affine transformation matrix,
Figure 355902DEST_PATH_IMAGE005
signifies
Figure 231454DEST_PATH_IMAGE006
and
Figure 463852DEST_PATH_IMAGE007
Affine transformation relation in the surrounding infinitesimal neighborhood. The corresponding unit direction vector of the point pair with the same name can be calculated by the following equation:
Figure 625712DEST_PATH_IMAGE008
,
Figure 638668DEST_PATH_IMAGE009
. The input conditions of the embodiment of the present invention are two affine matching point pairs (at least one affine matching point pair and one point pair with the same name) and the relative rotation angle of the single camera provided by the inertial measurement unit.

在其中一个实施例中，获取对旋转矩阵进行参数化得到的旋转矩阵为：In one of the embodiments, the rotation matrix obtained by parameterizing the rotation matrix is obtained as:

Figure 255594DEST_PATH_IMAGE010
Figure 255594DEST_PATH_IMAGE010

其中，

Figure 280182DEST_PATH_IMAGE011
是四元数齐次向量，R表示第一位姿关系对应的旋转矩阵；in,
Figure 280182DEST_PATH_IMAGE011
is a quaternion homogeneous vector, and R represents the rotation matrix corresponding to the first pose relationship;

根据第一视图和第二视图之间的相对旋转角度，确定旋转矩阵对应未知数的旋转参数约束为：According to the relative rotation angle between the first view and the second view, the rotation parameter constraint of the unknown corresponding to the rotation matrix is determined as:

Figure 753888DEST_PATH_IMAGE012
Figure 753888DEST_PATH_IMAGE012

其中，

Figure 926244DEST_PATH_IMAGE013
表示相对旋转角度。in,
Figure 926244DEST_PATH_IMAGE013
Indicates the relative rotation angle.

在其中一个实施例中，获取对平移向量进行参数化得到的第二位姿关系中平移向量和第三位姿关系中平移向量分别为：In one of the embodiments, obtaining the translation vector in the second pose relationship and the translation vector in the third pose relationship obtained by parameterizing the translation vector are:

Figure 268232DEST_PATH_IMAGE014
Figure 268232DEST_PATH_IMAGE014

其中，

Figure 271960DEST_PATH_IMAGE015
表示所述第二位姿关系中平移向量，
Figure 854251DEST_PATH_IMAGE016
表示所述第二位姿关系中平移向量参数化的未知深度参数，
Figure 186007DEST_PATH_IMAGE017
表示第一视图中归一化齐次图像坐标的单位向量，
Figure 207052DEST_PATH_IMAGE018
表示第三位姿关系中平移向量，
Figure 268549DEST_PATH_IMAGE019
表示第三位姿关系中平移向量的未知深度参数，
Figure 208692DEST_PATH_IMAGE020
表示第二视图中归一化齐次图像坐标的单位向量；in,
Figure 271960DEST_PATH_IMAGE015
represents the translation vector in the second pose relationship,
Figure 854251DEST_PATH_IMAGE016
represents the unknown depth parameter parameterized by the translation vector in the second pose relationship,
Figure 186007DEST_PATH_IMAGE017
a unit vector representing the normalized homogeneous image coordinates in the first view,
Figure 207052DEST_PATH_IMAGE018
represents the translation vector in the third pose relation,
Figure 268549DEST_PATH_IMAGE019
represents the unknown depth parameter of the translation vector in the third pose relation,
Figure 208692DEST_PATH_IMAGE020
a unit vector representing the normalized homogeneous image coordinates in the second view;

采用参数化后的旋转矩阵和平移向量表示第一位姿关系为：Using the parameterized rotation matrix and translation vector to represent the first pose relationship is:

Figure 417957DEST_PATH_IMAGE021
Figure 417957DEST_PATH_IMAGE021

其中，

Figure 586901DEST_PATH_IMAGE022
表示第一位姿关系中的平移向量，I表示单位矩阵；in,
Figure 586901DEST_PATH_IMAGE022
represents the translation vector in the first pose relation, and I represents the identity matrix;

以及获取对应的本质矩阵为：And get the corresponding essential matrix as:

Figure 299642DEST_PATH_IMAGE023
Figure 299642DEST_PATH_IMAGE023

其中，E表示本质矩阵，

Figure 410686DEST_PATH_IMAGE024
表示反对称矩阵。where E represents the essential matrix,
Figure 410686DEST_PATH_IMAGE024
represents an antisymmetric matrix.

在其中一个实施例中，对极几何约束为：In one embodiment, the epipolar geometric constraints are:

Figure 44930DEST_PATH_IMAGE025
Figure 44930DEST_PATH_IMAGE025

其中，

Figure 407778DEST_PATH_IMAGE026
为第一视图和第二视图中同名点对的归一化齐次图像坐标；in,
Figure 407778DEST_PATH_IMAGE026
is the normalized homogeneous image coordinates of the point pairs with the same name in the first view and the second view;

仿射变换约束为：The affine transformation constraints are:

Figure 850392DEST_PATH_IMAGE027
Figure 850392DEST_PATH_IMAGE027

其中，下标(1:2)表示前两个方程式。where the subscript (1:2) represents the first two equations.

在其中一个实施例中，根据旋转参数约束，确定单像机的相对运动参数为四个自由度；选择所述第j个仿射匹配点对应的两个仿射变换约束，以及其他仿射匹配点确定的一个对极几何约束，构建第一求解模型为：In one embodiment, according to the rotation parameter constraints, the relative motion parameters of the single camera are determined to be four degrees of freedom; two affine transformation constraints corresponding to the jth affine matching point, and other affine matching points are selected. An epipolar geometric constraint determined by the point, the first solution model is constructed as:

Figure 210966DEST_PATH_IMAGE028
；
Figure 210966DEST_PATH_IMAGE028
;

其中，

Figure 394823DEST_PATH_IMAGE029
中的元素项为未知数
Figure 233466DEST_PATH_IMAGE030
，
Figure 311012DEST_PATH_IMAGE031
和
Figure 639225DEST_PATH_IMAGE032
的二次项，
Figure 982482DEST_PATH_IMAGE033
表示
Figure 828078DEST_PATH_IMAGE034
矩阵大小为三行二列；in,
Figure 394823DEST_PATH_IMAGE029
The element term in is unknown
Figure 233466DEST_PATH_IMAGE030
,
Figure 311012DEST_PATH_IMAGE031
and
Figure 639225DEST_PATH_IMAGE032
the quadratic term of ,
Figure 982482DEST_PATH_IMAGE033
express
Figure 828078DEST_PATH_IMAGE034
The size of the matrix is three rows and two columns;

选择其他仿射匹配点建立世界参考系，得到第二求解模型为：Select other affine matching points to establish the world reference system, and obtain the second solution model as:

Figure 573180DEST_PATH_IMAGE035
Figure 573180DEST_PATH_IMAGE035

其中，

Figure 337874DEST_PATH_IMAGE036
中的元素项为未知数
Figure 168427DEST_PATH_IMAGE037
，
Figure 535823DEST_PATH_IMAGE038
和
Figure 197749DEST_PATH_IMAGE039
的二次项；in,
Figure 337874DEST_PATH_IMAGE036
The element term in is unknown
Figure 168427DEST_PATH_IMAGE037
,
Figure 535823DEST_PATH_IMAGE038
and
Figure 197749DEST_PATH_IMAGE039
the quadratic term;

根据第一求解模型和第二求解模型，得到关于未知数

Figure 71027DEST_PATH_IMAGE040
的六个方程为：According to the first solution model and the second solution model, it is obtained about the unknown
Figure 71027DEST_PATH_IMAGE040
The six equations are:

Figure 326559DEST_PATH_IMAGE041
Figure 326559DEST_PATH_IMAGE041

通过Gröbner基求解法获得六个方程的代数解，根据代数解确定第一位姿关系的旋转矩阵R，根据

Figure 107433DEST_PATH_IMAGE042
中零空间确定
Figure 748499DEST_PATH_IMAGE043
，根据
Figure 792678DEST_PATH_IMAGE044
计算得到第二位姿关系中平移向量和第三位姿关系中平移向量，
Figure 394561DEST_PATH_IMAGE045
表示矩阵
Figure 120071DEST_PATH_IMAGE046
前二行前二列的子矩阵，
Figure 491010DEST_PATH_IMAGE047
表示矩阵
Figure 706090DEST_PATH_IMAGE048
前二行前二列的子矩阵，
Figure 919903DEST_PATH_IMAGE049
表示矩阵的行列式，
Figure 308159DEST_PATH_IMAGE050
表示由未知数
Figure 736866DEST_PATH_IMAGE051
组成的二行二列子矩阵。The algebraic solutions of the six equations are obtained by the Gröbner basis solution method, and the rotation matrix R of the first attitude relationship is determined according to the algebraic solutions.
Figure 107433DEST_PATH_IMAGE042
Medium null space determination
Figure 748499DEST_PATH_IMAGE043
,according to
Figure 792678DEST_PATH_IMAGE044
Calculate the translation vector in the second pose relationship and the translation vector in the third pose relationship,
Figure 394561DEST_PATH_IMAGE045
representation matrix
Figure 120071DEST_PATH_IMAGE046
The submatrix of the first two rows and the first two columns,
Figure 491010DEST_PATH_IMAGE047
representation matrix
Figure 706090DEST_PATH_IMAGE048
The submatrix of the first two rows and the first two columns,
Figure 919903DEST_PATH_IMAGE049
represents the determinant of the matrix,
Figure 308159DEST_PATH_IMAGE050
represented by the unknown
Figure 736866DEST_PATH_IMAGE051
A two-row two-column submatrix.

在其中一个实施例中，当像机为多像机系统时，获取多像机系统中像机

Figure 794952DEST_PATH_IMAGE052
的外参数
Figure 637006DEST_PATH_IMAGE053
；其中，多像机系统包括：拍摄第一视图或第二视图的像机
Figure 766636DEST_PATH_IMAGE054
，以及透视像机；In one embodiment, when the camera is a multi-camera system, the camera in the multi-camera system is acquired
Figure 794952DEST_PATH_IMAGE052
extrinsic parameters
Figure 637006DEST_PATH_IMAGE053
; wherein, the multi-camera system includes: a camera for capturing a first view or a second view
Figure 766636DEST_PATH_IMAGE054
, and a perspective camera;

采用Plücker向量对所述多像机系统中平移向量进行参数化，得到平移向量为：Using the Plücker vector to parameterize the translation vector in the multi-camera system, the translation vector obtained is:

Figure 994659DEST_PATH_IMAGE055
Figure 994659DEST_PATH_IMAGE055

其中，

Figure 613859DEST_PATH_IMAGE056
是像机的序列号，
Figure 552996DEST_PATH_IMAGE057
是仿射匹配点对的序列号，
Figure 17476DEST_PATH_IMAGE058
是视图的序列号。单位方向向量
Figure 420775DEST_PATH_IMAGE059
可以通过
Figure 601090DEST_PATH_IMAGE060
计算得出，
Figure 824261DEST_PATH_IMAGE061
，
Figure 623589DEST_PATH_IMAGE062
是像机
Figure 553499DEST_PATH_IMAGE063
中
Figure 452185DEST_PATH_IMAGE064
对应的归一化齐次图像坐标；in,
Figure 613859DEST_PATH_IMAGE056
is the serial number of the camera,
Figure 552996DEST_PATH_IMAGE057
is the sequence number of the affine matching point pair,
Figure 17476DEST_PATH_IMAGE058
is the serial number of the view. unit direction vector
Figure 420775DEST_PATH_IMAGE059
able to pass
Figure 601090DEST_PATH_IMAGE060
Calculated,
Figure 824261DEST_PATH_IMAGE061
,
Figure 623589DEST_PATH_IMAGE062
camera
Figure 553499DEST_PATH_IMAGE063
middle
Figure 452185DEST_PATH_IMAGE064
the corresponding normalized homogeneous image coordinates;

根据参数化后的旋转矩阵和平移向量，得到第一视图和第二视图对应的两个透视相机之间的第四位姿关系为：According to the parameterized rotation matrix and translation vector, the fourth pose relationship between the two perspective cameras corresponding to the first view and the second view is obtained as:

Figure 490548DEST_PATH_IMAGE065
Figure 490548DEST_PATH_IMAGE065

计算第四位姿关系对应的本质矩阵为：The essential matrix corresponding to the calculation of the fourth pose relationship is:

Figure 765672DEST_PATH_IMAGE066
。
Figure 765672DEST_PATH_IMAGE066
.

在其中一个实施例中，根据所述旋转参数约束，确定多像机系统的相对运动参数为五个自由度；In one of the embodiments, according to the rotation parameter constraint, the relative motion parameter of the multi-camera system is determined to be five degrees of freedom;

选择所述第j个所述仿射匹配点对应的两个仿射变换约束，以及其他所述仿射匹配点确定的一个对极几何约束和一个仿射变换约束，构建第三求解模型为：Select two affine transformation constraints corresponding to the jth affine matching point, as well as an epipolar geometric constraint and an affine transformation constraint determined by the other affine matching points, and construct a third solution model as follows:

Figure 799356DEST_PATH_IMAGE067
Figure 799356DEST_PATH_IMAGE067

选择其他所述仿射匹配点建立世界参考系，得到第四求解模型为：Select other affine matching points to establish the world reference system, and obtain the fourth solution model as:

Figure 196839DEST_PATH_IMAGE068
Figure 196839DEST_PATH_IMAGE068

根据所述第三求解模型和所述第四求解模型，得到关于未知数

Figure 66706DEST_PATH_IMAGE069
的八个方程为：According to the third solution model and the fourth solution model, it is obtained about the unknown
Figure 66706DEST_PATH_IMAGE069
The eight equations are:

Figure 942258DEST_PATH_IMAGE070
Figure 942258DEST_PATH_IMAGE070

通过Gröbner基求解法获得八个方程的代数解，根据所述代数解确定多像机系统第一位姿关系的旋转矩阵R，根据

Figure 96028DEST_PATH_IMAGE071
中零空间确定
Figure 336516DEST_PATH_IMAGE072
，根据
Figure 83892DEST_PATH_IMAGE073
计算得到第二位姿关系中平移向量和第三位姿关系中平移向量。The algebraic solutions of the eight equations are obtained by the Gröbner basis solution method, and the rotation matrix R of the first attitude relationship of the multi-camera system is determined according to the algebraic solutions, according to
Figure 96028DEST_PATH_IMAGE071
Medium null space determination
Figure 336516DEST_PATH_IMAGE072
,according to
Figure 83892DEST_PATH_IMAGE073
The translation vector in the second pose relationship and the translation vector in the third pose relationship are obtained by calculation.

以下分别以单像机和多相机两种情况进行进一步说明。The following are further descriptions in the case of a single camera and a multi-camera.

单像机Mono camera

选择任意一个仿射匹配点对来定义世界参考系W，如图2所示。假设当前选择了第j个仿射匹配点对，选取第j个仿射匹配点对在三维空间中的位置作为W的原点，且W的坐标轴方向与视图1（图2中View1）一致。将视图1和视图2（图2中View2）之间的位姿关系表示为

Figure 638502DEST_PATH_IMAGE074
，将视图1和参考系W之间的位姿关系表示为
Figure 725406DEST_PATH_IMAGE075
，视图2和参考系W之间的位姿关系表示为
Figure 199113DEST_PATH_IMAGE076
。特别地，
Figure 292840DEST_PATH_IMAGE077
，
Figure 713457DEST_PATH_IMAGE078
。使用Cayley参数化来表示旋转矩阵
Figure 717185DEST_PATH_IMAGE079
，可以表示为：Choose any affine matching point pair to define the world reference frame W, as shown in Figure 2. Assuming that the jth affine matching point pair is currently selected, the position of the jth affine matching point pair in the three-dimensional space is selected as the origin of W, and the coordinate axis direction of W is consistent with view 1 (View1 in Figure 2). The pose relationship between View 1 and View 2 (View2 in Figure 2) is expressed as
Figure 638502DEST_PATH_IMAGE074
, the pose relationship between view 1 and reference frame W is expressed as
Figure 725406DEST_PATH_IMAGE075
, the pose relationship between view 2 and reference frame W is expressed as
Figure 199113DEST_PATH_IMAGE076
. Particularly,
Figure 292840DEST_PATH_IMAGE077
,
Figure 713457DEST_PATH_IMAGE078
. Use Cayley parameterization to represent rotation matrices
Figure 717185DEST_PATH_IMAGE079
,It can be expressed as:

Figure 237159DEST_PATH_IMAGE080
Figure 896811DEST_PATH_IMAGE081
Figure 237159DEST_PATH_IMAGE080
Figure 896811DEST_PATH_IMAGE081

其中

Figure 652277DEST_PATH_IMAGE082
是四元数齐次向量。假设视图1和视图2之间的相对旋转角度是已知的。值得注意的是，相对旋转角度即使在惯性测量单元与像机之间的安装关系未知或存在变化的情况下，它也可以直接由惯性测量单元提供。三个未知数
Figure 900725DEST_PATH_IMAGE083
满足以下约束：in
Figure 652277DEST_PATH_IMAGE082
is a quaternion homogeneous vector. It is assumed that the relative rotation angle between view 1 and view 2 is known. It is worth noting that the relative rotation angle can be directly provided by the inertial measurement unit even when the installation relationship between the inertial measurement unit and the camera is unknown or there is variation. three unknowns
Figure 900725DEST_PATH_IMAGE083
Satisfy the following constraints:

Figure 716234DEST_PATH_IMAGE084
Figure 597602DEST_PATH_IMAGE085
Figure 716234DEST_PATH_IMAGE084
Figure 597602DEST_PATH_IMAGE085

其中

Figure 297705DEST_PATH_IMAGE086
是两个视图之间的相对旋转角度。in
Figure 297705DEST_PATH_IMAGE086
is the relative rotation angle between the two views.

接下来，将

Figure 213708DEST_PATH_IMAGE087
和
Figure 465698DEST_PATH_IMAGE088
参数化为两个未知深度参数
Figure 21313DEST_PATH_IMAGE089
的线性函数：Next, will
Figure 213708DEST_PATH_IMAGE087
and
Figure 465698DEST_PATH_IMAGE088
parameterized as two unknown depth parameters
Figure 21313DEST_PATH_IMAGE089
The linear function of :

Figure 321845DEST_PATH_IMAGE090
Figure 154671DEST_PATH_IMAGE091
Figure 321845DEST_PATH_IMAGE090
Figure 154671DEST_PATH_IMAGE091

两个视图之间的相对运动由两个变换的组合确定：（i）从视图1到W，（ii）从W到视图2。未知数

Figure 515246DEST_PATH_IMAGE092
，
Figure 43310DEST_PATH_IMAGE093
和
Figure 944270DEST_PATH_IMAGE094
被参数化为
Figure 21816DEST_PATH_IMAGE095
。形式上，相对运动
Figure 350029DEST_PATH_IMAGE096
表示为：The relative motion between the two views is determined by the combination of two transformations: (i) from view 1 to W, (ii) from W to view 2. unknown
Figure 515246DEST_PATH_IMAGE092
,
Figure 43310DEST_PATH_IMAGE093
and
Figure 944270DEST_PATH_IMAGE094
is parameterized as
Figure 21816DEST_PATH_IMAGE095
. Formally, relative motion
Figure 350029DEST_PATH_IMAGE096
Expressed as:

Figure 693286DEST_PATH_IMAGE097
Figure 7724DEST_PATH_IMAGE098
Figure 693286DEST_PATH_IMAGE097
Figure 7724DEST_PATH_IMAGE098

本质矩阵可以表示为：The essential matrix can be expressed as:

Figure 815143DEST_PATH_IMAGE099
Figure 517520DEST_PATH_IMAGE100
Figure 815143DEST_PATH_IMAGE099
Figure 517520DEST_PATH_IMAGE100

通过将式

Figure 535023DEST_PATH_IMAGE101
带入式
Figure 777786DEST_PATH_IMAGE102
可知，本质矩阵中的每项元素都与
Figure 642973DEST_PATH_IMAGE103
线性相关。through the formula
Figure 535023DEST_PATH_IMAGE101
Bring in
Figure 777786DEST_PATH_IMAGE102
It can be seen that each element in the essential matrix is related to
Figure 642973DEST_PATH_IMAGE103
Linear correlation.

一个仿射匹配点对能够为几何模型估计提取三个独立的约束，包括一个从同名点对关系导出的对极几何约束

Figure 453935DEST_PATH_IMAGE104
和两个从局部仿射变换矩阵
Figure 302942DEST_PATH_IMAGE105
导出的仿射变换约束。在像机内参已知的情况下，视图1和视图2之间的对极几何约束如下所示：An affine matched point pair can extract three independent constraints for geometric model estimation, including an epipolar geometric constraint derived from the homonymous point pair relationship
Figure 453935DEST_PATH_IMAGE104
and two from the local affine transformation matrix
Figure 302942DEST_PATH_IMAGE105
Exported affine transformation constraints. With the camera intrinsics known, the epipolar geometric constraints between view 1 and view 2 are as follows:

Figure 552658DEST_PATH_IMAGE106
Figure 459303DEST_PATH_IMAGE107
Figure 552658DEST_PATH_IMAGE106
Figure 459303DEST_PATH_IMAGE107

描述本质矩阵

Figure 300220DEST_PATH_IMAGE108
与局部仿射变换矩阵
Figure 839785DEST_PATH_IMAGE109
的关系的仿射变换约束可以表示如下：Description Essential Matrix
Figure 300220DEST_PATH_IMAGE108
with local affine transformation matrix
Figure 839785DEST_PATH_IMAGE109
The affine transformation constraints of the relation can be expressed as follows:

Figure 565296DEST_PATH_IMAGE110
Figure 936234DEST_PATH_IMAGE111
Figure 565296DEST_PATH_IMAGE110
Figure 936234DEST_PATH_IMAGE111

其中下标(1:2)表示前两个方程式。where the subscript (1:2) represents the first two equations.

由于已经选择一个仿射匹配点对作为世界参考系的原点来对平移向量进行特殊参数化，发现所选仿射匹配点对中的同名点对应关系不能贡献一个新的约束，因为所得方程的系数都为零。因此，当第j个仿射匹配点对被用于建立世界参考系W时，两个仿射匹配点对能够提供五个方程。具体而言，第j个仿射匹配点对基于式

Figure 151315DEST_PATH_IMAGE112
提供两个方程。另一个仿射匹配点对提供了基于式
Figure 630707DEST_PATH_IMAGE113
和式
Figure 956646DEST_PATH_IMAGE112
的三个方程。通过将式
Figure 713249DEST_PATH_IMAGE114
带入到式
Figure 771335DEST_PATH_IMAGE113
和式
Figure 285493DEST_PATH_IMAGE112
中并使用隐变量方法，可以将两个仿射匹配点对提供的五个方程写为：Since an affine matched point pair has been chosen as the origin of the world reference frame to specifically parameterize the translation vector, it is found that the homonymic point correspondence in the chosen affine matched point pair cannot contribute a new constraint because the coefficients of the resulting equation are all zero. Therefore, when the jth affine matched point pair is used to establish the world reference frame W, two affine matched point pairs can provide five equations. Specifically, the jth affine matching point pair is based on the formula
Figure 151315DEST_PATH_IMAGE112
Two equations are provided. Another affine matching point pair provides a
Figure 630707DEST_PATH_IMAGE113
Japanese
Figure 956646DEST_PATH_IMAGE112
the three equations. through the formula
Figure 713249DEST_PATH_IMAGE114
bring-in
Figure 771335DEST_PATH_IMAGE113
Japanese
Figure 285493DEST_PATH_IMAGE112
and using the latent variable approach, the five equations provided by the two affine matched point pairs can be written as:

Figure 477440DEST_PATH_IMAGE115
Figure 947605DEST_PATH_IMAGE116
Figure 477440DEST_PATH_IMAGE115
Figure 947605DEST_PATH_IMAGE116

其中，

Figure 504488DEST_PATH_IMAGE117
中的元素项为未知数
Figure 568259DEST_PATH_IMAGE118
，
Figure 173684DEST_PATH_IMAGE119
和
Figure 576983DEST_PATH_IMAGE120
的二次项。in,
Figure 504488DEST_PATH_IMAGE117
The element term in is unknown
Figure 568259DEST_PATH_IMAGE118
,
Figure 173684DEST_PATH_IMAGE119
and
Figure 576983DEST_PATH_IMAGE120
the quadratic term.

在通过惯性测量单元获得两个像机之间的相对旋转角度后，单像机的相对运动估计问题为四个自由度。但是，两个仿射匹配点对可以提供六个独立的约束。这意味着约束的数量大于未知数的数量，并且存在冗余的约束。因此，最少一个仿射匹配点对和一个同名点对就足以估计已知相对旋转角度条件下的单像机相对运动。可以从式

Figure 367085DEST_PATH_IMAGE121
中任选三个方程来探索最小解的情况。更具体地说，将第j个仿射匹配点对的两个仿射变换约束和另一个仿射匹配点对的一个对极几何约束进行联立组合，得到具有5个未知数的3个方程，即式
Figure 777206DEST_PATH_IMAGE121
的前三个方程：After the relative rotation angle between the two cameras is obtained by the inertial measurement unit, the relative motion estimation problem of a single camera is four degrees of freedom. However, two affine matched point pairs can provide six independent constraints. This means that the number of constraints is greater than the number of unknowns, and there are redundant constraints. Therefore, at least one affine matching point pair and one point pair with the same name is sufficient to estimate the relative motion of the single camera under the condition of known relative rotation angle. can be from the formula
Figure 367085DEST_PATH_IMAGE121
Choose three of the equations to explore the case of the minimum solution. More specifically, the simultaneous combination of two affine transformation constraints for the jth affine matching point pair and one epipolar geometric constraint for the other affine matching point pair yields 3 equations with 5 unknowns, Instant
Figure 777206DEST_PATH_IMAGE121
The first three equations of :

Figure 514218DEST_PATH_IMAGE122
Figure 834341DEST_PATH_IMAGE123
Figure 514218DEST_PATH_IMAGE122
Figure 834341DEST_PATH_IMAGE123

由于式

Figure 405131DEST_PATH_IMAGE124
具有非零解，因此
Figure 381177DEST_PATH_IMAGE125
的秩满足
Figure 718617DEST_PATH_IMAGE126
。因此，
Figure 752301DEST_PATH_IMAGE125
的所有2×2子行列式必须为零。这给出了关于三个未知数
Figure 87468DEST_PATH_IMAGE127
的三个方程。due to the formula
Figure 405131DEST_PATH_IMAGE124
has a nonzero solution, so
Figure 381177DEST_PATH_IMAGE125
rank satisfaction
Figure 718617DEST_PATH_IMAGE126
. therefore,
Figure 752301DEST_PATH_IMAGE125
All 2x2 sub-determinants of must be zero. This gives about three unknowns
Figure 87468DEST_PATH_IMAGE127
the three equations.

总而言之，假设选择第j个仿射匹配点对来建立世界基准系W。由于在最小解情况下需要有两个仿射匹配点对，所以也可以选择另外一个仿射匹配点对来建立世界参考系W。假设选择了第j'个AC，就可以得到一个类似于式

Figure 347548DEST_PATH_IMAGE124
的方程组：To summarize, suppose that the jth affine matching point pair is chosen to establish the world reference frame W. Since two affine matching point pairs are required in the case of the minimum solution, another affine matching point pair can also be selected to establish the world reference frame W. Assuming that the j'th AC is selected, an equation similar to
Figure 347548DEST_PATH_IMAGE124
system of equations:

Figure 426362DEST_PATH_IMAGE128
Figure 65285DEST_PATH_IMAGE129
Figure 426362DEST_PATH_IMAGE128
Figure 65285DEST_PATH_IMAGE129

将式

Figure 633670DEST_PATH_IMAGE124
和式
Figure 318729DEST_PATH_IMAGE130
进行联立，就可以得到关于三个未知数
Figure 122606DEST_PATH_IMAGE131
的六个方程；general
Figure 633670DEST_PATH_IMAGE124
Japanese
Figure 318729DEST_PATH_IMAGE130
Simultaneously, we can get about the three unknowns
Figure 122606DEST_PATH_IMAGE131
the six equations;

Figure 6248DEST_PATH_IMAGE132
Figure 683217DEST_PATH_IMAGE133
Figure 6248DEST_PATH_IMAGE132
Figure 683217DEST_PATH_IMAGE133

上式为关于

Figure 793256DEST_PATH_IMAGE134
的四次方程组。The above formula is about
Figure 793256DEST_PATH_IMAGE134
of the quadratic equations.

对于式

Figure 213873DEST_PATH_IMAGE135
和
Figure 217601DEST_PATH_IMAGE136
组成的多项式方程组，可通过Gröbner基方法获得代数解。为了保持数值稳定性并避免在Gröbner基的计算过程中进行大量运算，在有限场
Figure 986843DEST_PATH_IMAGE137
中构造了多项式方程组的随机实例。然后，使用计算机代数系统Macaulay 2来计算Gröbner基。最后，使用自动Gröbner基求解算法找到相应的解。For the formula
Figure 213873DEST_PATH_IMAGE135
and
Figure 217601DEST_PATH_IMAGE136
consists of a system of polynomial equations, which can be solved algebraically by the Gröbner basis method. In order to maintain numerical stability and avoid extensive computations during the computation of Gröbner basis, the finite field
Figure 986843DEST_PATH_IMAGE137
A random instance of a system of polynomial equations is constructed in . The Gröbner basis is then calculated using the computer algebra system Macaulay 2. Finally, the corresponding solution is found using the automatic Gröbner basis solving algorithm.

上述求解方法最多有20个复数解和尺寸为36×56的消除模板。一旦获得了旋转参数

Figure 646494DEST_PATH_IMAGE138
，则立即使用式
Figure 667540DEST_PATH_IMAGE139
获得
Figure 729037DEST_PATH_IMAGE140
。然后，利用式
Figure 419912DEST_PATH_IMAGE124
，通过找到
Figure 566860DEST_PATH_IMAGE141
的零空间来确定
Figure 391596DEST_PATH_IMAGE142
。接下来，可以通过式
Figure 228971DEST_PATH_IMAGE143
计算
Figure 418644DEST_PATH_IMAGE144
和
Figure 849625DEST_PATH_IMAGE145
。最后，根据式
Figure 150156DEST_PATH_IMAGE146
计算单像机的相对运动。The above solution method has up to 20 complex solutions and elimination templates of size 36×56. Once the rotation parameters are obtained
Figure 646494DEST_PATH_IMAGE138
, the immediate use
Figure 667540DEST_PATH_IMAGE139
get
Figure 729037DEST_PATH_IMAGE140
. Then, using the formula
Figure 419912DEST_PATH_IMAGE124
, by finding
Figure 566860DEST_PATH_IMAGE141
the null space to determine
Figure 391596DEST_PATH_IMAGE142
. Next, it is possible to pass the formula
Figure 228971DEST_PATH_IMAGE143
calculate
Figure 418644DEST_PATH_IMAGE144
and
Figure 849625DEST_PATH_IMAGE145
. Finally, according to the formula
Figure 150156DEST_PATH_IMAGE146
Calculate the relative motion of the single camera.

多像机multi camera

选择第j个仿射匹配点对来定义世界参考系W，如图3所示。选取第j个仿射匹配点对在三维空间中的位置作为W的原点，且W的坐标轴方向与视图1（图3中View1）一致。在多像机系统的参考中将

Figure 123929DEST_PATH_IMAGE147
的外参数表示为
Figure 281241DEST_PATH_IMAGE148
。将视图1和参考系W之间的转换表示为
Figure 137201DEST_PATH_IMAGE149
，将视图2（图3中View1）和参考系W之间的转换表示为
Figure 428374DEST_PATH_IMAGE150
。注意，
Figure 318970DEST_PATH_IMAGE151
，
Figure 912762DEST_PATH_IMAGE152
。接下来，对
Figure 928123DEST_PATH_IMAGE153
和
Figure 570457DEST_PATH_IMAGE154
进行参数化。可以将由Plücker向量
Figure 377876DEST_PATH_IMAGE155
描述的线上的所有点参数化为：The jth affine matching point pair is selected to define the world reference frame W, as shown in Figure 3. The position of the jth affine matching point pair in three-dimensional space is selected as the origin of W, and the coordinate axis direction of W is consistent with view 1 (View1 in Figure 3). In the reference to the multi-camera system will be
Figure 123929DEST_PATH_IMAGE147
The external parameters are expressed as
Figure 281241DEST_PATH_IMAGE148
. Denote the transformation between view 1 and reference frame W as
Figure 137201DEST_PATH_IMAGE149
, the transformation between View 2 (View1 in Figure 3) and the reference frame W is expressed as
Figure 428374DEST_PATH_IMAGE150
. Notice,
Figure 318970DEST_PATH_IMAGE151
,
Figure 912762DEST_PATH_IMAGE152
. Next, yes
Figure 928123DEST_PATH_IMAGE153
and
Figure 570457DEST_PATH_IMAGE154
to parameterize. can be converted by the Plücker vector
Figure 377876DEST_PATH_IMAGE155
All points on the described line are parameterized as:

Figure 80252DEST_PATH_IMAGE156
Figure 97756DEST_PATH_IMAGE157
Figure 80252DEST_PATH_IMAGE156
Figure 97756DEST_PATH_IMAGE157

其中

Figure 543781DEST_PATH_IMAGE158
是单位方向矢量，
Figure 205706DEST_PATH_IMAGE159
是矩矢量，
Figure 751088DEST_PATH_IMAGE160
是未知深度参数。in
Figure 543781DEST_PATH_IMAGE158
is the unit direction vector,
Figure 205706DEST_PATH_IMAGE159
is the moment vector,
Figure 751088DEST_PATH_IMAGE160
is the unknown depth parameter.

假设选择第j个仿射匹配点对对应的三维空间位置

Figure 68937DEST_PATH_IMAGE161
来定义世界参考W的原点。将连接
Figure 380970DEST_PATH_IMAGE162
和摄像机
Figure 835085DEST_PATH_IMAGE163
的光心的Plücker线表示为
Figure 66215DEST_PATH_IMAGE164
。则在视图k中，点
Figure 668097DEST_PATH_IMAGE165
满足：Assume that the 3D space position corresponding to the jth affine matching point pair is selected
Figure 68937DEST_PATH_IMAGE161
to define the origin of the world reference W. will connect
Figure 380970DEST_PATH_IMAGE162
and camera
Figure 835085DEST_PATH_IMAGE163
The Plücker line of the optical center is expressed as
Figure 66215DEST_PATH_IMAGE164
. Then in view k, point
Figure 668097DEST_PATH_IMAGE165
Satisfy:

Figure 455925DEST_PATH_IMAGE166
Figure 967809DEST_PATH_IMAGE167
Figure 455925DEST_PATH_IMAGE166
Figure 967809DEST_PATH_IMAGE167

等效地，也可以表述为：Equivalently, it can also be expressed as:

Figure 182889DEST_PATH_IMAGE168
Figure 537647DEST_PATH_IMAGE169
Figure 182889DEST_PATH_IMAGE168
Figure 537647DEST_PATH_IMAGE169

其中，

Figure 784958DEST_PATH_IMAGE170
是像机的序列号，
Figure 213665DEST_PATH_IMAGE171
是仿射匹配点对的序列号，
Figure 927543DEST_PATH_IMAGE172
是视图的序列号。单位方向向量
Figure 441701DEST_PATH_IMAGE173
可以通过
Figure 243435DEST_PATH_IMAGE174
计算得出，其中
Figure 854545DEST_PATH_IMAGE175
是像机
Figure 411428DEST_PATH_IMAGE176
中
Figure 599833DEST_PATH_IMAGE177
对应的归一化齐次图像坐标。在这里，将
Figure 533154DEST_PATH_IMAGE178
和
Figure 998770DEST_PATH_IMAGE179
参数化为两个未知深度参数
Figure 398659DEST_PATH_IMAGE180
的线性函数。in,
Figure 784958DEST_PATH_IMAGE170
is the serial number of the camera,
Figure 213665DEST_PATH_IMAGE171
is the sequence number of the affine matching point pair,
Figure 927543DEST_PATH_IMAGE172
is the serial number of the view. unit direction vector
Figure 441701DEST_PATH_IMAGE173
able to pass
Figure 243435DEST_PATH_IMAGE174
calculated, where
Figure 854545DEST_PATH_IMAGE175
camera
Figure 411428DEST_PATH_IMAGE176
middle
Figure 599833DEST_PATH_IMAGE177
The corresponding normalized homogeneous image coordinates. Here, will
Figure 533154DEST_PATH_IMAGE178
and
Figure 998770DEST_PATH_IMAGE179
parameterized as two unknown depth parameters
Figure 398659DEST_PATH_IMAGE180
the linear function of .

每个仿射匹配点对关联着视图1和视图2中两个透视像机。两个像机之间的相对运动

Figure 887409DEST_PATH_IMAGE181
由四个变换的组合确定：（i）从一个透视像机到视图1，（ii）从视图1 到W，（iii）从W到视图2，（iv）从视图2到另一台透视像机。在这四个转换中，（i）和（iv）部分由已知的外参确定。在（ii）和（iii）部分中，存在未知数
Figure 421159DEST_PATH_IMAGE182
，
Figure 678965DEST_PATH_IMAGE183
和
Figure 764601DEST_PATH_IMAGE184
，它们被参数化为
Figure 802964DEST_PATH_IMAGE185
。相对运动
Figure 812509DEST_PATH_IMAGE186
可以表示为：Each affine matching point pair is associated with two perspective cameras in view 1 and view 2. Relative motion between two cameras
Figure 887409DEST_PATH_IMAGE181
Determined by a combination of four transformations: (i) from one perspective camera to view1, (ii) from view1 to W, (iii) from W to view2, (iv) from view2 to another perspective machine. Of these four transformations, parts (i) and (iv) are determined by known extrinsic parameters. In parts (ii) and (iii) there are unknowns
Figure 421159DEST_PATH_IMAGE182
,
Figure 678965DEST_PATH_IMAGE183
and
Figure 764601DEST_PATH_IMAGE184
, they are parameterized as
Figure 802964DEST_PATH_IMAGE185
. relative motion
Figure 812509DEST_PATH_IMAGE186
It can be expressed as:

Figure 128083DEST_PATH_IMAGE187
Figure 994408DEST_PATH_IMAGE188
Figure 128083DEST_PATH_IMAGE187
Figure 994408DEST_PATH_IMAGE188

每个仿射匹配点对中两个透视像机之间相对运动

Figure 457751DEST_PATH_IMAGE189
被表示出来后，本质矩阵
Figure 457937DEST_PATH_IMAGE190
可以表示为：The relative motion between the two perspective cameras in each affine matched point pair
Figure 457751DEST_PATH_IMAGE189
After being represented, the essential matrix
Figure 457937DEST_PATH_IMAGE190
It can be expressed as:

Figure 690335DEST_PATH_IMAGE191
Figure 993140DEST_PATH_IMAGE192
Figure 690335DEST_PATH_IMAGE191
Figure 993140DEST_PATH_IMAGE192

通过将式

Figure 615882DEST_PATH_IMAGE193
代入式
Figure 232809DEST_PATH_IMAGE194
，本质矩阵中的元素项都与
Figure 382030DEST_PATH_IMAGE195
呈线性关系。然后，将式
Figure 980371DEST_PATH_IMAGE194
带入到式
Figure 152726DEST_PATH_IMAGE196
和式
Figure 370081DEST_PATH_IMAGE197
，可以从两个仿射匹配点对中获得五个方程，这些方程来自第j个仿射匹配点对的两个仿射变换约束和另一个仿射匹配点对的三个方程组成。这些方程可以表示为：through the formula
Figure 615882DEST_PATH_IMAGE193
Substitute
Figure 232809DEST_PATH_IMAGE194
, the element entries in the essential matrix are all the same as
Figure 382030DEST_PATH_IMAGE195
a linear relationship. Then, the formula
Figure 980371DEST_PATH_IMAGE194
bring-in
Figure 152726DEST_PATH_IMAGE196
Japanese
Figure 370081DEST_PATH_IMAGE197
, five equations can be obtained from the two affine matched point pairs consisting of two affine transformation constraints for the jth affine matched point pair and three equations for the other affine matched point pair. These equations can be expressed as:

Figure 577071DEST_PATH_IMAGE198
Figure 97045DEST_PATH_IMAGE199
Figure 577071DEST_PATH_IMAGE198
Figure 97045DEST_PATH_IMAGE199

Figure 819014DEST_PATH_IMAGE200
中的元素项均为未知数
Figure 777742DEST_PATH_IMAGE030
，
Figure 760611DEST_PATH_IMAGE201
，和
Figure 576120DEST_PATH_IMAGE202
的二次项。
Figure 819014DEST_PATH_IMAGE200
The elements in are all unknowns
Figure 777742DEST_PATH_IMAGE030
,
Figure 760611DEST_PATH_IMAGE201
,and
Figure 576120DEST_PATH_IMAGE202
the quadratic term.

在通过惯性测量单元获得两个多像机系统之间的相对旋转角度后，多像机的相对运动估计问题为五个自由度。考虑到两个仿射匹配点对提供六个独立的约束，约束的数量大于未知数，并且存在冗余约束。因此，从式

Figure 723068DEST_PATH_IMAGE203
中随机选择四个方程来探索最小解的情况。例如，第j个仿射匹配点对的两个仿射变换约束以及另一个仿射匹配点对的一个对极几何约束和第一个仿射变换约束联立成含有五个未知数中的四个方程，即式
Figure 423170DEST_PATH_IMAGE204
的前四个方程：After the relative rotation angle between the two multi-camera systems is obtained by the inertial measurement unit, the relative motion estimation problem of the multi-camera is five degrees of freedom. Considering that two affine matched point pairs provide six independent constraints, the number of constraints is greater than the unknown, and there are redundant constraints. Therefore, from the formula
Figure 723068DEST_PATH_IMAGE203
In the case where four equations are randomly selected to explore the minimum solution. For example, two affine transformation constraints for the jth affine matching point pair and an epipolar geometric constraint and the first affine transformation constraint for another affine matching point pair are simultaneously combined into four equations with five unknowns , i.e.
Figure 423170DEST_PATH_IMAGE204
The first four equations of :

Figure 135911DEST_PATH_IMAGE205
Figure 60005DEST_PATH_IMAGE206
Figure 135911DEST_PATH_IMAGE205
Figure 60005DEST_PATH_IMAGE206

由于式

Figure 881199DEST_PATH_IMAGE207
具有非零解，因此
Figure 978468DEST_PATH_IMAGE208
的秩满足
Figure 14558DEST_PATH_IMAGE209
。因此，
Figure 47236DEST_PATH_IMAGE210
的所有3×3子行列式必须为零。这给出了关于三个未知数
Figure 965513DEST_PATH_IMAGE211
的四个方程。due to the formula
Figure 881199DEST_PATH_IMAGE207
has a nonzero solution, so
Figure 978468DEST_PATH_IMAGE208
rank satisfaction
Figure 14558DEST_PATH_IMAGE209
. therefore,
Figure 47236DEST_PATH_IMAGE210
All 3x3 sub-determinants of must be zero. This gives about three unknowns
Figure 965513DEST_PATH_IMAGE211
of the four equations.

同样，可以选择另一个仿射匹配点对来建立世界参考系W。假设选择了第

Figure 69735DEST_PATH_IMAGE212
个仿射匹配点对，可以建立一个类似于式
Figure 881702DEST_PATH_IMAGE213
的方程组：Likewise, another affine matching point pair can be chosen to establish the world reference frame W. Suppose you choose the
Figure 69735DEST_PATH_IMAGE212
affine matching point pairs, it is possible to establish a formula similar to
Figure 881702DEST_PATH_IMAGE213
system of equations:

Figure 413178DEST_PATH_IMAGE214
Figure 818751DEST_PATH_IMAGE215
Figure 413178DEST_PATH_IMAGE214
Figure 818751DEST_PATH_IMAGE215

将式

Figure 398768DEST_PATH_IMAGE213
和式
Figure 143871DEST_PATH_IMAGE216
进行联立，可以得到关于三个未知数
Figure 908564DEST_PATH_IMAGE217
的八个方程；general
Figure 398768DEST_PATH_IMAGE213
Japanese
Figure 143871DEST_PATH_IMAGE216
Simultaneously, we can get about the three unknowns
Figure 908564DEST_PATH_IMAGE217
the eight equations;

Figure 739117DEST_PATH_IMAGE218
Figure 372092DEST_PATH_IMAGE219
Figure 739117DEST_PATH_IMAGE218
Figure 372092DEST_PATH_IMAGE219

这些方程的次数为6。此外，在刚刚的问题中发现了一个额外的约束，即

Figure 971701DEST_PATH_IMAGE220
的秩为1。The degree of these equations is 6. Furthermore, an additional constraint was found in the question just now, namely
Figure 971701DEST_PATH_IMAGE220
rank is 1.

仿射变换约束为上述问题提供了额外的方程。只有当使用第j个仿射匹配点对来建立世界参考系W时，第j个仿射匹配点对的两个仿射变换约束才用于构造额外方程。因此，对于多像机的相对运动估计问题，存在三个额外的方程：Affine transformation constraints provide additional equations for the above problem. The two affine transformation constraints of the jth affine matching point pair are used to construct additional equations only when the jth affine matching point pair is used to establish the world reference frame W. Therefore, there are three additional equations for the relative motion estimation problem of multi-cameras:

Figure 641717DEST_PATH_IMAGE221
Figure 897249DEST_PATH_IMAGE222
Figure 641717DEST_PATH_IMAGE221
Figure 897249DEST_PATH_IMAGE222

上式为关于

Figure 881385DEST_PATH_IMAGE223
的四次方程组。The above formula is about
Figure 881385DEST_PATH_IMAGE223
of the quadratic equations.

使用Gröbner基方法进行求解。将式

Figure 663397DEST_PATH_IMAGE224
和式
Figure 707576DEST_PATH_IMAGE225
中的这些多项式方程分别表示为约束
Figure 699672DEST_PATH_IMAGE226
和约束
Figure 284237DEST_PATH_IMAGE227
。单独使用约束
Figure 858437DEST_PATH_IMAGE226
可用于非交叉或交叉仿射匹配点对条件下的相对运动估计。但是，同时使用
Figure 11201DEST_PATH_IMAGE226
和
Figure 38063DEST_PATH_IMAGE227
可以减少可能的解的数量。Solve using the Gröbner basis method. general
Figure 663397DEST_PATH_IMAGE224
Japanese
Figure 707576DEST_PATH_IMAGE225
These polynomial equations in are represented as constraints
Figure 699672DEST_PATH_IMAGE226
and constraints
Figure 284237DEST_PATH_IMAGE227
. Use constraints alone
Figure 858437DEST_PATH_IMAGE226
Can be used for relative motion estimation under the condition of non-cross or cross-affine matched point pairs. However, using both
Figure 11201DEST_PATH_IMAGE226
and
Figure 38063DEST_PATH_IMAGE227
The number of possible solutions can be reduced.

一旦获得了旋转参数

Figure 426319DEST_PATH_IMAGE228
，就可以立即计算得
Figure 41977DEST_PATH_IMAGE229
。然后利用式
Figure 427959DEST_PATH_IMAGE230
，通过找到
Figure 270013DEST_PATH_IMAGE231
的零空间来确定
Figure 399643DEST_PATH_IMAGE232
。接下来，可以通过式
Figure 620540DEST_PATH_IMAGE233
计算
Figure 239740DEST_PATH_IMAGE234
和
Figure 241194DEST_PATH_IMAGE235
。最后，通过组合变换
Figure 361466DEST_PATH_IMAGE236
和
Figure 499186DEST_PATH_IMAGE237
来计算多像机的相对运动。Once the rotation parameters are obtained
Figure 426319DEST_PATH_IMAGE228
, it can be calculated immediately
Figure 41977DEST_PATH_IMAGE229
. Then use the formula
Figure 427959DEST_PATH_IMAGE230
, by finding
Figure 270013DEST_PATH_IMAGE231
the null space to determine
Figure 399643DEST_PATH_IMAGE232
. Next, it is possible to pass the formula
Figure 620540DEST_PATH_IMAGE233
calculate
Figure 239740DEST_PATH_IMAGE234
and
Figure 241194DEST_PATH_IMAGE235
. Finally, by combining transformations
Figure 361466DEST_PATH_IMAGE236
and
Figure 499186DEST_PATH_IMAGE237
to calculate the relative motion of the multi-camera.

本发明可以达到以下的技术效果：The present invention can achieve the following technical effects:

1)本发明针对惯性测量单元直接提供相对旋转角度的条件下，充分利用视图之间的仿射匹配点对信息，极大减少求解单像机和多像机系统相对运动估计问题所需要的点对数量，明显提高了算法的精度及鲁棒性。1) Under the condition that the inertial measurement unit directly provides the relative rotation angle, the present invention makes full use of the affine matching point pair information between the views, and greatly reduces the points required to solve the relative motion estimation problem of the single-camera and multi-camera systems For the number of pairs, the accuracy and robustness of the algorithm are significantly improved.

2)本发明利用惯性测量单元与像机固联安装条件下，惯性测量单元和像机在运动过程中各自的相对旋转角相同这一性质，惯性测量单元输出的相对旋转角可以直接用作像机或多像机的相对旋转角，可应用于惯性测量单元与像机之间的安装关系未知或存在变化等更广泛的场景。2) The present invention utilizes the property that the inertial measurement unit and the camera have the same relative rotation angle during the motion process under the fixed installation condition of the inertial measurement unit and the camera, and the relative rotation angle output by the inertial measurement unit can be directly used as the image. The relative rotation angle of the camera or multi-camera can be applied to a wider range of scenarios where the installation relationship between the inertial measurement unit and the camera is unknown or there are changes.

3)针对惯性测量单元直接提供单像机相对旋转角度的条件下，单像机相对运动具有4个自由度。提出了一种新的单像机相对运动估计最小配置解求解方法，该求解方法通过2个仿射匹配点对可以准确估计出单像机的相对运动。3) Under the condition that the inertial measurement unit directly provides the relative rotation angle of the single camera, the relative motion of the single camera has 4 degrees of freedom. A new method for solving the minimum configuration solution for relative motion estimation of a single camera is proposed, which can accurately estimate the relative motion of a single camera through two affine matching point pairs.

4)针对惯性测量单元直接提供多像机相对旋转角度的条件下，多像机相对运动具有5个自由度。提出了一种新的多像机相对运动估计最小配置解求解方法，该求解方法通过2个交叉或非交叉仿射匹配点对可以准确估计多像机系统的相对运动。4) Under the condition that the inertial measurement unit directly provides the relative rotation angle of the multi-camera, the relative motion of the multi-camera has 5 degrees of freedom. A new method for solving the minimum configuration solution for multi-camera relative motion estimation is proposed, which can accurately estimate the relative motion of the multi-camera system through two cross or non-cross affine matching point pairs.

5)本发明的方法不需要对惯性测量单元与像机进行标定，使得惯性测量单元和像机的融合更加灵活方便，而且提出的方法具有更高的精度和效率，适用于计算能力有限的设备，例如无人机自主导航、自动驾驶汽车和增强现实设备等。5) The method of the present invention does not need to calibrate the inertial measurement unit and the camera, which makes the integration of the inertial measurement unit and the camera more flexible and convenient, and the proposed method has higher precision and efficiency, and is suitable for equipment with limited computing capabilities. , such as autonomous drone navigation, self-driving cars, and augmented reality devices.

本发明对于固联安装惯性测量单元和像机的手机、无人机等电子设备，利用惯性测量单元为像机提供相对旋转角度信息，利用二个仿射匹配点对估计单像机和多像机系统的相对运动过程如下：The invention uses the inertial measurement unit to provide relative rotation angle information for the camera, and uses two affine matching point pairs to estimate the single-camera and multi-image The relative motion process of the machine system is as follows:

1）通过ASIFT和MODS等算法提取像机相对运动中两视图之间的仿射匹配点对，包括同名点对图像坐标和相应领域信息之间的局部仿射矩阵；1) Extract the affine matching point pair between the two views in the relative motion of the camera through algorithms such as ASIFT and MODS, including the local affine matrix between the image coordinates of the point pair with the same name and the corresponding field information;

2）利用与像机固联安装的惯性测量单元直接输出的相对旋转角度信息；2) Use the relative rotation angle information directly output by the inertial measurement unit fixedly installed with the camera;

3）根据两视图之间提取的仿射匹配点对，惯性测量单元输出的相对旋转角度和本发明提出相对运动估计算法，求解单像机和多像机系统之间的相对运动。同时结合RANSAC框架剔除仿射匹配点对中的误匹配点对，并恢复像机的相对运动结果。3) According to the affine matching point pair extracted between the two views, the relative rotation angle output by the inertial measurement unit and the relative motion estimation algorithm proposed by the present invention, the relative motion between the single-camera and multi-camera systems is solved. At the same time, the RANSAC framework is combined to eliminate the incorrect matching point pairs in the affine matching point pairs, and restore the relative motion results of the camera.

以上实施例的各技术特征可以进行任意的组合，为使描述简洁，未对上述实施例中的各个技术特征所有可能的组合都进行描述，然而，只要这些技术特征的组合不存在矛盾，都应当认为是本说明书记载的范围。The technical features of the above embodiments can be combined arbitrarily. In order to make the description simple, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features It is considered to be the range described in this specification.

以上所述实施例仅表达了本申请的几种实施方式，其描述较为具体和详细，但并不能因此而理解为对发明专利范围的限制。应当指出的是，对于本领域的普通技术人员来说，在不脱离本申请构思的前提下，还可以做出若干变形和改进，这些都属于本申请的保护范围。因此，本申请专利的保护范围应以所附权利要求为准。The above-mentioned embodiments only represent several embodiments of the present application, and the descriptions thereof are specific and detailed, but should not be construed as a limitation on the scope of the invention patent. It should be pointed out that for those skilled in the art, without departing from the concept of the present application, several modifications and improvements can be made, which all belong to the protection scope of the present application. Therefore, the scope of protection of the patent of the present application shall be subject to the appended claims.

Claims (7)

1. A camera relative motion estimation method under a known relative rotation angle condition, the method comprising:
acquiring at least two affine matching point pairs in a first view and a second view shot by a camera, and selecting a jth affine matching point pair to establish a world reference system; the origin of the world reference system is the position of the jth affine matching point pair in the three-dimensional space, and the coordinate axis direction of the world reference system is consistent with the first view direction;
acquiring a first posture relation between the first view and the second view, acquiring a second posture relation between the first view and the world reference system, and acquiring a third posture relation between the second view and the world reference system; the first position relation, the second position relation and the third position relation comprise a rotation matrix and a translation vector;
parameterizing the rotation matrix and the translation vector, and determining rotation parameter constraint of unknown numbers corresponding to the rotation matrix according to a relative rotation angle between the first view and the second view;
representing the first position and posture relation by adopting the parameterized rotation matrix and translation vector and acquiring a corresponding essential matrix;
acquiring two affine transformation constraints corresponding to the essential matrix determined by the jth affine matching point and affine matching matrixes in the affine matching points, acquiring one epipolar geometric constraint of the first view and the second view determined by other affine matching points and two affine transformation constraints corresponding to the essential matrix and the affine matching matrixes in the affine matching points;
and solving to obtain the rotation matrix and the translation vector according to the two affine transformation constraints corresponding to the jth affine matching point and the epipolar geometric constraint and the two affine transformation constraints determined by the other affine matching points, and determining the relative motion relationship of the camera according to the rotation matrix and the translation vector.
2. The method of claim 1, wherein determining a rotation parameter constraint for the rotation matrix corresponding to the unknown number based on the relative rotation angle between the first view and the second view comprises:
obtaining the rotation matrix obtained by parameterizing the rotation matrix as follows:
Figure 402615DEST_PATH_IMAGE001
wherein,
Figure 726280DEST_PATH_IMAGE002
is a quaternion homogeneous vector, R represents a rotation matrix corresponding to the first attitude relationship;
according to the relative rotation angle between the first view and the second view, determining the rotation parameter constraint of the unknown number corresponding to the rotation matrix as follows:
Figure 240438DEST_PATH_IMAGE003
wherein,
Figure 432385DEST_PATH_IMAGE004
indicating the relative angle of rotation.
3. The method of claim 1, wherein the representing the first pose relationship using the parameterized rotation matrix and translation vector and obtaining a corresponding essential matrix comprises:
obtaining a translation vector in the second position posture relation and a translation vector in the third position posture relation obtained by parameterizing the translation vector, wherein the obtained translation vectors are respectively:
Figure 981178DEST_PATH_IMAGE005
wherein,
Figure 725012DEST_PATH_IMAGE006
representing a translation vector in the second attitude relationship,
Figure 523203DEST_PATH_IMAGE007
an unknown depth parameter representing a translation vector parameterization in the second pose relationship,
Figure 456524DEST_PATH_IMAGE008
a unit vector representing the normalized homogeneous image coordinates in the first view,
Figure 797507DEST_PATH_IMAGE009
representing the translation vector in the third posture relationship,
Figure 322029DEST_PATH_IMAGE010
an unknown depth parameter representing the translation vector in the third pose relationship,
Figure 810779DEST_PATH_IMAGE011
a unit vector representing the normalized homogeneous image coordinates in the second view;
the parameterized rotation matrix and translation vector are adopted to represent the first attitude relationship as follows:
Figure 469163DEST_PATH_IMAGE012
wherein,
Figure 726969DEST_PATH_IMAGE013
representing a translation vector in the first position posture relation, I representing a unit matrix, and R representing a rotation matrix corresponding to the first position posture relation;
and obtaining the corresponding essential matrix as follows:
Figure 953551DEST_PATH_IMAGE014
wherein, E represents an essential matrix,
Figure 929597DEST_PATH_IMAGE015
representing an anti-symmetric matrix.
4. The method of claim 3, wherein the epipolar geometry constraint is:
Figure 611245DEST_PATH_IMAGE016
wherein,
Figure 51454DEST_PATH_IMAGE017
normalized homogeneous image coordinates for homonymous point pairs in the first view and the second view;
the affine transformation constraint is:
Figure 121041DEST_PATH_IMAGE018
where the subscript (1:2) represents the first two equations,
Figure 505755DEST_PATH_IMAGE019
representing a local affine transformation matrix corresponding to the normalized homogeneous image coordinates.
5. The method according to any one of claims 1 to 4, wherein solving the rotation matrix and the translation vector according to the two affine transformation constraints corresponding to the jth affine matching point and the one epipolar geometric constraint and the two affine transformation constraints determined by the other affine matching points comprises:
determining the relative motion parameters of the single camera to be four degrees of freedom according to the rotation parameter constraint;
selecting two affine transformation constraints corresponding to the jth affine matching point and other epipolar geometric constraints determined by the affine matching points, and constructing a first solution model as follows:
Figure 381307DEST_PATH_IMAGE020
wherein,
Figure 613705DEST_PATH_IMAGE021
the element term in (1) is unknown number
Figure 791877DEST_PATH_IMAGE022
，
Figure 539253DEST_PATH_IMAGE023
And
Figure 156179DEST_PATH_IMAGE024
the second-order term of (a) is,
Figure 430034DEST_PATH_IMAGE025
to represent
Figure 107003DEST_PATH_IMAGE026
The matrix size is three rows and two columns;
selecting other affine matching points to establish a world reference system, and obtaining a second solving model as follows:
Figure 76096DEST_PATH_IMAGE027
wherein,
Figure 168817DEST_PATH_IMAGE028
the element term in (1) is unknown number
Figure 375808DEST_PATH_IMAGE029
，
Figure 285995DEST_PATH_IMAGE030
And
Figure 945646DEST_PATH_IMAGE031
the second order term of (d);
according to the first solution model and the second solution modelSolving the model to obtain the unknown number
Figure 91326DEST_PATH_IMAGE032
The six equations of (1) are:
Figure 887243DEST_PATH_IMAGE033
obtaining algebraic solutions of six equations through a Gr baby basis solution, determining a rotation matrix R of a first attitude relationship according to the algebraic solutions, and obtaining a rotation matrix R of a first attitude relationship according to the rotation matrix R
Figure 702753DEST_PATH_IMAGE034
Mid-null space determination
Figure 787383DEST_PATH_IMAGE035
According to
Figure 549803DEST_PATH_IMAGE036
Calculating to obtain a translation vector in the second position posture relation and a translation vector in the third position posture relation,
Figure 528123DEST_PATH_IMAGE037
representation matrix
Figure 452217DEST_PATH_IMAGE038
The sub-matrices of the first two rows and the first two columns,
Figure 7832DEST_PATH_IMAGE039
representation matrix
Figure 370680DEST_PATH_IMAGE040
The sub-matrices of the first two rows and the first two columns,
Figure 406769DEST_PATH_IMAGE041
a determinant representing a matrix is provided,
Figure 439448DEST_PATH_IMAGE042
representing by unknowns
Figure 560987DEST_PATH_IMAGE043
Forming two rows and two columns of submatrices.
6. The method of claim 5, wherein when the camera is a multi-camera system, the method further comprises:
camera in system for acquiring multiple cameras
Figure 727526DEST_PATH_IMAGE044
External parameters of
Figure 352543DEST_PATH_IMAGE045
，
Figure 805390DEST_PATH_IMAGE046
A matrix of rotations is represented, which is,
Figure 210963DEST_PATH_IMAGE047
representing a translation vector; wherein, the multi-camera system includes: camera for taking first view or second view
Figure 118876DEST_PATH_IMAGE048
And a clairvoyance camera;
and parameterizing the translation vector in the multi-camera system by adopting a Pl ü cker vector to obtain the translation vector as follows:
Figure 536082DEST_PATH_IMAGE049
wherein,
Figure 300776DEST_PATH_IMAGE050
is the serial number of the camera,
Figure 131329DEST_PATH_IMAGE051
is the sequence number of the affine match point pair,
Figure 764304DEST_PATH_IMAGE052
is the view's serial number;
unit direction vector
Figure 363913DEST_PATH_IMAGE053
By passing
Figure 33929DEST_PATH_IMAGE054
The calculation results in that,
Figure 289461DEST_PATH_IMAGE055
camera of person being
Figure 273597DEST_PATH_IMAGE056
In
Figure 55608DEST_PATH_IMAGE057
The corresponding normalized homogeneous image coordinates are then compared,
Figure 21159DEST_PATH_IMAGE058
the force vector is represented by a force vector,
Figure 826304DEST_PATH_IMAGE059
an unknown depth parameter representing the translation vector parameterization;
obtaining a fourth attitude relationship between the two perspective cameras corresponding to the first view and the second view according to the parameterized rotation matrix and translation vector, wherein the fourth attitude relationship is as follows:
Figure 676449DEST_PATH_IMAGE060
and calculating an essential matrix corresponding to the fourth pose relation as follows:
Figure 250649DEST_PATH_IMAGE061
wherein,
Figure 403413DEST_PATH_IMAGE062
and
Figure 430275DEST_PATH_IMAGE063
respectively representing camera
Figure 818531DEST_PATH_IMAGE064
Video and audio player
Figure 434189DEST_PATH_IMAGE065
The rotation matrix of (a) is,
Figure 820171DEST_PATH_IMAGE066
and
Figure 662225DEST_PATH_IMAGE067
are respectively cameras
Figure 791855DEST_PATH_IMAGE068
Video and audio player
Figure 12752DEST_PATH_IMAGE069
The translation vector of (a);
Figure 631952DEST_PATH_IMAGE070
representing a rotation matrix between a first view and a second view,
Figure 633406DEST_PATH_IMAGE071
representing a translation vector between the first view and the second view.
7. The method according to claim 6, wherein said solving for said rotation matrix and said translation vector according to two affine transformation constraints corresponding to the jth said affine matching point and one epipolar geometric constraint and two affine transformation constraints determined by other said affine matching points further comprises:
determining the relative motion parameters of the multi-camera system to be five degrees of freedom according to the rotation parameter constraint;
selecting two affine transformation constraints corresponding to the jth affine matching point and an epipolar geometric constraint and an affine transformation constraint determined by the other affine matching points, and constructing a third solution model as follows:
Figure 753678DEST_PATH_IMAGE072
selecting other affine matching points to establish a world reference system, and obtaining a fourth solution model as follows:
Figure 891398DEST_PATH_IMAGE073
obtaining the unknown number according to the third solving model and the fourth solving model
Figure 947079DEST_PATH_IMAGE074
The eight equations of (1) are:
Figure 435829DEST_PATH_IMAGE075
obtaining algebraic solutions of eight equations through a Gr baby basis solution, determining a rotation matrix R of a first attitude relationship of the multi-camera system according to the algebraic solutions, and determining a rotation matrix R of the first attitude relationship of the multi-camera system according to the rotation matrix R
Figure 844945DEST_PATH_IMAGE076
Mid-null space determination
Figure 102751DEST_PATH_IMAGE077
According to
Figure 329333DEST_PATH_IMAGE078
And calculating to obtain a translation vector in the second position posture relation and a translation vector in the third position posture relation.
CN202110596663.4A 2021-05-31 2021-05-31 Estimation method of camera relative motion under the condition of known relative rotation angle Active CN113048985B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110596663.4A CN113048985B (en) 2021-05-31 2021-05-31 Estimation method of camera relative motion under the condition of known relative rotation angle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110596663.4A CN113048985B (en) 2021-05-31 2021-05-31 Estimation method of camera relative motion under the condition of known relative rotation angle

Publications (2)

Publication Number Publication Date
CN113048985A true CN113048985A (en) 2021-06-29
CN113048985B CN113048985B (en) 2021-08-06

Family

ID=76518592

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110596663.4A Active CN113048985B (en) 2021-05-31 2021-05-31 Estimation method of camera relative motion under the condition of known relative rotation angle

Country Status (1)

Country Link
CN (1) CN113048985B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115451958A (en) * 2022-11-10 2022-12-09 中国人民解放军国防科技大学 Camera Absolute Pose Optimization Method Based on Relative Rotation Angle

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6137902A (en) * 1997-07-22 2000-10-24 Atr Human Information Processing Research Laboratories Linear estimation method for three-dimensional position with affine camera correction
CN101226638A (en) * 2007-01-18 2008-07-23 中国科学院自动化研究所 A calibration method and device for a multi-camera system
CN111476842A (en) * 2020-04-10 2020-07-31 中国人民解放军国防科技大学 A camera relative pose estimation method and system
CN111696158A (en) * 2020-06-04 2020-09-22 中国人民解放军国防科技大学 Affine matching point pair-based multi-camera system relative pose estimation method and device
CN112629565A (en) * 2021-03-08 2021-04-09 中国人民解放军国防科技大学 Method, device and equipment for calibrating rotation relation between camera and inertial measurement unit

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6137902A (en) * 1997-07-22 2000-10-24 Atr Human Information Processing Research Laboratories Linear estimation method for three-dimensional position with affine camera correction
CN101226638A (en) * 2007-01-18 2008-07-23 中国科学院自动化研究所 A calibration method and device for a multi-camera system
CN111476842A (en) * 2020-04-10 2020-07-31 中国人民解放军国防科技大学 A camera relative pose estimation method and system
CN111696158A (en) * 2020-06-04 2020-09-22 中国人民解放军国防科技大学 Affine matching point pair-based multi-camera system relative pose estimation method and device
CN112629565A (en) * 2021-03-08 2021-04-09 中国人民解放军国防科技大学 Method, device and equipment for calibrating rotation relation between camera and inertial measurement unit

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
DÁNIEL BARÁTH,ET.AL: "Making affine correspondences work in camera geometry computation", 《COMPUTER VISION - ECCV 2020 16TH EUROPEAN CONFERENCE》 *
GUAN BANGLEI: "Relative Pose Estimation With a Single Affine Correspondence", 《IEEE TRANSACTIONS ON CYBERNETICS》 *
涂国勇等: "仿射变形下的异源图像匹配方法", 《计算机辅助设计与图形学学报》 *
田苗等: "一种无公共视场的多相机系统相对位姿解耦估计方法", 《光学学报》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115451958A (en) * 2022-11-10 2022-12-09 中国人民解放军国防科技大学 Camera Absolute Pose Optimization Method Based on Relative Rotation Angle
CN115451958B (en) * 2022-11-10 2023-02-03 中国人民解放军国防科技大学 Camera Absolute Pose Optimization Method Based on Relative Rotation Angle

Also Published As

Publication number Publication date
CN113048985B (en) 2021-08-06

Similar Documents

Publication Publication Date Title
Quan et al. Affine structure from line correspondences with uncalibrated affine cameras
Ramalingam et al. A unifying model for camera calibration
Sturm et al. Camera models and fundamental concepts used in geometric computer vision
Devernay et al. From projective to euclidean reconstruction
CN102750704B (en) Step-by-step video camera self-calibration method
CN110264528A (en) Quick self-calibration method for fisheye lens binocular camera
CN104537707A (en) Image space type stereo vision on-line movement real-time measurement system
CN111754579A (en) Method and device for determining external parameters of multi-view camera
US20200294269A1 (en) Calibrating cameras and computing point projections using non-central camera model involving axial viewpoint shift
CN116030136B (en) Cross-angle visual positioning method, device and computer equipment based on geometric features
CN107809610A (en) Camera parameter set calculating apparatus, camera parameter set calculation method and program
KR102361133B1 (en) Method for acquiring distance to at least one object located in omni-direction of vehicle and vision device using the same
CN113223163A (en) Point cloud map construction method and device, equipment and storage medium
CN117974801A (en) A multi-view camera calibration method based on deep learning
Guan et al. Efficient recovery of multi-camera motion from two affine correspondences
CN113048985B (en) Estimation method of camera relative motion under the condition of known relative rotation angle
KR102372298B1 (en) Method for acquiring distance to at least one object located in omni-direction of vehicle and vision device using the same
Rousso et al. Robust recovery of camera rotation from three frames
Fusiello Elements of geometric computer vision
Raposo et al. Extrinsic calibration of multi-modal sensor arrangements with non-overlapping field-of-view
CN107330934B (en) Low-dimensional cluster adjustment calculation method and system
CN106558082B (en) Tessellated projection matrix, which is mapped, using Veronese demarcates central catadiotric video camera
Wang et al. Perspective 3-D Euclidean reconstruction with varying camera parameters
CN117351047A (en) Local map point cloud global alignment method and device, electronic equipment and storage medium
Wang et al. What can we learn about the scene structure from three orthogonal vanishing points in images

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant