WO2021098765A1 - 一种基于运动状态的关键帧选取方法及装置 - Google Patents

一种基于运动状态的关键帧选取方法及装置 Download PDF

Info

Publication number
WO2021098765A1
WO2021098765A1 PCT/CN2020/130050 CN2020130050W WO2021098765A1 WO 2021098765 A1 WO2021098765 A1 WO 2021098765A1 CN 2020130050 W CN2020130050 W CN 2020130050W WO 2021098765 A1 WO2021098765 A1 WO 2021098765A1
Authority
WO
WIPO (PCT)
Prior art keywords
key frame
images
feature points
image
matrix
Prior art date
Application number
PCT/CN2020/130050
Other languages
English (en)
French (fr)
Inventor
李春宾
Original Assignee
北京影谱科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 北京影谱科技股份有限公司 filed Critical 北京影谱科技股份有限公司
Priority to US17/778,411 priority Critical patent/US20220398845A1/en
Publication of WO2021098765A1 publication Critical patent/WO2021098765A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/62—Extraction of image or video features relating to a temporal dimension, e.g. time-based feature extraction; Pattern tracking
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G06V20/46—Extracting features or characteristics from the video content, e.g. video fingerprints, representative shots or key frames
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/30—Determination of transform parameters for the alignment of images, i.e. image registration
    • G06T7/33—Determination of transform parameters for the alignment of images, i.e. image registration using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/74—Image or video pattern matching; Proximity measures in feature spaces
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/52—Surveillance or monitoring of activities, e.g. for recognising suspicious objects
    • G06V20/54—Surveillance or monitoring of activities, e.g. for recognising suspicious objects of traffic, e.g. cars on the road, trains or boats
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle

Definitions

  • This application relates to the field of traffic image processing, in particular to a method and device for selecting a key frame based on a motion state.
  • the usual method is to select some key frames instead of all frames from the sequence of images or videos for processing, which can greatly reduce the computational pressure on the premise of ensuring accuracy and reliability.
  • an appropriate key frame selection strategy can also improve the accuracy and consistency of VO/VSLAM local motion estimation. Therefore, how to select key frames is an important factor to improve the accuracy and real-time performance of the visual SLAM (simultaneous localization and mapping) algorithm.
  • the existing key frame selection methods can be roughly divided into the following categories: a. Select key frames at the same interval or equal distance. Parallel tracking and mapping (PTAM) need to meet the preset tracking conditions when inserting key frames. The distance of the key frame needs to meet the preset translation and rotation angle; b. The key frame selection of image overlap, when the matching point of the overlap area is less than 50% of the detection point, the key is generated by the non-linear optimized visual inertial SLAM (OKVIS) Frame, while marginalizing the farthest key frame, keeping the latest set of frames and another set of key frames; c. Key frame selection based on disparity, if the average disparity of the tracking feature exceeds a certain threshold, the frame is regarded as a key frame D. Based on the key frame selection of the image content index, the feature clustering space of the current frame is established, and then the feature distance between the current frame and the next frame is calculated, and the key frame is selected according to the feature distance threshold.
  • PTAM Parallel tracking and mapping
  • the purpose of this application is to overcome the above-mentioned problems or to at least partially solve or alleviate the above-mentioned problems.
  • a method for selecting a key frame based on a motion state including:
  • Initialization step sequentially store several adjacent groups of images in the key frame sequence F, each group of images contains two adjacent frames of images, and preprocess the images, the key frame sequence F in The images are f 1 to f n in sequence;
  • Decomposition step Calculate the basic matrix E between adjacent frames in the key frame sequence F according to the obtained feature point pairs, and decompose the basic matrix E into a rotation matrix R and a translation vector If the rotation matrix R is a singular matrix or the translation scale of the translation vector exceeds the preset threshold, the basic matrix E is recalculated until the rotation matrix R is a non-singular matrix and the translation scale of the translation vector does not exceed the preset threshold;
  • Deflection angle calculation step decompose the non-singular rotation matrix R according to the direction of the coordinate axis to obtain the deflection angle of each coordinate axis;
  • the threshold condition in the key frame selection step is: ⁇ m ⁇
  • the methods used to calculate the basic matrix E are the five-point method and the RANSAC algorithm.
  • the method used to extract feature points is the FAST method.
  • the data set used in the method is a KITTI data set.
  • a device for selecting a key frame based on a motion state including:
  • An initialization module which is configured to sequentially store several adjacent groups of images in the key frame sequence F, each group of images contains two adjacent frames of images, and preprocess the images, the key frame sequence
  • the images in F are f 1 to f n in sequence;
  • the initial value of i is 3, k is the number of interval frames, and the initial value of k is 1;
  • the decomposition module is configured to calculate the basic matrix E between adjacent frames in the key frame sequence F according to the obtained feature point pairs, and decompose the basic matrix E into a rotation matrix R and a translation vector If the rotation matrix R is a singular matrix or the translation scale of the translation vector exceeds the preset threshold, the basic matrix E is recalculated until the rotation matrix R is a non-singular matrix and the translation scale of the translation vector does not exceed the preset threshold;
  • a deflection angle calculation module which is configured to decompose the non-singular rotation matrix R according to the direction of the coordinate axis to obtain the deflection angle of each coordinate axis;
  • the threshold condition in the key frame selection module is: ⁇ m ⁇
  • the methods used to calculate the basic matrix E are the five-point method and the RANSAC algorithm.
  • the method used to extract feature points is the FAST method.
  • the data set used by the device is a KITTI data set.
  • the method and device for selecting key frames based on the motion state of the present application predicts the motion state of the object and then performs the key frame selection through the posture changes between frames within a certain time interval, so it can balance the flexibility of key frames and real-time performance, In addition, the above method and device can also evaluate the impact of the corner tracking threshold and the object motion offset angle on the key frame.
  • Fig. 1 is a schematic flowchart of a method for selecting a key frame based on a motion state according to an embodiment of the present application
  • Fig. 2 is a schematic structural block diagram of a device for selecting a key frame based on a motion state according to an embodiment of the present application
  • Fig. 3 is a schematic structural block diagram of a computing device according to an embodiment of the present application.
  • Fig. 4 is a schematic structural block diagram of a computer-readable storage medium according to an embodiment of the present application.
  • the embodiment of the application provides a method for selecting a key frame based on a motion state.
  • the experimental data set used by the method is the KITTI data set (co-founded by the Düsseldorf Institute of Technology in Germany and the Toyota American Institute of Technology).
  • the set is currently the world's largest computer vision algorithm evaluation data set in autonomous driving scenarios.
  • the KITTI data acquisition platform includes 2 grayscale cameras, 2 color cameras, a Velodyne 3D lidar, 4 optical lenses, and a GPS navigation system.
  • the entire data set consists of 389 pairs of stereo images and optical flow maps (each image contains up to 15 vehicles and 30 pedestrians, and there are varying degrees of occlusion), 39.2 kilometers of visual ranging sequence and more than 200,000 3D-annotated object images.
  • the pose of the vehicle changes under these conditions: a. The change in the yaw angle around the Y axis when traveling along the horizontal plane; b. The change in the pitch angle around the X axis when going uphill and downhill; c. When The change in the roll angle around the Z axis when lateral jitter occurs. The local motion of the camera is consistent in a short time interval, and then the key frame is selected according to the change of the pose angle.
  • Fig. 1 is a schematic flowchart of a method for selecting a key frame based on a motion state according to an embodiment of the present application.
  • the method may generally include:
  • the first frame image and the second frame image are respectively stored in F, and the next frame is tracked. If it fails, two adjacent frames are selected and stored in F in sequence.
  • Feature point matching step use the FAST method to detect the feature points in the image f i (the initial value of i is 3), and then track the feature points in the image f i+k (the initial value of k is 1), that is, the image f i and the image f i+k are matched with feature points. If the number of matched feature points is less than the preset threshold, the feature points in the image f i can be re-detected, and the image f i and the image f i+k can be re-detected Perform feature point matching.
  • the basic matrix E is calculated by the five-point method and the RANSAC algorithm, and the basic matrix E is decomposed into a rotation matrix R and a translation vector
  • R is called the rotation matrix, also called the directional cosine matrix (DCM). If R is a singular matrix or the translation scale of the translation vector exceeds the preset threshold (only one of the two conditions is satisfied), then the basic Matrix E, until the rotation matrix R is a non-singular matrix and the translation scale of the translation vector does not exceed the preset threshold;
  • DCM directional cosine matrix
  • Deflection angle calculation step Calculate the components of the Euler angle in the X, Y, and Z directions of the three coordinate axes, and the three components obtained are the pitch angle ⁇ , the heading angle ⁇ , and the roll angle ⁇ .
  • the calculation formula of matrix R is as follows:
  • R z ( ⁇ ) represents the angle of rotation around the Z axis
  • R y ( ⁇ ) represents the angle of rotation around the Y axis
  • R x ( ⁇ ) represents the angle of rotation around the X axis
  • c ⁇ , c ⁇ , and c ⁇ are the abbreviations of cos ⁇ , cos ⁇ , and cos ⁇ , respectively, and s ⁇ , s ⁇ and s ⁇ are the abbreviations of sin ⁇ , sin ⁇ , and sin ⁇ , respectively;
  • ⁇ is a preset positive number that is small enough, such as 10 -10 ;
  • attitude angle can be approximately expressed as:
  • the above-mentioned key frame selection method based on motion state ignores large-scale motion other than the forward direction.
  • the corner tracking algorithm reduces the constraint of slight motion, evaluates the consistency of feature points between discontinuous frames, and determines the threshold of the attitude angle change between frames. And the interval step length, to ensure that the corner tracking is not lost and the motion state of the object is accurately restored, which can balance the flexibility and real-time performance of key frames.
  • the embodiment of the application also provides a device for selecting a key frame based on a motion state.
  • the experimental data set used by the device is the KITTI data set (co-founded by the Düsseldorf Institute of Technology in Germany and the Toyota Institute of Technology).
  • the data set is currently the world's largest computer vision algorithm evaluation data set in autonomous driving scenarios.
  • the KITTI data acquisition platform includes 2 grayscale cameras, 2 color cameras, a Velodyne 3D lidar, 4 optical lenses, and a GPS navigation system.
  • the entire data set consists of 389 pairs of stereo images and optical flow maps (each image contains up to 15 cars and 30 pedestrians, and there are different degrees of occlusion), 39.2 kilometers of visual ranging sequence and more than 200,000 3D-annotated object images.
  • the pose of the vehicle changes under these conditions: a. The change in the yaw angle around the Y axis when traveling along the horizontal plane; b. The change in the pitch angle around the X axis when going uphill and downhill; c. When The change in the roll angle around the Z axis when lateral jitter occurs. The local motion of the camera is consistent in a short time interval, and then the key frame is selected according to the change of the pose angle.
  • Fig. 2 is a schematic structural block diagram of a device for selecting a key frame based on a motion state according to another embodiment of the present application.
  • the device may generally include:
  • Initialization module 1 Read the serialized images f 1 , f 2 , ..., f n , and initialize the key frame sequence F. During the initialization process, the first frame image and the second frame image are stored in F respectively, and Track the next frame. If it fails, select two adjacent frames and store them in F.
  • Feature point matching module 2 This module uses the FAST method to detect the feature points in the image f i (the initial value of i is 3), and then tracks the feature points in the image f i+k (the initial value of k is 1), that is, the image Perform feature point matching between f i and image f i+k . If the number of matched feature points is less than the preset threshold, the feature points in image f i can be detected again, and the image f i and image f i+ k performs feature point matching.
  • Decomposition module 3 According to the obtained feature point pairs between the image f i and the image f q , the basic matrix E is calculated by the five-point method and the RANSAC algorithm, and the basic matrix E is decomposed into a rotation matrix R and a translation vector
  • R is called the rotation matrix, also called the directional cosine matrix (DCM). If R is a singular matrix or the translation scale of the translation vector exceeds the preset threshold (only one of the two conditions is satisfied), then the basic Matrix E, until the rotation matrix R is a non-singular matrix and the translation scale of the translation vector does not exceed the preset threshold;
  • DCM directional cosine matrix
  • Deflection angle calculation module 4 Calculate the components of the Euler angle in the X, Y, and Z directions of the three coordinate axes, and the three components obtained are the pitch angle ⁇ , the heading angle ⁇ , and the roll angle ⁇ .
  • the calculation formula of matrix R is as follows:
  • Rz( ⁇ ) represents the angle of rotation around the Z axis
  • Ry( ⁇ ) represents the angle of rotation around the Y axis
  • Rx( ⁇ ) represents the angle of rotation around the X axis
  • c ⁇ , c ⁇ , and c ⁇ are the abbreviations of cos ⁇ , cos ⁇ , and cos ⁇ , respectively, and s ⁇ , s ⁇ and s ⁇ are the abbreviations of sin ⁇ , sin ⁇ , and sin ⁇ , respectively;
  • ⁇ is a preset positive number that is small enough, such as 10 -10 ;
  • attitude angle can be approximately expressed as:
  • Key frame selection module 5 If ⁇ m ⁇
  • the above-mentioned key frame selection module based on the motion state ignores large motions other than the forward direction, and reduces the constraints of slight motion through the corner tracking algorithm, evaluates the consistency of feature points between discontinuous frames, and determines the threshold of the attitude angle change between frames And the interval step length, to ensure that the corner tracking is not lost and the motion state of the object is accurately restored, which can balance the flexibility and real-time performance of key frames.
  • the embodiment of the present application also provides a computing device.
  • the computing device includes a memory 1120, a processor 1110, and a computer program stored in the memory 1120 and capable of being run by the processor 1110.
  • the computer program A space 1130 for program codes stored in the memory 1120, and when the computer program is executed by the processor 1110, it is used to execute any method step 1131 according to the present invention.
  • the embodiment of the present application also provides a computer-readable storage medium.
  • the computer-readable storage medium includes a storage unit for program code, the storage unit is provided with a program 1131' for executing the method steps according to the present invention, and the program is executed by a processor.
  • the embodiments of the present application also provide a computer program product containing instructions.
  • the computer program product runs on the computer, the computer is caused to execute the method steps according to the present invention.
  • the computer may be implemented in whole or in part by software, hardware, firmware, or any combination thereof.
  • software it can be implemented in the form of a computer program product in whole or in part.
  • the computer program product includes one or more computer instructions.
  • the computer loads and executes the computer program instructions, the processes or functions described in the embodiments of the present application are generated in whole or in part.
  • the computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices.
  • the computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions may be transmitted from a website, computer, server, or data center.
  • the computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center integrated with one or more available media.
  • the usable medium may be a magnetic medium (for example, a floppy disk, a hard disk, and a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid state disk (SSD)).
  • a person of ordinary skill in the art can understand that all or part of the steps in the method of the foregoing embodiments can be implemented by a program instructing a processor to complete, and the program can be stored in a computer-readable storage medium, which is non-transitory ( English: non-transitory media, such as random access memory, read-only memory, flash memory, hard disk, solid state drive, magnetic tape (English: magnetic tape), floppy disk (English: floppydisk), optical disc (English: optical disc) and Any combination of it.
  • non-transitory such as random access memory, read-only memory, flash memory, hard disk, solid state drive, magnetic tape (English: magnetic tape), floppy disk (English: floppydisk), optical disc (English: optical disc) and Any combination of it.

Abstract

本申请公开了一种基于运动状态的关键帧选取方法及装置。所述方法将相邻的若干组图像依次存储到关键帧序列F中，每组图像包含相邻的两帧图像；从图像中提取特征点，并将第i张图像的特征点依次与后面图像的特征点进行匹配，直到匹配到的特征点数达到预设阈值为止，形成新的关键帧序列F；计算新的关键帧序列F中相邻帧间的基本矩阵E并将其分解为旋转矩阵R和平移矢量aa，将非奇异的旋转矩阵R按照坐标轴方向分解，得到各坐标轴的偏转角度；将偏转角度和预定的阈值作比较，将偏转角度大于阈值的当前帧选作关键帧，添加到最终的关键帧序列中。所述装置包括初始化模块、特征点匹配模块、分解模块、偏转角度计算模块及关键帧选择模块。

Description

一种基于运动状态的关键帧选取方法及装置
本申请要求于2019年11月20日提交中国专利局、申请号为CN201911142539.X、申请名称为“一种基于运动状态的关键帧选取方法及装置”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及交通图像处理领域，特别是涉及一种基于运动状态的关键帧选取方法及装置。
背景技术
实时VO/VSLAM和来自运动的大规模结构(SFM)对有限的计算资源提出了严峻的挑战。为了克服这个问题并减少数据冗余，通常的方法是从序列图像或视频中选择一些关键帧而不是所有帧进行处理，这样可以在保证准确性和可靠性的前提下大大降低计算压力。同时，适当的关键帧选择策略还可以提高VO/VSLAM局部运动估计的准确性和一致性。所以，如何选择关键帧是提高视觉SLAM(simultaneous localization and mapping)算法精度及实时性的重要因素。
现有的关键帧的选取方法大致可以分为以下几类：a.以相同间隔或相等距离选取关键帧，并行跟踪和建图(PTAM)需要在插入关键帧时满足预设跟踪条件，前一个关键帧的距离需满足预设的平移和旋转角度；b.图像重叠的关键帧选择，当重叠区域的匹配点小于检测点的50％时，通过非线性优化的视觉惯性SLAM(OKVIS)生成关键帧，同时边缘化最远关键帧，保留最新的一组帧和另一组关键帧；c.基于视差的关键帧选择，如果跟踪特征的平均视差超过某个 阈值，把该帧视为关键帧；d.基于图像内容索引的关键帧选择，建立当前帧的特征聚类空间，然后计算当前帧与下一帧之间的特征距离，并根据特征距离阈值选择关键帧。
以相同间隔的关键帧选取方法虽然容易实现，不需要太多额外的计算，但是灵活性不足。而其他的方法(如图像重叠，视差)性能好一些，但是特征会重复提取和匹配，视差与协方差的计算更加耗时，降低了实时性。
发明内容
本申请的目的在于克服上述问题或者至少部分地解决或缓减解决上述问题。
根据本申请的一个方面，提供了一种基于运动状态的关键帧选取方法，包括：
初始化步骤：将相邻的若干组图像依次存储到关键帧序列F中，每组图像包含相邻的两帧图像，并对所述图像进行预处理，所述的关键帧序列F中的图像依次为f 1至f n；
特征点匹配步骤：从关键帧序列F的图像中提取特征点，并将图像f i的特征点与图像f i+k的特征点进行匹配，若匹配到的特征点数未达到预设的阈值，则令k＝k+1，然后将图像f i的特征点与图像f i+k的特征点进行匹配，以此类推，直到匹配到的特征点数达到预设的阈值为止，得到图像的帧间特征点对，i的初始值为3，k为间隔帧数，k的初始值为1；
分解步骤：根据得到的特征点对计算关键帧序列F中相邻帧间的基本矩阵E，并将基本矩阵E分解为旋转矩阵R和平移矢量
Figure PCTCN2020130050-appb-000001
若旋转矩阵R为奇异矩阵、或平移矢量的平移尺度超过预设的阈值，则重新计算基本矩阵E，直到旋转矩阵R为非奇异矩阵、且平移矢量的平移尺度未超过预设的阈值为止；
偏转角度计算步骤：将非奇异的旋转矩阵R按照坐标轴的方向分解，得到各个坐标轴的偏转角度；
关键帧选择步骤：若得到的各个坐标轴的偏转角度满足阈值条件，则将当 前帧选作关键帧，并添加到最终的关键帧序列中，否则，令k＝k+1，然后返回特征点提取步骤；若k＝m时，得到的各个坐标轴的偏转角度仍不满足阈值条件，则令k＝1且i＝i+1，然后返回特征点提取步骤。
可选地，所述的关键帧选择步骤中的阈值条件为：α<mα||β<mβ||γ<mγ，其中，α、β和γ分别为欧拉角在X轴、Y轴和Z轴方向的偏转角度。
可选地，所述的分解步骤中，计算基本矩阵E所采用的方法为五点法与RANSAC算法。
可选地，所述特征点匹配步骤中，提取特征点所采用的方法为FAST方法。
可选地，所述方法所采用的数据集为KITTI数据集。
根据本申请的另一个方面，提供了一种基于运动状态的关键帧选取装置，包括：
初始化模块，其配置成将相邻的若干组图像依次存储到关键帧序列F中，每组图像包含相邻的两帧图像，并对所述图像进行预处理，所述的关键帧序列F中的图像依次为f 1至f n；
特征点匹配模块，其配置成从关键帧序列F的图像中提取特征点，并将图像f i的特征点与图像f i+k的特征点进行匹配，若匹配到的特征点数未达到预设的阈值，则令k＝k+1，然后将图像f i的特征点与图像f i+k的特征点进行匹配，以此类推，直到匹配到的特征点数达到预设的阈值为止，得到图像的帧间特征点对，i的初始值为3，k为间隔帧数，k的初始值为1；
分解模块，其配置成根据得到的特征点对计算关键帧序列F中相邻帧间的基本矩阵E，并将基本矩阵E分解为旋转矩阵R和平移矢量
Figure PCTCN2020130050-appb-000002
若旋转矩阵R为奇异矩阵、或平移矢量的平移尺度超过预设的阈值，则重新计算基本矩阵E，直到旋转矩阵R为非奇异矩阵、且平移矢量的平移尺度未超过预设的阈值为止；
偏转角度计算模块，其配置成将非奇异的旋转矩阵R按照坐标轴的方向分解，得到各个坐标轴的偏转角度；
关键帧选择模块，其配置成若得到的各个坐标轴的偏转角度满足阈值条 件，则将当前帧选作关键帧，并添加到最终的关键帧序列中，否则，令k＝k+1，然后返回特征点提取步骤；若k＝m时，得到的各个坐标轴的偏转角度仍不满足阈值条件，则令k＝1且i＝i+1，然后返回特征点提取步骤。
可选地，所述的关键帧选择模块中的阈值条件为：α<mα||β<mβ||γ<mγ，其中，α、β和γ分别为欧拉角在X轴、Y轴和Z轴方向的偏转角度。
可选地，所述的分解模块中，计算基本矩阵E所采用的方法为五点法与RANSAC算法。
可选地，所述特征点匹配模块中，提取特征点所采用的方法为FAST方法。
可选地，所述装置所采用的数据集为KITTI数据集。
本申请的基于运动状态的关键帧选取方法及装置，由于通过一定时间间隔内的帧间姿态变化，进而预测对象的运动状态，然后执行关键帧选择，因此能够平衡关键帧灵活性和实时性，此外，上述方法及装置还能够评估角点跟踪的阈值与对象运动偏移角度对关键帧影响。
根据下文结合附图对本申请的具体实施例的详细描述，本领域技术人员将会更加明了本申请的上述以及其他目的、优点和特征。
附图说明
后文将参照附图以示例性而非限制性的方式详细描述本申请的一些具体实施例。附图中相同的附图标记标示了相同或类似的部件或部分。本领域技术人员应该理解，这些附图未必是按比例绘制的。附图中：
图1是根据本申请一个实施例的一种基于运动状态的关键帧选取方法的示意性流程图；
图2是根据本申请一个实施例的一种基于运动状态的关键帧选取装置的示意性结构框图；
图3是根据本申请一个实施例的一种计算设备的示意性结构框图；
图4是根据本申请一个实施例的一种计算机可读存储介质的示意性结构框图。
具体实施方式
本申请实施例提供了一种基于运动状态的关键帧选取方法，所述方法采用的实验数据集为KITTI数据集(由德国卡尔斯鲁厄理工学院和丰田美国技术研究院联合创办)，该数据集是目前国际上最大的自动驾驶场景下的计算机视觉算法评测数据集。KITTI数据采集平台包括2个灰度摄像机、2个彩色摄像机、一个Velodyne 3D激光雷达、4个光学镜头、以及1个GPS导航系统。整个数据集由389对立体图像和光流图(每张图像最多包含15辆车及30个行人，并且存在不同程度的遮挡)、39.2公里视觉测距序列以及超过200,0003D标注物体的图像组成。
车辆的位姿在这几种情况下发生变化：a.沿水平面行进时，绕Y轴的偏航角度的变化；b.上坡和下坡时绕X轴的俯仰角的变化；c.当发生横向抖动时，绕Z轴的滚动角的变化。像机的局部运动在短时间间隔内是一致的，然后根据位姿角的变化选择关键帧。
图1是根据本申请一个实施例的一种基于运动状态的关键帧选取方法的示意性流程图。所述方法一般性地可包括：
S1、初始化步骤：读取序列化图像f 1、f 2、……、f n；
初始化过程中，将第一帧图像和第二帧图像分别存储到F中，并跟踪下一帧，如果失败，则依次选择相邻的两帧存储到F中。
S2、特征点匹配步骤：采用FAST方法检测图像f i(i的初始值为3)中的特征点，然后跟踪图像f i+k(k的初始值为1)中的特征点，即将图像f i与图像f i+k进行特征点匹配，如果匹配到的特征点个数小于预设的阈值，则可以重新检测图像f i中的特征点，并重新将图像f i与图像f i+k进行特征点匹配，若再次匹配到的特征点个数仍然小于所述的阈值，则舍弃图像f i+k，增加间隔，即令k＝k+1，然后将图像f i与新的图像f i+k进行特征点匹配……不断增加k的值，直到图像f i与某帧图像f q匹配到的特征点数达到阈值为止，得到图像f i与图像f q之间的特征点对。
S3、分解步骤：根据得到的图像f i与图像f q之间的特征点对，采用五点法与RANSAC算法计算基本矩阵E，并将基本矩阵E分解为旋转矩阵R和平移矢量
Figure PCTCN2020130050-appb-000003
假设两图片的坐标空间P＝{p1,p2,…,pn}，Q＝{q1,q2,…,qn}，在旋转和平移之后通过外部旋转元素(R|t)表示为：Q＝RP+t，
其中，
Figure PCTCN2020130050-appb-000004
这里的R称为旋转矩阵，也称为方向余弦矩阵(DCM)，若R为奇异矩阵、或平移矢量的平移尺度超过预设的阈值(两个条件只要满足一个即可)，则重新计算基本矩阵E，直到旋转矩阵R为非奇异矩阵、且平移矢量的平移尺度未超过预设的阈值为止；
S4、偏转角度计算步骤：计算欧拉角在三个坐标轴X、Y、Z方向的分量，得到的三个分量分别为螺距角α、航向角β、以及横摇角γ。矩阵R的计算公式如下:
Figure PCTCN2020130050-appb-000005
其中，R z(γ)表示绕Z轴的旋转角度，R y(β)表示绕Y轴的旋转角度、R x(α)表示绕X轴的旋转角度；
c α、c β、c γ分别是cosα、cosβ、cosγ的缩写，s α、s β、s γ分别是sinα、sinβ、sinγ的缩写；
然后可以得到如下的姿态角：
(1)当|r 20|≤1-ξ时，姿态角可以表示如下:
Figure PCTCN2020130050-appb-000006
其中，ξ是预设的一个足够小的正数，例如10 -10；
(2)当r 20>1-ξ，且β→π/2时，做一个近似cos(β)≈0和sin(β)≈1，那么姿态角可以近似表示为：
Figure PCTCN2020130050-appb-000007
(3)当r 20<1-ξ，且β→-π/2时，做一个近似cos(β)≈0和sin(β)≈-1，然后姿态角可以近似表示为：
Figure PCTCN2020130050-appb-000008
S5、关键帧选择步骤：如果α<mα||β<mβ||γ<mγ，则将当前帧放入最终的关键帧序列F中，m为预设的间隔帧数的最大值，mα、mβ和mγ为预设的三个姿态角阈值；如果得到的三个偏转角α、β和γ不满足α<mα||β<mβ||γ<mγ，则令k＝1且i＝i+1，然后返回步骤S2。
上述基于运动状态的关键帧选取方法忽略了向前方向以外的大幅度运动，通过角点跟踪算法减轻轻微运动的约束，评估不连续帧间特征点的一致性，确定帧间姿态角度变化的阈值和间隔步长，保证角点跟踪没有丢失并且对象的运动状态被准确地恢复，能够平衡关键帧的灵活性和实时性。
本申请实施例还提供了一种基于运动状态的关键帧选取装置，所述装置采用的实验数据集为KITTI数据集(由德国卡尔斯鲁厄理工学院和丰田美国技术研究院联合创办)，该数据集是目前国际上最大的自动驾驶场景下的计算机视觉算法评测数据集。KITTI数据采集平台包括2个灰度摄像机、2个彩色摄像机、一个Velodyne 3D激光雷达、4个光学镜头、以及1个GPS导航系统。整个数据集由389对立体图像和光流图(每张图像最多包含15辆车及30个行人， 并且存在不同程度的遮挡)、39.2公里视觉测距序列以及超过200,0003D标注物体的图像组成。
车辆的位姿在这几种情况下发生变化：a.沿水平面行进时，绕Y轴的偏航角度的变化；b.上坡和下坡时绕X轴的俯仰角的变化；c.当发生横向抖动时，绕Z轴的滚动角的变化。像机的局部运动在短时间间隔内是一致的，然后根据位姿角的变化选择关键帧。
图2是根据本申请另一个实施例的一种基于运动状态的关键帧选取装置的示意结构框图。所述装置一般性地可包括：
初始化模块1：读取序列化图像f 1、f 2、……、f n，对关键帧序列F进行初始化，初始化过程中，将第一帧图像和第二帧图像分别存储到F中，并跟踪下一帧，如果失败，则依次选择相邻的两帧存储到F中。
特征点匹配模块2：该模块采用FAST方法检测图像f i(i的初始值为3)中的特征点，然后跟踪图像f i+k(k的初始值为1)中的特征点，即将图像f i与图像f i+k进行特征点匹配，如果匹配到的特征点个数小于预设的阈值，则可以重新检测图像f i中的特征点，并重新将图像f i与图像f i+k进行特征点匹配，若再次匹配到的特征点个数仍然小于所述的阈值，则舍弃图像f i+k，增加间隔，即令k＝k+1，然后将图像f i与新的图像f i+k进行特征点匹配……不断增加k的值，直到图像f i与某帧图像f q匹配到的特征点数达到阈值为止，得到图像f i与图像f q之间的特征点对。
分解模块3：根据得到的图像f i与图像f q之间的特征点对，采用五点法与RANSAC算法计算基本矩阵E，并将基本矩阵E分解为旋转矩阵R和平移矢量
Figure PCTCN2020130050-appb-000009
假设两图片的坐标空间P＝{p1,p2,…,pn}，Q＝{q1,q2,…,qn}，在旋转和平移之后通过外部旋转元素(R|t)表示为：Q＝RP+t，
其中，
Figure PCTCN2020130050-appb-000010
这里的R称为旋转矩阵，也称为方向余弦矩阵(DCM)，若R为奇异矩阵、或平移矢量的平移尺度超过预设的阈值(两个条件只要满足一个即可)，则重新计算基本矩阵E，直到旋转矩阵R为非奇异矩阵、且平移矢量的平移尺度未超过预设的阈值为止；
偏转角度计算模块4：计算欧拉角在三个坐标轴X、Y、Z方向的分量，得到的三个分量分别为螺距角α、航向角β、以及横摇角γ。矩阵R的计算公式如下:
Figure PCTCN2020130050-appb-000011
其中，Rz(γ)表示绕Z轴的旋转角度，Ry(β)表示绕Y轴的旋转角度、Rx(α)表示绕X轴的旋转角度；
c α、c β、c γ分别是cosα、cosβ、cosγ的缩写，s α、s β、s γ分别是sinα、sinβ、sinγ的缩写；
然后可以得到如下的姿态角：
(1)当|r 20|≤1-ξ时，姿态角可以表示如下:
Figure PCTCN2020130050-appb-000012
其中，ξ是预设的一个足够小的正数，例如10 -10；
(2)当r 20>1-ξ，且β→π/2时，做一个近似cos(β)≈0和sin(β)≈1，那么姿态角可以近似表示为：
Figure PCTCN2020130050-appb-000013
(3)当r 20<1-ξ，且β→-π/2时，做一个近似cos(β)≈0和sin(β)≈-1，然后姿态角可以近似表示为：
Figure PCTCN2020130050-appb-000014
关键帧选择模块5：如果α<mα||β<mβ||γ<mγ，则将当前帧放入最终的关键帧序列F中，m为预设的间隔帧数的最大值，mα、mβ和mγ为预设的三个姿态角阈值；如果得到的三个偏转角α、β和γ不满足α<mα||β<mβ||γ<mγ，则令k＝1且i＝i+1，然后返回特征点匹配模块2。
上述基于运动状态的关键帧选取模块忽略了向前方向以外的大幅度运动，通过角点跟踪算法减轻轻微运动的约束，评估不连续帧间特征点的一致性，确定帧间姿态角度变化的阈值和间隔步长，保证角点跟踪没有丢失并且对象的运动状态被准确地恢复，能够平衡关键帧的灵活性和实时性。
本申请实施例还提供了一种计算设备，参照图3，该计算设备包括存储器1120、处理器1110和存储在所述存储器1120内并能由所述处理器1110运行的计算机程序，该计算机程序存储于存储器1120中的用于程序代码的空间1130，该计算机程序在由处理器1110执行时实现用于执行任一项根据本发明的方法步骤1131。
本申请实施例还提供了一种计算机可读存储介质。参照图4，该计算机可读存储介质包括用于程序代码的存储单元，该存储单元设置有用于执行根据本发明的方法步骤的程序1131′，该程序被处理器执行。
本申请实施例还提供了一种包含指令的计算机程序产品。当该计算机程序产品在计算机上运行时，使得计算机执行根据本发明的方法步骤。
在上述实施例中，可以全部或部分地通过软件、硬件、固件或者其任意组合来实现。当使用软件实现时，可以全部或部分地以计算机程序产品的形式实现。所述计算机程序产品包括一个或多个计算机指令。在计算机加载和执行所述计算机程序指令时，全部或部分地产生按照本申请实施例所述的流程或功能。所述计算机可以是通用计算机、专用计算机、计算机网络、获取其他可编程装置。所述计算机指令可以存储在计算机可读存储介质中，或者从一个计算机可读存储介质向另一个计算机可读存储介质传输，例如，所述计算机指令可 以从一个网站站点、计算机、服务器或数据中心通过有线(例如同轴电缆、光纤、数字用户线(DSL))或无线(例如红外、无线、微波等)方式向另一个网站站点、计算机、服务器或数据中心进行传输。所述计算机可读存储介质可以是计算机能够存取的任何可用介质或者是包含一个或多个可用介质集成的服务器、数据中心等数据存储设备。所述可用介质可以是磁性介质，(例如，软盘、硬盘、磁带)、光介质(例如，DVD)、或者半导体介质(例如固态硬盘Solid State Disk(SSD))等。
专业人员应该还可以进一步意识到，结合本文中所公开的实施例描述的各示例的单元及算法步骤，能够以电子硬件、计算机软件或者二者的结合来实现，为了清楚地说明硬件和软件的可互换性，在上述说明中已经按照功能一般性地描述了各示例的组成及步骤。这些功能究竟以硬件还是软件方式来执行，取决于技术方案的特定应用和设计约束条件。专业技术人员可以对每个特定的应用来使用不同方法来实现所描述的功能，但是这种实现不应认为超出本申请的范围。
本领域普通技术人员可以理解实现上述实施例方法中的全部或部分步骤是可以通过程序来指令处理器完成，所述的程序可以存储于计算机可读存储介质中，所述存储介质是非短暂性(英文：non-transitory)介质，例如随机存取存储器，只读存储器，快闪存储器，硬盘，固态硬盘，磁带(英文：magnetic tape)，软盘(英文：floppydisk)，光盘(英文：optical disc)及其任意组合。
以上所述，仅为本申请较佳的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到的变化或替换，都应涵盖在本申请的保护范围之内。因此，本申请的保护范围应该以权利要求的保护范围为准。

Claims (10)

  1. 一种基于运动状态的关键帧选取方法，包括：
    初始化步骤：将相邻的若干组图像依次存储到关键帧序列F中，每组图像包含相邻的两帧图像，并对所述图像进行预处理，所述的关键帧序列F中的图像依次为f 1至f n；
    特征点匹配步骤：从关键帧序列F的图像中提取特征点，并将图像f i的特征点与图像f i+k的特征点进行匹配，若匹配到的特征点数未达到预设的阈值，则令k＝k+1，然后将图像f i的特征点与图像f i+k的特征点进行匹配，以此类推，直到匹配到的特征点数达到预设的阈值为止，得到图像的帧间特征点对，i的初始值为3，k为间隔帧数，k的初始值为1；
    分解步骤：根据得到的特征点对计算关键帧序列F中相邻帧间的基本矩阵E，并将基本矩阵E分解为旋转矩阵R和平移矢量
    Figure PCTCN2020130050-appb-100001
    若旋转矩阵R为奇异矩阵、或平移矢量的平移尺度超过预设的阈值，则重新计算基本矩阵E，直到旋转矩阵R为非奇异矩阵、且平移矢量的平移尺度未超过预设的阈值为止；
    偏转角度计算步骤：将非奇异的旋转矩阵R按照坐标轴的方向分解，得到各个坐标轴的偏转角度；
    关键帧选择步骤：若得到的各个坐标轴的偏转角度满足阈值条件，则将当前帧选作关键帧，并添加到最终的关键帧序列中，否则，令k＝k+1，然后返回特征点提取步骤；若k＝m时，得到的各个坐标轴的偏转角度仍不满足阈值条件，则令k＝1且i＝i+1，然后返回特征点提取步骤。
  2. 根据权利要求1所述的方法，其特征在于，所述的关键帧选择步骤中的阈值条件为：α<mα||β<mβ||γ<mγ，其中，α、β和γ分别为欧拉角在X轴、Y轴和Z轴方向的偏转角度。
  3. 根据权利要求1或2所述的方法，其特征在于，所述的分解步骤中，计算基本矩阵E所采用的方法为五点法与RANSAC算法。
  4. 根据权利要求1-3中任一项所述的方法，其特征在于，所述特征点匹配步骤中，提取特征点所采用的方法为FAST方法。
  5. 根据权利要求1-4中任一项所述的方法，其特征在于，所述方法所采用的数据集为KITTI数据集。
  6. 一种基于运动状态的关键帧选取装置，包括：
    初始化模块，其配置成将相邻的若干组图像依次存储到关键帧序列F中，每组图像包含相邻的两帧图像，并对所述图像进行预处理，所述的关键帧序列F中的图像依次为f 1至f n；
    特征点匹配模块，其配置成从关键帧序列F的图像中提取特征点，并将图像f i的特征点与图像f i+k的特征点进行匹配，若匹配到的特征点数未达到预设的阈值，则令k＝k+1，然后将图像f i的特征点与图像f i+k的特征点进行匹配，以此类推，直到匹配到的特征点数达到预设的阈值为止，得到图像的帧间特征点对，i的初始值为3，k为间隔帧数，k的初始值为1；
    分解模块，其配置成根据得到的特征点对计算关键帧序列F中相邻帧间的基本矩阵E，并将基本矩阵E分解为旋转矩阵R和平移矢量
    Figure PCTCN2020130050-appb-100002
    若旋转矩阵R为奇异矩阵、或平移矢量的平移尺度超过预设的阈值，则重新计算基本矩阵E，直到旋转矩阵R为非奇异矩阵、且平移矢量的平移尺度未超过预设的阈值为止；
    偏转角度计算模块，其配置成将非奇异的旋转矩阵R按照坐标轴的方向分解，得到各个坐标轴的偏转角度；
    关键帧选择模块，其配置成若得到的各个坐标轴的偏转角度满足阈值条件，则将当前帧选作关键帧，并添加到最终的关键帧序列中，否则，令k＝k+1，然后返回特征点提取步骤；若k＝m时，得到的各个坐标轴的偏转角度仍不满足阈值条件，则令k＝1且i＝i+1，然后返回特征点提取步骤。
  7. 根据权利要求6所述的装置，其特征在于，所述的关键帧选择模块中的阈值条件为：α<mα||β<mβ||γ<mγ，其中，α、β和γ分别为欧拉角在X轴、Y轴和Z轴方向的偏转角度。
  8. 根据权利要求6或7所述的装置，其特征在于，所述的分解模块中，计算基本矩阵E所采用的方法为五点法与RANSAC算法。
  9. 根据权利要求6-8中任一项所述的装置，其特征在于，所述特征点匹配模块中，提取特征点所采用的方法为FAST方法。
  10. 根据权利要求6-9中任一项所述的装置，其特征在于，所述装置所采用的数据集为KITTI数据集。
PCT/CN2020/130050 2019-11-20 2020-11-19 一种基于运动状态的关键帧选取方法及装置 WO2021098765A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/778,411 US20220398845A1 (en) 2019-11-20 2020-11-19 Method and device for selecting keyframe based on motion state

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911142539.X 2019-11-20
CN201911142539.XA CN110992392A (zh) 2019-11-20 2019-11-20 一种基于运动状态的关键帧选取方法及装置

Publications (1)

Publication Number Publication Date
WO2021098765A1 true WO2021098765A1 (zh) 2021-05-27

Family

ID=70085393

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/130050 WO2021098765A1 (zh) 2019-11-20 2020-11-19 一种基于运动状态的关键帧选取方法及装置

Country Status (3)

Country Link
US (1) US20220398845A1 (zh)
CN (1) CN110992392A (zh)
WO (1) WO2021098765A1 (zh)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115273068A (zh) * 2022-08-02 2022-11-01 湖南大学无锡智能控制研究院 一种激光点云动态障碍物剔除方法、装置及电子设备

Families Citing this family (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110992392A (zh) * 2019-11-20 2020-04-10 北京影谱科技股份有限公司 一种基于运动状态的关键帧选取方法及装置
CN111836072B (zh) * 2020-05-21 2022-09-13 北京嘀嘀无限科技发展有限公司 视频处理方法、装置、设备和存储介质
CN111723713B (zh) * 2020-06-09 2022-10-28 上海合合信息科技股份有限公司 一种基于光流法的视频关键帧提取方法及系统
CN112911281B (zh) * 2021-02-09 2022-07-15 北京三快在线科技有限公司 一种视频质量评价方法及装置
CN116758058B (zh) * 2023-08-10 2023-11-03 泰安市中心医院(青岛大学附属泰安市中心医院、泰山医养中心) 一种数据处理方法、装置、计算机及存储介质
CN117649454A (zh) * 2024-01-29 2024-03-05 北京友友天宇系统技术有限公司 双目相机外参自动校正方法、装置、电子设备及存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104463788A (zh) * 2014-12-11 2015-03-25 西安理工大学 基于运动捕捉数据的人体运动插值方法
CN106296693A (zh) * 2016-08-12 2017-01-04 浙江工业大学 基于3d点云fpfh特征实时三维空间定位方法
CN107027051A (zh) * 2016-07-26 2017-08-08 中国科学院自动化研究所 一种基于线性动态系统的视频关键帧提取方法
CN110992392A (zh) * 2019-11-20 2020-04-10 北京影谱科技股份有限公司 一种基于运动状态的关键帧选取方法及装置

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108955687A (zh) * 2018-05-31 2018-12-07 湖南万为智能机器人技术有限公司 移动机器人的综合定位方法

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN104463788A (zh) * 2014-12-11 2015-03-25 西安理工大学 基于运动捕捉数据的人体运动插值方法
CN107027051A (zh) * 2016-07-26 2017-08-08 中国科学院自动化研究所 一种基于线性动态系统的视频关键帧提取方法
CN106296693A (zh) * 2016-08-12 2017-01-04 浙江工业大学 基于3d点云fpfh特征实时三维空间定位方法
CN110992392A (zh) * 2019-11-20 2020-04-10 北京影谱科技股份有限公司 一种基于运动状态的关键帧选取方法及装置

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115273068A (zh) * 2022-08-02 2022-11-01 湖南大学无锡智能控制研究院 一种激光点云动态障碍物剔除方法、装置及电子设备
CN115273068B (zh) * 2022-08-02 2023-05-12 湖南大学无锡智能控制研究院 一种激光点云动态障碍物剔除方法、装置及电子设备

Also Published As

Publication number Publication date
CN110992392A (zh) 2020-04-10
US20220398845A1 (en) 2022-12-15

Similar Documents

Publication Publication Date Title
WO2021098765A1 (zh) 一种基于运动状态的关键帧选取方法及装置
CN108230361B (zh) 用无人机探测器和追踪器融合来增强目标追踪方法及系统
CN107687850B (zh) 一种基于视觉和惯性测量单元的无人飞行器位姿估计方法
US20190204084A1 (en) Binocular vision localization method, device and system
US9098766B2 (en) Controlled human pose estimation from depth image streams
CN110363817B (zh) 目标位姿估计方法、电子设备和介质
WO2023016271A1 (zh) 位姿确定方法、电子设备及可读存储介质
US11788845B2 (en) Systems and methods for robust self-relocalization in a visual map
US9367922B2 (en) High accuracy monocular moving object localization
CN111127524A (zh) 一种轨迹跟踪与三维重建方法、系统及装置
CN108022254B (zh) 一种基于征点辅助的时空上下文目标跟踪方法
US20190301871A1 (en) Direct Sparse Visual-Inertial Odometry Using Dynamic Marginalization
CN112819860B (zh) 视觉惯性系统初始化方法及装置、介质和电子设备
CN111882602B (zh) 基于orb特征点和gms匹配过滤器的视觉里程计实现方法
CN110570453A (zh) 一种基于双目视觉的闭环式跟踪特征的视觉里程计方法
JP2022546643A (ja) 不確実性を有するランドマーク位置推定のための画像処理システムおよび方法
CN112561978A (zh) 深度估计网络的训练方法、图像的深度估计方法、设备
CN107704813B (zh) 一种人脸活体识别方法及系统
CN112651997A (zh) 地图构建方法、电子设备和存储介质
CN113177432A (zh) 基于多尺度轻量化网络的头部姿态估计方法、系统、设备及介质
Zhu et al. PairCon-SLAM: Distributed, online, and real-time RGBD-SLAM in large scenarios
WO2022021028A1 (zh) 目标检测方法、装置、无人机及计算机可读存储介质
CN111738085B (zh) 实现自动驾驶同时定位与建图的系统构建方法及装置
CN113888603A (zh) 基于光流跟踪和特征匹配的回环检测及视觉slam方法
CN113763468A (zh) 一种定位方法、装置、系统及存储介质

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20889852

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20889852

Country of ref document: EP

Kind code of ref document: A1