WO2022142855A1 - 回环检测方法、装置、终端设备和可读存储介质 - Google Patents

回环检测方法、装置、终端设备和可读存储介质 Download PDF

Info

Publication number
WO2022142855A1
WO2022142855A1 PCT/CN2021/132115 CN2021132115W WO2022142855A1 WO 2022142855 A1 WO2022142855 A1 WO 2022142855A1 CN 2021132115 W CN2021132115 W CN 2021132115W WO 2022142855 A1 WO2022142855 A1 WO 2022142855A1
Authority
WO
WIPO (PCT)
Prior art keywords
feature vector
image frame
signature
feature
matched
Prior art date
Application number
PCT/CN2021/132115
Other languages
English (en)
French (fr)
Inventor
黄德斌
王轶丹
杨若鹄
崔宁
陈鹏
Original Assignee
深圳市优必选科技股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳市优必选科技股份有限公司 filed Critical 深圳市优必选科技股份有限公司
Publication of WO2022142855A1 publication Critical patent/WO2022142855A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/44—Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10016—Video; Image sequence
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Definitions

  • the present application relates to the technical field of image processing, and in particular, to a loop closure detection method, apparatus, terminal device and readable storage medium.
  • Loop closure detection also known as closed loop detection, for example, for robots, refers to the ability of the robot to recognize that it has reached a certain scene and make the map closed loop. In other words, when the robot turns left and right to build a map, it can realize that a certain place has passed by, and then match the map generated at the moment with the map just generated.
  • the traditional visual SLAM simultaneous localization and mapping, simultaneous localization and mapping
  • SLAM simultaneous localization and mapping
  • the traditional visual SLAM (simultaneous localization and mapping, simultaneous localization and mapping) technology collects a large number of pictures when implementing loopback detection, then uses feature extraction method to extract feature points, and then uses some classification methods to classify these feature points into For different types, establish a dictionary of feature point sets. For the new image collected by the robot, extract the feature points again, and then query the feature point dictionary. Determine which type the feature point belongs to, and then detect the loop closure based on the similarity of the two images, also known as the visual word bag model.
  • the purpose of this application is to overcome the deficiencies in the prior art, and to provide a loop closure detection method, apparatus, terminal device and readable storage medium.
  • Embodiments of the present application provide a loopback detection method, including:
  • Hash calculation is performed on each eigenvector in the eigenvector matrix according to the constructed locality-sensitive hash function to obtain the signature of the corresponding eigenvector, and a hash table set of the eigenvector matrix is constructed with the signatures of all the eigenvectors ;
  • the method further includes:
  • All the found feature vectors to be matched are sequentially stored in the set to be matched according to the acquisition order of the corresponding historical image frames;
  • the method when using the constructed locality-sensitive hash function to calculate the signature of the feature vector, the method further includes:
  • the hash function family of the locality-sensitive hash function is generated according to one or both of "AND construction” and "OR construction", and the hash function family is used to calculate the signature of the feature vector.
  • the method when generating the hash function family, the method includes:
  • the hash function family of the locality-sensitive hash function is generated in a cascade manner of "AND construct” and then "OR construct".
  • the locality-sensitive hash function is constructed based on cosine distances.
  • the performing feature extraction on the current image frame to obtain a corresponding feature vector matrix includes:
  • a matrix of eigenvectors is constructed using each of the generated eigenvectors.
  • feature extraction is performed on the current image frame or the historical image frame through a MoblieNet neural network.
  • the hash table set of constructing the eigenvector matrix with the signatures of all eigenvectors includes:
  • a hash table set of the feature vector matrix is constructed according to the hash table indexes of all clusters.
  • a feature vector with a preset offset from the signature in the hash table set obtained by screening several stored historical image frames is used as the feature vector to be matched, including:
  • the signature of each candidate feature vector is calculated by the locality-sensitive hash function, and a candidate feature vector with a preset offset between the signature and the signature in the hash table set is found as the feature vector to be matched .
  • Embodiments of the present application also provide a loopback detection device, including:
  • a feature extraction module used to obtain a current image frame, and perform feature extraction on the current image frame to obtain a corresponding feature vector matrix
  • a hash calculation module configured to perform hash calculation on each feature vector in the feature vector matrix according to the constructed locality-sensitive hash function to obtain the signature of each feature vector, and construct the said feature vector with the signatures of all feature vectors
  • the candidate matching module is used to screen several stored historical image frames to obtain a feature vector with a preset offset from the signature in the hash table set as the feature vector to be matched;
  • a loopback judging module is configured to determine whether a corresponding historical image frame has a loopback according to the similarity between the feature vector of the current image frame and the feature vector to be matched, and output the historical image frame with loopback.
  • An embodiment of the present application further provides a terminal device, the terminal device includes a processor and a memory, the memory stores a computer program, and when the computer program is executed on the processor, the above-mentioned loop closure detection method is implemented.
  • the terminal device is a robot, an unmanned vehicle or an intelligent positioning backpack.
  • Embodiments of the present application further provide a readable storage medium, which stores a computer program, and when the computer program is executed on a processor, implements the above-mentioned loop closure detection method.
  • the loopback detection method of the embodiment of the present application realizes fast loopback detection by extracting the feature information of the image, using a local-sensitive hash function and pre-screening historical frame data, etc., which can greatly improve the real-time performance of the loopback of the terminal device, and further It can effectively solve the problem of trajectory drift caused by terminal equipment in the process of moving, and improve the robustness of terminal equipment.
  • FIG. 1 shows a schematic flowchart of a loopback detection method according to an embodiment of the present application
  • FIG. 2 shows a schematic diagram of a feature extraction process of a loop closure detection method according to an embodiment of the present application
  • FIG. 3 shows a schematic diagram of a feature extraction application of the loop closure detection method according to an embodiment of the present application
  • Figure 4a, Figure 4b, and Figure 4c respectively show the effect diagrams of the "AND structure” mode, the "OR structure” mode, and the "AND structure” and then the "OR structure” mode of the loop closure detection method according to the embodiment of the present application;
  • FIG. 5 shows a schematic flowchart of a hash table generation process of the loopback detection method according to an embodiment of the present application
  • FIG. 6 shows a schematic flowchart of the historical image frame screening of the loop closure detection method according to the embodiment of the present application
  • FIG. 7 shows another schematic flowchart of the loopback detection method according to the embodiment of the present application.
  • FIG. 8 shows a schematic structural diagram of a loopback detection apparatus according to an embodiment of the present application.
  • FIG. 1 shows a schematic flowchart of a loopback detection method according to an embodiment of the present application.
  • the loop closure detection method can be applied to terminals such as robots and unmanned vehicles to achieve real-time positioning and map construction.
  • the loop closure detection method includes:
  • Step S110 acquiring the current image frame, and performing feature extraction on the current image frame to obtain a corresponding feature vector matrix.
  • the terminal When the terminal acquires a frame of image, it can extract features of the current image frame through a preset neural network model, such as CNN network series, YOLO network series, SSD network series, etc. Further, considering that the loopback detection method is mainly used in mobile terminal devices such as robots, since the computing power of the system platform it carries is often limited, and the real-time performance of the loopback detection is also relatively high. To this end, this embodiment uses a lightweight network-based architecture for feature extraction. For example, the MoblieNet network can be used as the front-end feature network, and the back-end uses the YOLO network for target detection. Among them, when the MoblieNet network is used for characterization, higher-dimensional feature information can be obtained, which will speed up the subsequent loop closure detection.
  • a preset neural network model such as CNN network series, YOLO network series, SSD network series, etc.
  • this embodiment uses a lightweight network-based architecture for feature extraction.
  • the MoblieNet network can be used as the front-end feature
  • the eigenvector matrix includes several eigenvectors. It is worth noting that when extracting the features in the image frame, the corresponding feature vectors are formed according to the target position and category classification in the image. Among them, in the process of feature vectorization, if there are multiple identical objects in the image, it is also necessary to distinguish the identical objects in the image.
  • step S110 feature extraction is performed on the current image frame to obtain a corresponding feature vector matrix, including:
  • Sub-step S111 it is determined whether there are multiple identical objects in the current image frame. If there are multiple identical targets, execute sub-step S112; otherwise, execute sub-step S113.
  • Sub-step S112 if there is, then construct corresponding feature vectors in ascending or descending order based on the pixel distance from each identical target to the origin of the image coordinates, and construct corresponding feature vectors based on the positions of other different targets in the image.
  • the current image frame includes multiple identical objects A and one object B, wherein the object A includes A 1 , A 2 and A 3 , in order to distinguish the three identical objects in the image A.
  • it can be distinguished according to the pixel distances from A 1 , A 2 and A 3 to the origin of the image coordinates, that is, P(A 1 ) ⁇ P(A 2 ) ⁇ P(A3).
  • P(A 1 ), P(A 2 ), P(A3)] in ascending order.
  • the corresponding feature vector can be generated according to its position in the image combined with its category classification.
  • sub-step S113 if there is no such object, a corresponding feature vector is generated based on the position of each different target in the image.
  • the feature vector corresponding to each target is constructed according to the position of each target in the image.
  • Sub-step S114 constructing an eigenvector matrix using each of the obtained eigenvectors.
  • a feature vector matrix is formed by using the feature vectors of all the targets, which is the feature vector matrix of the current image frame.
  • Step S120 Hash calculation is performed on each feature vector according to the constructed locality-sensitive hash function to obtain a signature of each feature vector, and a hash table set of feature vector matrices is constructed according to the signatures of all feature vectors.
  • LSH Locality-Sensitive Hashing
  • the signature of each feature vector is generated by constructing a locality-sensitive hash function, and the signature between the feature vectors of the historical image frame and the current image frame is used to perform feature vector matching, thereby realizing loop closure detection.
  • the locality-sensitive hash function can be pre-built, for example, the locality-sensitive hash function can be defined according to the distance between two feature vectors.
  • the cosine distance may be used to measure the local-sensitive hash, and of course, other distances may also be used to measure, such as the Hamming distance, etc., which are not limited here.
  • the locality-sensitive hash function g is exemplarily defined as the product of the feature vector and the random hyperplane. If the product is greater than 0, the locality-sensitive hash value is 1; otherwise, the locality-sensitive hash value is 1. The value is 0.
  • k n-dimensional vectors [r 1 , r 2 ,...,r k ] can be randomly generated from the plane (that is, the random hyperplane ), and further, if the dot product of the feature vector V and the vector ri is greater than 0, the local-sensitive hash value of the i -th bit of the generated signature is 1, otherwise it is 0.
  • the signature S of the feature vector V is obtained, and the signature includes the k-bit locality-sensitive hash value.
  • the locality-sensitive hash function based on the construction can also use the "AND construction” (ie AND) and the “OR construction”. "(ie OR) one or both of these two strategies to construct a hash function family, through which hash function family can perform hash calculation, so that the constructed locality-sensitive hash function has the best Distance discrimination, so that feature vectors with similar distances generate the same signature, and large distances generate different signatures.
  • the hash function family F2 of the locality -sensitive hash function can be constructed by means of "OR construction".
  • Figures 4a and 4b illustrate the effect of distance discrimination between two vectors by way of "AND-constructed" and "OR-constructed", respectively.
  • a hash function family can be constructed by a cascade method of "AND construction” and then “OR construction”, and at this time, the distance discrimination effect as shown in FIG. 4c can be obtained. It can be seen from the three effects that for different vectors, the best distance discrimination can be obtained by the cascade strategy of "AND” and then “OR”, which means that vectors with similar distances will generate the same signature, while the distance Larger vectors will generate different signatures.
  • clustering processing can be performed according to the signatures of all feature vectors to construct a hash table set of feature vector matrices of the current image frame.
  • the feature vectors with the same signature are divided into the same class, which can speed up the matching query speed of the feature vectors, thereby improving the matching efficiency and so on.
  • the construction process of the hash table set in step S120 includes:
  • sub-step S121 the signatures of all feature vectors are stored in the signature set.
  • sub-step S122 the signatures in the signature set are clustered, and a hash table index of each cluster is generated.
  • Sub-step S123 construct a hash table set of the feature vector matrix according to the hash table indexes of all clusters.
  • the eigenvectors with the same signature can be classified into one class according to the signature calculated by each eigenvector through the local-sensitive hash function, that is, signature clustering is performed, and each cluster obtained by division has a corresponding index, By performing bit operations on these indices, the construction of the hash table is completed.
  • the set of hash tables obtained from the feature vector matrix will be used for subsequent matching with feature vectors of historical image frames, so as to select historical image frames where loop closure detection may occur.
  • Step S130 Screen several stored historical image frames to obtain a feature vector with a preset offset from the signature in the hash table set as the feature vector to be matched.
  • each historical image frame will be stored in a database to facilitate loop closure detection.
  • this embodiment will preliminarily screen out the historical image frames that may have loopbacks, so as to obtain a comparison between the current image frame and the current image frame. The feature vector to be matched for matching.
  • step S130 includes the following sub-steps:
  • Sub-step S131 perform feature extraction on several stored historical image frames to obtain corresponding historical feature vectors, and select historical feature vectors whose similarity with the feature vector of the current image frame exceeds a preset threshold as a candidate feature vector.
  • the same network model in step S110 can be used for feature extraction to obtain a feature vector matrix of each historical image frame, wherein the feature vector matrix of the historical image frame includes multiple historical feature vectors.
  • the similarity between each historical image frame and the current image frame can be compared, specifically, by calculating the difference between the historical feature vector and the feature vector of the current image frame.
  • Distance to calculate similarity such as Hamming distance. If the calculated similarity exceeds the preset distance threshold, it is preliminarily determined that the historical image frame corresponding to the historical feature vector has a possibility of loopback, so the historical feature vector is used as a candidate feature vector.
  • Sub-step S132 calculate the signature of each candidate feature vector through the local sensitive hash function, and find out the candidate feature vector with a preset bit offset between the signature and the signature in the hash table set as the feature to be matched vector.
  • the signatures corresponding to the candidate feature vectors can be calculated by the above-mentioned locality-sensitive hash function, and then the feature vectors to be matched can be selected according to the degree of offset between the signatures.
  • the signatures of these candidate feature vectors can be calculated through the hash function family, that is, the signature of each feature vector can be obtained by preferentially adopting the same hash calculation method.
  • each signature in the signature set P can be compared with the signatures in the hash table set T obtained in the above step S120. If there is a preset bit offset between the signatures, the candidate feature vectors corresponding to the signatures are stored in a set to be matched as the feature vectors to be matched.
  • the preset bit offset may be set according to empirical values obtained through a large number of tests.
  • Step S140 Determine whether the corresponding historical image frame has loopback according to the similarity between the feature vector of the current image frame and the feature vector to be matched, and output the historical image frame with loopback.
  • the similarity between the two feature vectors can be measured according to the cosine similarity of the feature vectors. For example, when the cosine similarity between the two feature vectors is greater than a preset threshold, it is determined that there is a loopback between the image frames corresponding to the current two feature vectors, so the historical image frames with loopbacks are output, and then these image frames with loopbacks are output. The output historical image frame and the current image frame are stored in the loopback image collection.
  • step S110 if there is no historical image frame with a similarity greater than the preset threshold, return to step S110 to acquire the next image frame and repeat the above steps S110-S140. It can be understood that by filtering out these looped images for map matching, it can help mobile terminal devices to perform obstacle avoidance and navigation more accurately and quickly.
  • the method further includes:
  • sub-step S210 all the found feature vectors to be matched are sequentially stored in the to-be-matched set according to the acquisition sequence of the corresponding historical image frames.
  • Sub-step S220 select from the to-be-matched set the to-be-matched feature vectors of each historical image frame whose frame interval with the current image frame is greater than the preset interval value, perform similarity calculation to determine whether there is loopback, and output the historical image frame with loopback .
  • the to-be-matched feature vectors corresponding to these historical image frames can be sequentially stored in the to-be-matched set according to the acquisition order of each historical image frame, and then, if the preset interval value is set to w, then the current image frame will be matched with the current image frame.
  • the feature vectors to be matched corresponding to the interval w frames and all historical image frames greater than w frames are used for similarity calculation.
  • the historical image frames in the interval less than w frames are not used for calculation. By not participating in the similarity comparison of frames with a closer interval with the current image frame, the calculation amount can be further reduced.
  • the loopback detection method of this embodiment extracts high-dimensional feature information of an image through a lightweight network, and adopts strategies such as a local-sensitive hash function and pre-screening historical frame data to achieve fast loopback detection.
  • the hash function family is constructed by the strategy of "and construction” and "or construction” to increase the distance discrimination of feature vectors.
  • this embodiment provides a loop closure detection apparatus 100 .
  • the loop closure detection apparatus 100 includes:
  • the feature extraction module 110 is configured to obtain a current image frame, and perform feature extraction on the current image frame to obtain a corresponding feature vector matrix.
  • the hash calculation module 120 is configured to perform hash calculation on each eigenvector in the eigenvector matrix according to the constructed locality-sensitive hash function to obtain the signature of each eigenvector, and the signatures of all eigenvectors are used for Build a hash table set of the eigenvector matrix.
  • the candidate matching module 130 is configured to screen several stored historical image frames to obtain a candidate feature vector with a preset offset from the signature in the hash table set as the feature vector to be matched.
  • the loopback judgment module 140 is configured to judge whether the corresponding historical image frame has loopback according to the similarity between the feature vector of the current image frame and the feature vector to be matched, and output the historical image frame with loopback.
  • the terminal device may include a robot, an unmanned vehicle, an intelligent positioning backpack for outdoor positioning, and the like.
  • the terminal device includes a processor and a memory, wherein the memory stores a computer program, and when executed by the processor, the computer program implements the above-mentioned loop closure detection method.
  • Yet another embodiment of the present application further provides a readable storage medium, which stores a computer program, and when the computer program is implemented by a processor, executes the above-mentioned loop closure detection method.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of code that contains one or more functions for implementing the specified logical function(s) executable instructions. It should also be noted that, in alternative implementations, the functions noted in the block may occur out of the order noted in the figures.
  • each block of the block diagrams and/or flow diagrams, and combinations of blocks in the block diagrams and/or flow diagrams can be implemented using dedicated hardware-based systems that perform the specified functions or actions. be implemented, or may be implemented in a combination of special purpose hardware and computer instructions.
  • each functional module or unit in each embodiment of the present application may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.
  • the functions are implemented in the form of software function modules and sold or used as independent products, they can be stored in a computer-readable storage medium.
  • the technical solution of the present application can be embodied in the form of a software product in essence, or the part that contributes to the prior art or the part of the technical solution.
  • the computer software product is stored in a storage medium, including Several instructions are used to cause a computer device (which may be a smart phone, a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application.
  • the aforementioned storage medium includes: U disk, mobile hard disk, Read-Only Memory (ROM, Read-Only Memory), Random Access Memory (RAM, Random Access Memory), magnetic disk or optical disk and other media that can store program codes .

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Evolutionary Computation (AREA)
  • General Engineering & Computer Science (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Biomedical Technology (AREA)
  • Health & Medical Sciences (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

本申请实施例提供一种回环检测方法、装置、终端设备和可读存储介质，该方法包括：获取当前图像帧，对当前图像帧进行特征提取以得到特征向量矩阵；根据构建的局部敏感哈希函数计算对应特征向量的签名并构建特征向量矩阵的哈希表集合；对若干历史图像帧进行筛选得到哈希表集合中的签名之间存在预设位偏移量的特征向量作为待匹配特征向量；根据相似度判断待匹配特征向量对应的历史图像帧是否存在回环，输出存在回环的历史图像帧。本申请的方案可提高终端设备的回环实时性，有效解决终端设备在移动过程中产生的轨迹漂移问题等。

Description

回环检测方法、装置、终端设备和可读存储介质
相关申请的交叉引用
本申请要求于2020年12月31日提交中国专利局的申请号为2020116326425、名称为“回环检测方法、装置、终端设备和可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及图像处理技术领域，尤其涉及一种回环检测方法、装置、终端设备和可读存储介质。
背景技术
回环检测，又称闭环检测，例如，对于机器人而言，是指机器人识别曾到达某场景，使得地图闭环的能力。换句话说，就是机器人在左转一下，右转一下进行建图的时候，能够意识到某个地方是自己曾经经过的，然后把此刻生成的地图与刚刚生成的地图做匹配。
传统的视觉SLAM(simultaneous localization and mapping，同步定位与建图)技术在实现回环检测时是通过收集大量的图片，然后采用特征提取方法提取特征点，接着采用一些分类方法，将这些特征点分为不同的类型，建立特征点集合字典。对于机器人采集的新图片，再次提取特征点，然后查询特征点字典。判断该特征点是属于哪一个类型，进而根据两张图片的相似度来检测回环，也称为视觉词袋模型。
然而，由于存在大量的图像，上述的利用图像的像素点提取及字典查询的方式存在耗时长的问题，这使得机器人在移动过程不能实时准确地判断识别自己是否到过这个地方，导致位姿漂移，进而给位姿局部优化造成困扰等等。
申请内容
有鉴于此，本申请的目的是为了克服现有技术中的不足，提供一种回环检测方法、装置、终端设备和可读存储介质。
本申请的实施例提供一种回环检测方法，包括：
获取当前图像帧，对所述当前图像帧进行特征提取以得到对应的特征向量矩阵；
根据构建的局部敏感哈希函数对所述特征向量矩阵中的每个特征向量进行哈希计算以得到对应特征向量的签名，并以所有特征向量的签名构建所述特征向量矩阵的哈希表集合；
对存储的若干历史图像帧进行筛选得到与所述哈希表集合中的签名之间存在预设位偏移量的特征向量作为待匹配特征向量；
根据所述当前图像帧的特征向量与所述待匹配特征向量的相似度判断对应的历史图像帧是否存在回环，输出存在回环的历史图像帧。
在一种实施例中，在查找出待匹配特征向量之后，该方法还包括：
将查找到的所有待匹配特征向量按照对应历史图像帧的获取顺序依次存储于待匹配集合中；
从所述待匹配集合中选取与所述当前图像帧的帧间隔数大于预设间隔值的各历史图像帧的待匹配特征向量进行相似度计算以判断是否存在回环。
在一种实施例中，利用构建的局部敏感哈希函数计算特征向量的签名时，该方法还包括：
根据“与构造”和“或构造”中的一种或两种方式生成所述局部敏感哈希函数的哈希函数族，所述哈希函数族用于计算特征向量的签名。
在上述实施例中，生成所述哈希函数族时，所述方法包括：
按照先“与构造”后“或构造”的级联方式生成所述局部敏感哈希函数的哈希函数族。
在一种实施例中，基于余弦距离构建所述局部敏感哈希函数。
在一种实施例中，所述对所述当前图像帧进行特征提取以得到对应的特征向量矩阵，包括：
判断所述当前图像帧中是否存在多个相同目标；
若存在多个相同目标，则根据各个所述相同目标到图像坐标原点的像素距离按照升序或降序方式构建对应的特征向量，以及根据其他的不同目标在图像中的位置构建对应的特征向量；
若不存在，则根据各个不同目标在图像中的位置构建对应的特征向量；
利用生成的每一特征向量构建一特征向量矩阵。
在一种实施例中，通过MoblieNet神经网络对所述当前图像帧或所述历史图像帧进行特征提取。
在一种实施例中，所述以所有特征向量的签名构建所述特征向量矩阵的哈希表集合，包括：
将每个特征向量的签名存储于签名集合中；
对所述签名集合中的签名进行聚类，并生成每个聚类的哈希表索引；
根据所有聚类的所述哈希表索引构建所述特征向量矩阵的哈希表集合。
在一种实施例中，所述对存储的若干历史图像帧进行筛选得到与所述哈希表集合中的签名之间存在预设位偏移量的特征向量作为待匹配特征向量，包括：
对存储的若干历史图像帧分别进行特征提取以得到对应的历史特征向量，并从中选取出与所述当前图像帧的特征向量的相似度超过预设阈值的历史特征向量以作为候选特征向量；
通过所述局部敏感哈希函数计算每个候选特征向量的签名，并查找出签名与所述哈希表集合中的签名之间存在预设位偏移量的候选特征向量以作为待匹配特征向量。
本申请的实施例还提供一种回环检测装置，包括：
特征提取模块，用于获取当前图像帧，对所述当前图像帧进行特征提取以得到对应的特征向量矩阵；
哈希计算模块，用于根据构建的局部敏感哈希函数对所述特征向量矩阵中的每个特征向量进行哈希计算以得到每个特征向量的签名，并以所有特征向量的签名构建所述特征向量矩阵的哈希表集合；
候选匹配模块，用于对存储的若干历史图像帧进行筛选得到与所述哈希表集合中的签名之间存在预设位偏移量的特征向量作为待匹配特征向量；
回环判断模块，用于根据所述当前图像帧的特征向量与所述待匹配特征向量的相似度判断对应的历史图像帧是否存在回环，输出存在回环的历史图像帧。
本申请的实施例还提供一种终端设备，所述终端设备包括处理器和存储器，所述存储器存储有计算机程序，所述计算机程序在所述处理器上执行时，实施上述的回环检测方法。
在一种实施例中，所述终端设备为机器人、无人驾驶汽车或智能定位背包。
本申请的实施例还提供一种可读存储介质，其存储有计算机程序，所述计算机程序在处理器上被执行时，实施上述的回环检测方法。
本申请的实施例具有如下有益效果：
本申请实施例的回环检测方法通过提取出图像的特征信息，并采用局部敏感哈希函数以及对历史帧数据进行预筛选等策略来实现快速回环检测，可以大大提高终端设备的回环实时性，进而有效解决终端设备在移动过程中产生的轨迹漂移问题，提高终端设备的鲁棒性等。
附图说明
为了更清楚地说明本申请实施例的技术方案，下面将对实施例中所需要使用的附图作简单地介绍，应当理解，以下附图仅示出了本申请的某些实施例，因此不应被看作是对范围的限定，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据这些附图获得其他相关的附图。
图1示出了本申请实施例的回环检测方法的一种流程示意图；
图2示出了本申请实施例的回环检测方法的特征提取流程示意图；
图3示出了本申请实施例的回环检测方法的特征提取应用示意图；
图4a、图4b和图4c分别示出了本申请实施例的回环检测方法的“与构造”方式、“或构造”方式、先“与构造”后“或构造”方式的效果图；
图5示出了本申请实施例的回环检测方法的哈希表生成流程示意图；
图6示出了本申请实施例的回环检测方法的历史图像帧筛选的流程示意图；
图7示出了本申请实施例的回环检测方法的另一流程示意图；
图8示出了本申请实施例的回环检测装置的结构示意图。
具体实施方式
下面将结合本申请实施例中附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。
通常在此处附图中描述和示出的本申请实施例的组件可以以各种不同的配置来布置和设计。因此，以下对在附图中提供的本申请的实施例的详细描述并非旨在限制要求保护的本申请的范围，而是仅仅表示本申请的选定实施例。基于本申请的实施例，本领域技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例，都属于本申请保 护的范围。
除非另有限定，否则在这里使用的所有术语(包括技术术语和科学术语)具有与本申请的各种实施例所属领域普通技术人员通常理解的含义相同的含义。所述术语(诸如在一般使用的词典中限定的术语)将被解释为具有与在相关技术领域中的语境含义相同的含义并且将不被解释为具有理想化的含义或过于正式的含义，除非在本申请的各种实施例中被清楚地限定。
实施例1
图1示出了本申请实施例的回环检测方法的一种流程示意图。该回环检测方法可应用于如机器人、无人驾驶汽车等终端，以实现即时定位及地图构建等。示范性地，该回环检测方法包括：
步骤S110，获取当前图像帧，对当前图像帧进行特征提取以得到对应的特征向量矩阵。
当终端获取到一帧图像时，可通过预设的神经网络模型对该当前图像帧进行特征提取，如CNN网络系列、YOLO网络系列、SSD网络系列等。进一步地，考虑到该回环检测方法主要运用于如机器人等移动式终端设备中，由于其搭载的系统平台的运算能力往往有限，同时对回环检测的实时性也要求较高。为此，本实施例将采用一种基于轻量级网络的架构进行特征提取，例如，可使用MoblieNet网络作为前端的特征化网络，而后端使用YOLO网络进行目标检测等。其中，利用MoblieNet网络进行特征化时，可得到更高维度的特征信息，这将加快后续回环检测的速度等。
其中，该特征向量矩阵包括若干个特征向量。值得注意的是，在提取该图像帧中的特征时，根据图像中的目标位置及类别分类，构成得到相应的特征向量。其中，在进行特征向量化的过程中，若图像中存在多个相同的目标，还需要将对图像中的相同目标进行区分。
如图2所示，对于步骤S110，对当前图像帧进行特征提取以得到对应的特征向量矩阵，包括：
子步骤S111，判断当前图像帧中是否存在多个相同的目标。若存在多个相同的目标，则执行子步骤S112；否则，执行子步骤S113。
子步骤S112，若存在，则基于各个相同目标到图像坐标原点的像素距离按照升序或降序方式构建对应的特征向量，以及基于其他的不同目标在图像中的位置构建对应的特 征向量。
例如，如图3所示，若当前图像帧中包括了多个相同的目标A和一个目标B，其中，目标A包括A 1、A 2和A 3，为了区分图像中的这三个相同目标A，在进行特征向量化时，可按照A 1、A 2和A 3分别到图像坐标原点的像素距离来进行区分，即有P(A 1)<P(A 2)<P(A3)。进而，根据A 1、A 2和A 3的距离大小按照升序保存为[P(A 1)，P(A 2)，P(A3)]，当然也可以按照降序来保存，从而生成目标A对应的特征向量。而对于其他的不同目标，如B等，则可以按照其在图像中的位置结合其类别分类等生成对应的特征向量。
子步骤S113，若不存在，则基于各个不同目标在图像中的位置生成对应的特征向量。
示范性地，若每个目标在图像中的数量都只有一个，即不存在多个相同的目标，则按照每个目标在图像中的位置来构建各目标对应的特征向量。
子步骤S114，利用得到的每一特征向量构建一特征向量矩阵。
示范性地，将图像中的各个目标进行特征化后，利用所有目标的特征向量构成一个特征向量矩阵，即为当前图像帧的特征向量矩阵。
步骤S120，根据构建的局部敏感哈希函数对每个特征向量进行哈希计算以得到每个特征向量的签名，并根据所有特征向量的签名构建特征向量矩阵的哈希表集合。
其中，局部敏感哈希(Locality-Sensitive Hashing，LSH)可运用于解决近似或最近邻问题，例如，如果原来的两个数据相似，那么经过哈希计算得到的哈希值(又称哈希签名)也保持一定的相似性。本实施例中，将通过构建局部敏感哈希函数来生成各个特征向量的签名，并利用历史图像帧与当前图像帧的特征向量之间的签名进行特征向量匹配，进而实现回环检测。
示范性地，该局部敏感哈希函数可预先构建得到，例如，可根据两个特征向量之间的距离来定义该局部敏感哈希函数。在一种实施方式中，可采用余弦距离来度量局部敏感哈希，当然，也可以采用其他的距离来度量，如海明距离等，在此并不作限定。
以余弦距离为例，示范性地，将局部敏感哈希函数g定义为特征向量与随机超平面的乘积，若该乘积大于0，则该局部敏感哈希值则为1，否则局部敏感哈希值为0。例如，对于一个n维特征向量V，若要得到一个k位的签名(k<n)，可由平面随机产生k个n维向量[r 1,r 2,…,r k](即随机超平面)，进而，若特征向量V与向量r i的点积大于0，则生成的签名的第i位的局部敏感哈希值为1，否则为0。由此，得到该特征向量V的签名S，该签名包括k位局部敏感哈希值。
作为另一种可选的方式，为增加匹配向量的查询范围以及提高签名的聚类区分度等，基于构造的局部敏感哈希函数，还可以利用“与构造”(即AND)和“或构造”(即OR)这两种策略中的一种或两种的方式来构造一个哈希函数族，通过该哈希函数族进行哈希计算，可以使构造的局部敏感哈希函数具有最好的距离区分度，进而使距离相近的特征向量生成相同的签名，距离大的生成不一样的签名。
在第一种实施方式中，可通过“与构造”方式构造该局部敏感哈希函数的哈希函数族F1，若记为F1＝{g i|i＝1,2,…,k}，其中，g i表示第i个局部敏感哈希函数，对于空间中任意两个向量V 1、V 2，当且仅当每个g i(V 1)＝g i(V 2)时，则存在g(V 1)＝g(V 2)。
在第二种实施方式中，可通过“或构造”方式构造该局部敏感哈希函数的哈希函数族F2，若记为F2＝{g i|i＝1,2,…,k}，此时，对于空间中任意两个向量V 1、V 2，只要任意一个g i(V 1)＝g i(V 2)时，即会存在g(V 1)＝g(V 2)。图4a和图4b示出了分别通过“与构造”和“或构造”方式的两个向量之间的距离区分度的效果。
而在第三种实施方式中，可通过先“与构造”后“或构造”的级联方式来构造哈希函数族，此时可得到如图4c所示的距离区分度效果。从三种效果可以看出，对于不同的向量，通过先“AND”后“OR”的级联策略可以得到最好的距离区分度，即意味着距离相近的向量将生成相同的签名，而距离较大的向量则将生成不一样的签名。
于是，在得到各个特征向量的签名后，可根据所有特征向量的签名进行聚类处理，构建当前图像帧的特征向量矩阵的哈希表集合。通过聚类处理，将具有相同签名的特征向量划分为同一类，这样可以加快特征向量的匹配查询速度，从而提高匹配效率等。
示范性地，如图5所示，步骤S120中的哈希表集合的构建过程，包括：
子步骤S121，将所有特征向量的签名存储于签名集合中。
子步骤S122，对签名集合中的签名进行聚类，并生成每个聚类的哈希表索引。
子步骤S123，根据所有聚类的哈希表索引构建该特征向量矩阵的哈希表集合。
示范性地，可根据每个特征向量通过局部敏感哈希函数计算得到的签名，将具有相同签名的特征向量归为一类，即进行签名聚类，划分得到的各聚类具有相应的索引，通过对这些索引进行位运算，完成哈希表的构建。
例如，若签名集合记为S＝{S 1,S 2,…,S m}，m为特征向量的数量，其中，第i个特征向量的签名表示为S i，若多个相同签名的被划分为一类，记为G j，通过二进制的位运算将G j转化为十进制的数a j，a j作为第j个哈希表的索引，最后得到一个哈希表集合 T＝{a 1,a 1,…,a j,…,a l}，其中，l为聚类数量。
于是，得到该特征向量矩阵的哈希表集合将用于后续的与历史图像帧的特征向量进行匹配，从而选取出可能出现回环检测的历史图像帧。
步骤S130，对存储的若干历史图像帧进行筛选得到与哈希表集合中的签名之间存在预设位偏移量的特征向量作为待匹配特征向量。
通常地，各个历史图像帧将会存储在数据库，以方便进行回环检测。考虑到存储的历史图像帧可能数量比较多，为了减少后续的回环检测的运算量并提高检测效率，本实施例将初步筛选出可能存在回环的历史图像帧，以得到用于与当前图像帧的特征向量进行匹配的待匹配特征向量。
在一种实施方式中，如图6所示，步骤S130包括以下子步骤：
子步骤S131，对存储的若干历史图像帧分别进行特征提取以得到对应的历史特征向量，并从中选取出与当前图像帧的特征向量的相似度超过预设阈值的历史特征向量作为候选特征向量。
示范性地，对于各个历史图像帧，可采用步骤S110中的相同网络模型进行特征提取，得到各历史图像帧的特征向量矩阵，其中，历史图像帧的特征向量矩阵包括多个历史特征向量。
为筛选出可能存在回环的候选特征向量，示范性地，可根据各历史图像帧分别与当前图像帧进行相似度比较，具体地，可通过计算历史特征向量与当前图像帧的特征向量之间的距离来计算相似度，如海明距离等。若计算得到的相似度超过预设的距离阈值，即初步判定该历史特征向量对应的历史图像帧存在回环的可能性，故将该历史特征向量作为候选特征向量。
子步骤S132，通过局部敏感哈希函数计算每个候选特征向量的签名，并查找出签名与该哈希表集合中的签名之间存在预设位偏移量的候选特征向量以作为待匹配特征向量。
示范性地，对于筛选出的这些候选特征向量，可通过上述的局部敏感哈希函数来计算对应候选特征向量的签名，进而根据签名之间的偏移程度来选取出待匹配的特征向量。可选地，若上述构造得到哈希函数族，则可通过哈希函数族来计算这些候选特征向量的签名，即优先采用相同的哈希计算方式得到各特征向量的签名。
例如，若这些候选特征向量的签名集合记为P，可将该签名集合P中的各个签名与 上述步骤S120得到的哈希表集合T中的签名进行逐位局部敏感哈希值比较，若两个签名之间存在预设的位偏移量，则将签名对应的候选特征向量存储到一待匹配集合中以作为待匹配特征向量。其中，该预设的位偏移量可根据通过大量测试得到的经验值来设置。
步骤S140，根据当前图像帧的特征向量与待匹配特征向量的相似度判断对应的历史图像帧是否存在回环，输出存在回环的历史图像帧。
在得到若干待匹配特征向量后，示范性地，可根据特征向量的余弦相似度来度量两个特征向量之间的相似性。例如，当两个特征向量之间的余弦相似度大于一预设阈值时，则判断当前两个特征向量所对应的图像帧之间存在回环，于是，输出存在回环的历史图像帧，进而将这些输出的历史图像帧与当前图像帧存储于回环图像集合中。
可选地，若不存在相似度大于该预设阈值的历史图像帧，则返回至步骤S110以获取下一图像帧并重复执行上述步骤S110-S140。可以理解，通过筛选出这些存在回环的图像进行地图匹配，可帮助移动式的终端设备更精准、快速的进行避障导航工作等。
进一步可选地，对于由步骤S130得到的待匹配特征向量，考虑到与当前图像帧的相邻几个历史图像帧由于间隔较短，其相似度通常较高，为了进一步减少匹配率，如图7所示，该方法还包括：
子步骤S210，将查找到的所有待匹配特征向量按照对应历史图像帧的获取顺序依次存储于待匹配集合中。
子步骤S220，从待匹配集合中选取与当前图像帧的帧间隔数大于预设间隔值的各历史图像帧的待匹配特征向量进行相似度计算以判断是否存在回环，输出存在回环的历史图像帧。
示范性地，可按照各历史图像帧的获取顺序依次存储这些历史图像帧对应的待匹配特征向量到待匹配集合中，进而，若该预设间隔值设置为w，则将与该当前图像帧间隔w帧及大于w帧的所有历史图像帧对应的待匹配特征向量用于相似度计算。而对于小于w帧这一区间内的历史图像帧则不用于计算，通过将与当前图像帧间隔较近的帧不参与相似度比较，这样可以进一步减少运算量等。
本实施例的回环检测方法通过轻量级网络来提取出图像的高维度特征信息，并采用局部敏感哈希函数以及对历史帧数据进行预筛选等策略来实现快速回环检测，此外，还可利用通过“与构造”、“或构造”的策略来构造哈希函数族以用于增加特征向量的距离区分度，对于如机器人、无人驾驶汽车等设备的回环检测过程，不仅可以大大提高 终端设备的回环实时性，还可提高回环准确性，进而有效解决终端设备在移动过程中产生的轨迹漂移问题，从而提高了终端设备的鲁棒性等。
实施例2
请参照图8，基于上述实施例1的方法，本实施例提出一种回环检测装置100，示范性地，该回环检测装置100包括：
特征提取模块110，用于获取当前图像帧，对所述当前图像帧进行特征提取以得到对应的特征向量矩阵。
哈希计算模块120，用于根据构建的局部敏感哈希函数对所述特征向量矩阵中的每个特征向量进行哈希计算以得到每个特征向量的签名，所有特征向量的所述签名用于构建所述特征向量矩阵的哈希表集合。
候选匹配模块130，用于对存储的若干历史图像帧进行筛选得到与所述哈希表集合中的签名之间存在预设位偏移量的候选特征向量作为待匹配特征向量。
回环判断模块140，用于根据所述当前图像帧的特征向量与所述待匹配特征向量的相似度判断对应的历史图像帧是否存在回环，输出存在回环的历史图像帧。
可以理解，本实施例的装置对应于上述实施例1的方法，上述实施例1中的可选项同样适用于本实施例，故在此不再重复描述。
本申请的另一实施例还提出一种终端设备，例如，该终端设备可包括机器人、无人驾驶汽车、用于户外定位的智能定位背包等。示范性地，该终端设备包括处理器和存储器，其中，存储器存储有计算机程序，该计算机程序在被处理器执行时，实施上述的回环检测方法。
本申请的又一实施例还提供了一种可读存储介质，其存储有计算机程序，在计算机程序被处理器实施时，执行上述的回环检测方法。
在本申请所提供的几个实施例中，应该理解到，所揭露的装置和方法，也可以通过其它的方式实现。以上所描述的装置实施例仅仅是示意性的，例如，附图中的流程图和结构图显示了根据本申请的多个实施例的装置、方法和计算机程序产品的可能实现的体系架构、功能和操作。在这点上，流程图或框图中的每个方框可以代表一个模块、程序段或代码的一部分，所述模块、程序段或代码的一部分包含一个或多个用于实现规定的逻辑功能的可执行指令。也应当注意，在作为替换的实现方式中，方框中所标注的功能 也可以以不同于附图中所标注的顺序发生。例如，两个连续的方框实际上可以基本并行地执行，它们有时也可以按相反的顺序执行，这依所涉及的功能而定。也要注意的是，结构图和/或流程图中的每个方框、以及结构图和/或流程图中的方框的组合，可以用执行规定的功能或动作的专用的基于硬件的系统来实现，或者可以用专用硬件与计算机指令的组合来实现。
另外，在本申请各个实施例中的各功能模块或单元可以集成在一起形成一个独立的部分，也可以是各个模块单独存在，也可以两个或更多个模块集成形成一个独立的部分。
所述功能如果以软件功能模块的形式实现并作为独立的产品销售或使用时，可以存储在一个计算机可读取存储介质中。基于这样的理解，本申请的技术方案本质上或者说对现有技术做出贡献的部分或者该技术方案的部分可以以软件产品的形式体现出来，该计算机软件产品存储在一个存储介质中，包括若干指令用以使得一台计算机设备(可以是智能手机、个人计算机、服务器、或者网络设备等)执行本申请各个实施例所述方法的全部或部分步骤。而前述的存储介质包括：U盘、移动硬盘、只读存储器(ROM，Read-Only Memory)、随机存取存储器(RAM，Random Access Memory)、磁碟或者光盘等各种可以存储程序代码的介质。
以上所述，仅为本申请的具体实施方式，但本申请的保护范围并不局限于此，任何熟悉本技术领域的技术人员在本申请揭露的技术范围内，可轻易想到变化或替换，都应涵盖在本申请的保护范围之内。

Claims (13)

  1. 一种回环检测方法，其特征在于，包括：
    获取当前图像帧，对所述当前图像帧进行特征提取以得到对应的特征向量矩阵；
    根据构建的局部敏感哈希函数对所述特征向量矩阵中的每个特征向量进行哈希计算以得到对应特征向量的签名，并以所有特征向量的签名构建所述特征向量矩阵的哈希表集合；
    对存储的若干历史图像帧进行筛选得到与所述哈希表集合中的签名之间存在预设位偏移量的特征向量作为待匹配特征向量；
    根据所述当前图像帧的特征向量与所述待匹配特征向量的相似度判断对应的历史图像帧是否存在回环，输出存在回环的历史图像帧。
  2. 根据权利要求1所述的回环检测方法，其特征在于，在查找出待匹配特征向量之后，所述方法还包括：
    将查找到的所有待匹配特征向量按照对应历史图像帧的获取顺序依次存储于待匹配集合中；
    从所述待匹配集合中选取与所述当前图像帧的帧间隔数大于预设间隔值的各历史图像帧的待匹配特征向量进行相似度计算以判断是否存在回环。
  3. 根据权利要求1所述的回环检测方法，其特征在于，利用构建的局部敏感哈希函数计算特征向量的签名时，所述方法还包括：
    根据“与构造”和“或构造”中的一种或两种方式生成所述局部敏感哈希函数的哈希函数族，所述哈希函数族用于计算特征向量的签名。
  4. 根据权利要求3所述的回环检测方法，其特征在于，生成所述哈希函数族时，所述方法包括：
    按照先“与构造”后“或构造”的级联方式生成所述局部敏感哈希函数的哈希函数族。
  5. 根据权利要求1至4中任一项所述的回环检测方法，其特征在于，基于余弦距离构建所述局部敏感哈希函数。
  6. 根据权利要求1所述的回环检测方法，其特征在于，所述对所述当前图像帧进行特征提取以得到对应的特征向量矩阵，包括：
    判断所述当前图像帧中是否存在多个相同目标；
    若存在多个相同目标，则根据各个所述相同目标到图像坐标原点的像素距离按照升序或降序方式构建对应的特征向量，以及根据其他的不同目标在图像中的位置构建对应的特征向量；
    若不存在，则根据各个不同目标在图像中的位置构建对应的特征向量；
    利用生成的每一特征向量构建一特征向量矩阵。
  7. 根据权利要求1或6所述的回环检测方法，其特征在于，通过MoblieNet神经网络对所述当前图像帧或所述历史图像帧进行特征提取。
  8. 根据权利要求1所述的回环检测方法，其特征在于，所述以所有特征向量的签名构建所述特征向量矩阵的哈希表集合，包括：
    将每个特征向量的签名存储于签名集合中；
    对所述签名集合中的签名进行聚类，并生成每个聚类的哈希表索引；
    根据所有聚类的所述哈希表索引构建所述特征向量矩阵的哈希表集合。
  9. 根据权利要求1所述的回环检测方法，其特征在于，所述对存储的若干历史图像帧进行筛选得到与所述哈希表集合中的签名之间存在预设位偏移量的特征向量作为待匹配特征向量，包括：
    对存储的若干历史图像帧分别进行特征提取以得到对应的历史特征向量，并从中选取出与所述当前图像帧的特征向量的相似度超过预设阈值的历史特征向量以作为候选特征向量；
    通过所述局部敏感哈希函数计算每个候选特征向量的签名，并查找出签名与所述哈希表集合中的签名之间存在预设位偏移量的候选特征向量以作为待匹配特征向量。
  10. 一种回环检测装置，其特征在于，包括：
    特征提取模块，用于获取当前图像帧，对所述当前图像帧进行特征提取以得到对应的特征向量矩阵；
    哈希计算模块，用于根据构建的局部敏感哈希函数对所述特征向量矩阵中的每个特征向量进行哈希计算以得到每个特征向量的签名，并以所有特征向量的签名构建所述特征向量矩阵的哈希表集合；
    候选匹配模块，用于对存储的若干历史图像帧进行筛选得到与所述哈希表集合中的签名之间存在预设位偏移量的特征向量作为待匹配特征向量；
    回环判断模块，用于根据所述当前图像帧的特征向量与所述待匹配特征向量的相似 度判断对应的历史图像帧是否存在回环，输出存在回环的历史图像帧。
  11. 一种终端设备，其特征在于，所述终端设备包括处理器和存储器，所述存储器存储有计算机程序，所述计算机程序在所述处理器上执行时，实施权利要求1-9中任一项所述的回环检测方法。
  12. 根据权利要求11所述的终端设备，其特征在于，所述终端设备为机器人、无人驾驶汽车或智能定位背包。
  13. 一种可读存储介质，其特征在于，其存储有计算机程序，所述计算机程序在处理器上被执行时，实施根据权利要求1-9中任一项所述的回环检测方法。
PCT/CN2021/132115 2020-12-31 2021-11-22 回环检测方法、装置、终端设备和可读存储介质 WO2022142855A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011632642.5 2020-12-31
CN202011632642.5A CN112614187B (zh) 2020-12-31 2020-12-31 回环检测方法、装置、终端设备和可读存储介质

Publications (1)

Publication Number Publication Date
WO2022142855A1 true WO2022142855A1 (zh) 2022-07-07

Family

ID=75252933

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/132115 WO2022142855A1 (zh) 2020-12-31 2021-11-22 回环检测方法、装置、终端设备和可读存储介质

Country Status (2)

Country Link
CN (1) CN112614187B (zh)
WO (1) WO2022142855A1 (zh)

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115242402A (zh) * 2022-07-12 2022-10-25 长春吉大正元信息技术股份有限公司 签名方法、验签方法及电子设备
CN115982399A (zh) * 2023-03-16 2023-04-18 北京集度科技有限公司 图像查找方法、移动设备、电子设备、及计算机程序产品
CN116304163A (zh) * 2023-05-11 2023-06-23 深圳兔展智能科技有限公司 一种图像检索方法、装置、计算机设备及介质
CN116823816A (zh) * 2023-08-28 2023-09-29 济南正邦电子科技有限公司 一种基于安防监控静态存储器的检测设备及检测方法
CN116883879A (zh) * 2023-09-04 2023-10-13 苏州浪潮智能科技有限公司 一种闭环检测的方法、装置及可读存储介质
CN116958266A (zh) * 2023-09-19 2023-10-27 苏州元脑智能科技有限公司 闭环检测方法、装置、通信设备及存储介质
CN117056086A (zh) * 2023-10-11 2023-11-14 国网山东省电力公司滨州市滨城区供电公司 基于排列熵算法的故障检测方法、系统、终端及存储介质
CN117173247A (zh) * 2023-11-02 2023-12-05 中国海洋大学 基于2D激光雷达与LightGBM的室外定位与构图方法及系统

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112614187B (zh) * 2020-12-31 2024-03-26 深圳市优必选科技股份有限公司 回环检测方法、装置、终端设备和可读存储介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20180113060A (ko) * 2017-04-05 2018-10-15 충북대학교 산학협력단 그래프 slam을 위한 키프레임 추출방법 및 이를 이용한 slam 장치
CN109871803A (zh) * 2019-02-18 2019-06-11 清华大学 机器人回环检测方法和装置
CN111598149A (zh) * 2020-05-09 2020-08-28 鹏城实验室 一种基于注意力机制的回环检测方法
CN112614187A (zh) * 2020-12-31 2021-04-06 深圳市优必选科技股份有限公司 回环检测方法、装置、终端设备和可读存储介质

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102622366B (zh) * 2011-01-28 2014-07-30 阿里巴巴集团控股有限公司 相似图像的识别方法和装置
CN103516682B (zh) * 2012-06-19 2016-12-21 Tcl集团股份有限公司 一种基于人脸追踪的身份认证方法、系统及电视机
US9996764B2 (en) * 2014-04-29 2018-06-12 Institute Of Automation Chinese Academy Of Sciences Image matching method based on cascaded binary encoding
CN104574440A (zh) * 2014-12-30 2015-04-29 安科智慧城市技术（中国）有限公司 一种视频运动目标跟踪方法及装置
CN114973677A (zh) * 2016-04-18 2022-08-30 北京嘀嘀无限科技发展有限公司 用于确定预计到达时间的方法和设备
CN107396112B (zh) * 2017-08-01 2021-03-09 深信服科技股份有限公司 一种编码方法及装置、计算机装置、可读存储介质
CN107885764B (zh) * 2017-09-21 2020-12-18 银江股份有限公司 基于多任务深度学习快速哈希车辆检索方法
CN109101981B (zh) * 2018-07-19 2021-08-24 东南大学 一种街景场景下基于全局图像条纹码的回环检测方法
CN110362627A (zh) * 2019-06-18 2019-10-22 深圳壹账通智能科技有限公司 基于区块链的业务分析方法及装置、电子设备、存储介质
CN111125093B (zh) * 2019-11-19 2023-09-15 贝壳技术有限公司 一种基于哈希存储的数据处理方法及装置
CN111862162B (zh) * 2020-07-31 2021-06-11 湖北亿咖通科技有限公司 回环检测方法及系统、可读存储介质、电子设备

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20180113060A (ko) * 2017-04-05 2018-10-15 충북대학교 산학협력단 그래프 slam을 위한 키프레임 추출방법 및 이를 이용한 slam 장치
CN109871803A (zh) * 2019-02-18 2019-06-11 清华大学 机器人回环检测方法和装置
CN111598149A (zh) * 2020-05-09 2020-08-28 鹏城实验室 一种基于注意力机制的回环检测方法
CN112614187A (zh) * 2020-12-31 2021-04-06 深圳市优必选科技股份有限公司 回环检测方法、装置、终端设备和可读存储介质

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
YU YU, ET AL.: "Loop closure detection method based on deep learning for visual SLAM", COMPUTER ENGINEERING AND DESIGN, vol. 41, no. 2, 29 February 2020 (2020-02-29), pages 529 - 536, XP055947973, ISSN: 1000-7024, DOI: 10.16208/j.issn1000-7024.2020.02.035 *
YU YU: "Research on Fast Loop Closure Detection Based on Deep Learning for Visual SLAM", MASTER THESIS, TIANJIN POLYTECHNIC UNIVERSITY, CN, no. 1, 15 January 2020 (2020-01-15), CN , XP055947969, ISSN: 1674-0246 *

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115242402A (zh) * 2022-07-12 2022-10-25 长春吉大正元信息技术股份有限公司 签名方法、验签方法及电子设备
CN115242402B (zh) * 2022-07-12 2023-05-30 长春吉大正元信息技术股份有限公司 签名方法、验签方法及电子设备
CN115982399A (zh) * 2023-03-16 2023-04-18 北京集度科技有限公司 图像查找方法、移动设备、电子设备、及计算机程序产品
CN115982399B (zh) * 2023-03-16 2023-05-16 北京集度科技有限公司 图像查找方法、移动设备、电子设备、及计算机程序产品
CN116304163A (zh) * 2023-05-11 2023-06-23 深圳兔展智能科技有限公司 一种图像检索方法、装置、计算机设备及介质
CN116304163B (zh) * 2023-05-11 2023-07-25 深圳兔展智能科技有限公司 一种图像检索方法、装置、计算机设备及介质
CN116823816A (zh) * 2023-08-28 2023-09-29 济南正邦电子科技有限公司 一种基于安防监控静态存储器的检测设备及检测方法
CN116823816B (zh) * 2023-08-28 2023-11-21 济南正邦电子科技有限公司 一种基于安防监控静态存储器的检测设备及检测方法
CN116883879A (zh) * 2023-09-04 2023-10-13 苏州浪潮智能科技有限公司 一种闭环检测的方法、装置及可读存储介质
CN116883879B (zh) * 2023-09-04 2024-01-16 苏州浪潮智能科技有限公司 一种闭环检测的方法、装置及可读存储介质
CN116958266A (zh) * 2023-09-19 2023-10-27 苏州元脑智能科技有限公司 闭环检测方法、装置、通信设备及存储介质
CN116958266B (zh) * 2023-09-19 2024-02-20 苏州元脑智能科技有限公司 闭环检测方法、装置、通信设备及存储介质
CN117056086A (zh) * 2023-10-11 2023-11-14 国网山东省电力公司滨州市滨城区供电公司 基于排列熵算法的故障检测方法、系统、终端及存储介质
CN117173247A (zh) * 2023-11-02 2023-12-05 中国海洋大学 基于2D激光雷达与LightGBM的室外定位与构图方法及系统
CN117173247B (zh) * 2023-11-02 2024-02-02 中国海洋大学 基于2D激光雷达与LightGBM的室外定位与构图方法及系统

Also Published As

Publication number Publication date
CN112614187A (zh) 2021-04-06
CN112614187B (zh) 2024-03-26

Similar Documents

Publication Publication Date Title
WO2022142855A1 (zh) 回环检测方法、装置、终端设备和可读存储介质
Lynen et al. Placeless place-recognition
CN103207898B (zh) 一种基于局部敏感哈希的相似人脸快速检索方法
Gehrig et al. Visual place recognition with probabilistic voting
Xia et al. Loop closure detection for visual SLAM using PCANet features
An et al. Fast and incremental loop closure detection with deep features and proximity graphs
Sunderrajan et al. Context-aware hypergraph modeling for re-identification and summarization
Zhang et al. ReYOLO: A traffic sign detector based on network reparameterization and features adaptive weighting
Tsintotas et al. Tracking‐DOSeqSLAM: A dynamic sequence‐based visual place recognition paradigm
CN111310662A (zh) 一种基于集成深度网络的火焰检测识别方法及系统
Xiong et al. Contrastive learning for automotive mmWave radar detection points based instance segmentation
CN111652910A (zh) 一种基于对象空间关系的目标跟踪算法
Ibrahim et al. A survey of performance optimization in neural network-based video analytics systems
Wang et al. Design and implementation of remote facial expression recognition surveillance system based on PCA and KNN algorithms
Tripathi et al. Object detection using YOLO: A survey
Li et al. Loop closure detection based on image semantic segmentation in indoor environment
Kavya Feature extraction technique for robust and fast visual tracking: a typical review
Hu et al. STRNN: End-to-end deep learning framework for video partial copy detection
WO2020233414A1 (zh) 物体识别方法、装置及车辆
Grekov et al. Application of the YOLOv5 Model for the Detection of Microobjects in the Marine Environment
CN116012744A (zh) 一种闭环检测方法、装置、设备及存储介质
Kamenou et al. Multi-level deep learning vehicle re-identification using ranked-based loss functions
Wang et al. Traffic-signs recognition system based on multi-features
Liu et al. A novel adaptive kernel correlation filter tracker with multiple feature integration
Huang et al. VLAD-based loop closure detection for monocular SLAM

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21913597

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 21913597

Country of ref document: EP

Kind code of ref document: A1