CN108108737A - Closed loop detecting system and method based on multi-feature fusion - Google Patents

Closed loop detecting system and method based on multi-feature fusion Download PDF

Info

Publication number
CN108108737A
CN108108737A CN201611034155.2A CN201611034155A CN108108737A CN 108108737 A CN108108737 A CN 108108737A CN 201611034155 A CN201611034155 A CN 201611034155A CN 108108737 A CN108108737 A CN 108108737A
Authority
CN
China
Prior art keywords
feature
scene
closed loop
fusion
geometry
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201611034155.2A
Other languages
Chinese (zh)
Inventor
陈墩金
覃争鸣
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Rich Intelligent Science And Technology Ltd Is Reflected In Guangzhou
Original Assignee
Rich Intelligent Science And Technology Ltd Is Reflected In Guangzhou
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Rich Intelligent Science And Technology Ltd Is Reflected In Guangzhou filed Critical Rich Intelligent Science And Technology Ltd Is Reflected In Guangzhou
Priority to CN201611034155.2A priority Critical patent/CN108108737A/en
Publication of CN108108737A publication Critical patent/CN108108737A/en
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/52—Scale-space analysis, e.g. wavelet analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/22—Matching criteria, e.g. proximity measures
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/23—Clustering techniques
    • G06F18/232—Non-hierarchical techniques
    • G06F18/2321—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions
    • G06F18/23213—Non-hierarchical techniques using statistics or function optimisation, e.g. modelling of probability density functions with fixed number of clusters, e.g. K-means clustering
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/46—Descriptors for shape, contour or point-related descriptors, e.g. scale invariant feature transform [SIFT] or bags of words [BoW]; Salient regional features
    • G06V10/462—Salient features, e.g. scale invariant feature transforms [SIFT]
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/40—Extraction of image or video features
    • G06V10/50—Extraction of image or video features by performing operations within image blocks; by using histograms, e.g. histogram of oriented gradients [HoG]; by summing image-intensity values; Projection analysis
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/10—Terrestrial scenes

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Evolutionary Computation (AREA)
  • Evolutionary Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Artificial Intelligence (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Probability & Statistics with Applications (AREA)
  • Image Analysis (AREA)

Abstract

The invention discloses a kind of closed loop detecting system based on multi-feature fusion and method, the closed loop detecting system based on multi-feature fusion, including：Data acquisition module gathers scene image using laser sensor and is pre-processed；Extraction of Geometrical Features module extracts the geometry feature of scene；Visual feature extraction module extracts the visual signature of scene；Fusion Features module merges geometry feature and visual signature with Method of Data with Adding Windows；Closed loop detection module carries out closed loop detection using fusion feature.The closed loop detecting system and method based on multi-feature fusion of the present invention passes through the visual signature of scene appearance and geometry Fusion Features, healthy and strong scene characteristic is obtained to represent, and utilize " perception is obscured " in the geometric space constraint reduction scene matching of characteristic matching, false drop rate is reduced, therefore the present invention program has good expansibility and robustness.

Description

Closed loop detecting system and method based on multi-feature fusion
Technical field
The present invention relates to closed loop detection techniques, and in particular to closed loop detecting system and method based on multi-feature fusion.
Background technology
With the raising of properties, service robot can complete more and more tasks in people's daily life, Such as cleaning, mobile object etc..In order to which task is made to complete more smooth, robot must carry out the environment of surrounding It perceives and recognizes in more detail and accurately.
Map expression is robot localization and builds the basis of figure, i.e., using some special point, line, surface in scene or field The pose of some visual signatures characterization robot in scape image, by carrying out matching comparison to the category feature, you can speculate machine The current pose of device people.
Closed loop detection represents that the stability of algorithm has extremely important effect, closed loop detection for improving robot map Basic definition be position that robot ceaselessly detects whether to have returned to that in heuristic process a past accessed.This Kind detection method can improve the position estimation accuracy of robot reality, and whether confirmation is also related to before by this region It is to Global localization problem or even all highly beneficial to solving robot abduction issue.
The closed loop detection method of mainstream depends on visual signature, i.e., gathers object and background in indoor environment by camera Visual signature, matched by visual signature, carry out closed loop detection.However the visual scene largely repeated in indoor environment Such as door and window can cause matched " perception is obscured " phenomenon of visual signature.In addition, the method for view-based access control model feature can not be abundant Utilize the structuring of geometric scene, semi-structured feature in indoor environment.
The content of the invention
Present invention aims to overcome that the deficiencies in the prior art, especially solve the vision largely repeated in indoor environment Scene such as door and window can cause the method for the problem of visual signature matched " perception is obscured " phenomenon and view-based access control model feature not The problem of structuring of geometric scene in indoor environment, semi-structured feature can be made full use of.
In order to solve the above technical problems, the present invention provides a kind of closed loop detecting system based on multi-feature fusion, wherein, institute Stating closed loop detecting system based on multi-feature fusion includes：Data acquisition module, using laser sensor acquisition scene image simultaneously It is pre-processed；Extraction of Geometrical Features module extracts the geometry feature of scene；Visual feature extraction module extracts scene Visual signature；Fusion Features module merges geometry feature and visual signature with Method of Data with Adding Windows；Closed loop detects mould Block carries out closed loop detection using fusion feature.
The present invention also provides a kind of closed loop detection method based on multi-feature fusion, wherein, it is described to be based on multiple features fusion Closed loop detection method include：S1 gathers scene image using laser sensor and is pre-processed；S2 extracts the several of scene What structure feature；S3 extracts the visual signature of scene；S4 merges geometry feature and visual signature with Method of Data with Adding Windows； S5 carries out closed loop detection using fusion feature.
Further, HOG features, edge feature or other suitable geometry can be used in the geometric properties operation for extracting scene Structure feature method for expressing.
Further, extract scene visual structure operation can be used SIFT feature, SUFR features or other suitably regard Feel character representation method.
Further, it is described to be grasped using fusion feature progress closed loop detection operation comprising vocabulary training, similarity calculation Make.
Further, the vocabulary training can be kmeans clusters, fuzzy C-means clustering or other suitable cluster sides Method.
The advantageous effect of the scheme of the invention is, by visual signature and the geometry Fusion Features of scene appearance, obtains It obtained healthy and strong scene characteristic to represent, and " perceiving mixed in scene matching is reduced using the geometric space constraint of characteristic matching Confuse ", false drop rate is reduced, therefore the present invention program has good expansibility and robustness.
Description of the drawings
Fig. 1 is the closed loop detecting system schematic diagram based on multi-feature fusion of the embodiment of the present invention.
Fig. 2 is the flow chart of the closed loop detection method based on multi-feature fusion of the embodiment of the present invention.
Fig. 3 is that the local point cloud chart picture of the embodiment of the present invention matches certain case effect figure.
Specific embodiment
Below in conjunction with the accompanying drawings and specific embodiment the present invention is carried out in further detail with complete explanation.It is appreciated that It is that specific embodiment described herein is only used for explaining the present invention rather than limitation of the invention.
Fig. 1 is closed loop detecting system schematic diagram based on multi-feature fusion according to an embodiment of the invention.
With reference to Fig. 1, the closed loop detecting system based on multi-feature fusion includes：Data acquisition module 10, uses laser Sensor gathers scene image and is pre-processed；Extraction of Geometrical Features module 20 extracts the geometry feature of scene；Vision Characteristic extracting module 30 extracts the visual signature of scene；Fusion Features module 40, it is special with Method of Data with Adding Windows fusion geometry It seeks peace visual signature；Closed loop detection module 50 carries out closed loop detection using fusion feature.
Corresponding, the present invention also provides a kind of closed loop detection methods based on multi-feature fusion, specifically refer to Fig. 2, It is the flow chart of the closed loop detection method based on multi-feature fusion of the embodiment of the present invention.
With reference to Fig. 2, closed loop detection method based on multi-feature fusion according to an embodiment of the invention includes：S1, use Laser sensor gathers scene image and is pre-processed；S2, the geometry feature for extracting scene；S3, regarding for scene is extracted Feel feature；S4, geometry feature and visual signature are merged with Method of Data with Adding Windows；S5, closed loop inspection is carried out using fusion feature It surveys.
In addition, histograms of oriented gradients (HOG features), not bending moment can be used in the geometric properties operation of the S2 extractions scene Feature or other suitable geometry character representation methods.HOG features are by calculating the ladder with statistical picture regional area Degree direction histogram carrys out constitutive characteristic, for describing the edge of object；Invariant moment features use insensitive based on area to converting Several squares in domain express the geometric properties of image-region, have the characteristics such as rotation, translation, scale not as shape feature Become feature.The present embodiment is illustrated using invariant moment features below.
The visual signature operation of the S3 extractions scene can be used Scale invariant features transform (SIFT) feature, accelerate robust (SUFR) feature or other suitable visual signature method for expressing.SIFT feature is that a kind of common image local represents feature side Method using difference of Gaussian pyramid, stable characteristic point is calculated under multiscale space, and uses characteristic point and adjacent domain structure Build local feature description.SUFR features are that one kind of SIFT feature is speeded up to, it may have scale invariability and invariable rotary Property.The present embodiment is illustrated using SIFT feature below.
The S5 using fusion feature carry out closed loop detection operation in similarity calculation operation can be used kmeans cluster, Fuzzy C-means clustering or other suitable clustering methods.Kmeans clusters are a kind of common clustering methods based on distance, are recognized Smaller for two object distances, similarity is bigger.Fuzzy C-means clustering does not know sample class using fuzzy theory Description, objectively responds real world.The present embodiment is illustrated using kmeans clusters below.
With reference to Fig. 3, Fig. 3 is the closed loop detection method Organization Chart based on multi-feature fusion of the present embodiment.
Specifically, the present embodiment is described as follows：
Step S1：Scene image is gathered using laser sensor and is pre-processed.By laser sensor to field around After scape is scanned, the dotted data that multiple discrete laser scanning points return are obtained, since original laser scan data carries Noise, the scene for scanning gained not fully meets scene expression, thus needs the pretreatment carried out to laser scanning.Using certainly It adapts to neighbor point cluster dividing method and carries out neighbor point sub-clustering, as shown in formula (1)：
Δ l=m ρk-1Δφ (1)
Wherein, ρk-1For the observation of former point, Δ l is the distance of consecutive points, and Δ φ is corresponding partially for two adjacent scanning elements Gyration, m are empirical coefficient.It is considered neighbor point cluster if actual 2 points of distances are less than Δ l.
Step S2：Extract the geometry feature of scene.Represent the image as distributed function f (x, y), zeroth order square M00 Represent the quality of gray level, first moment (M01,M10) represent, (xc,yc) central coordinate of circle is origin, centralized moments of image is expressed as：
Mpq=∫ ∫ [(x-xc)p]×[(y-yc)q]f(x,y)dxdy (2)
Wherein, p and q represents the exponent number of square, then invariant moment features { I1,I2,I3,I4,I5By multiple high-orders, bending moment does not combine It forms, each High Order Moment is got by central moment calculating, and calculation formula is as follows：
I1=M20+M02 (3)
I2=(M20-M02)2+(2M11)2 (4)
I3=(M30-3M12)2+(3M21-M03)2 (5)
I4=(M30+M12)2+(M21+M03)2 (6)
Step S3：Extract the visual signature of scene.Specifically include following operation：
S31：Calculate image level gradient and vertical gradient.Using horizontal, vertical difference operator to each pixel of image (x, y) is filtered to take horizontal gradient lxWith vertical gradient ly, as shown in formula (8).
S32：Calculate image Harris angle points.Harris angle point value c (x, y) such as formula (9) institutes of each pixel (x, y) Show.
When the value of c (x, y) is more than given threshold value, then it is assumed that the pixel is a Harris angle point.
S33：Build multi-scale image space.For a sub-picture, various sizes of subgraph is obtained by down-sampling, it will Subgraph multiplies carry out convolutional calculation with Gaussian convolution nuclear phase, so as to obtain multi-scale image space.
S34：Find the extreme point of metric space.Each Harris angle point will be all with it consecutive points compare, see it Whether than it image area and scale domain consecutive points it is big or small.Sampled point and it with scale 8 consecutive points and phase up and down Totally 26 points compare corresponding 9 × 2 points of adjacent scale, to ensure all to detect extreme point in metric space and image space.One If a Harris angle points are maximum or minimum value in multi-scale image this layer of space and bilevel 26 fields, It is an extreme point of the image under the scale to be considered as the point.
S35：Calculate SIFT feature.It is specified using the gradient direction distribution characteristic of characteristic point neighborhood territory pixel for each key point Directioin parameter calculates modulus value and the direction of this feature point gradient.The ladder in 8 directions is calculated on the fritter of feature vertex neighborhood 4 × 4 Direction histogram is spent, draws the accumulated value of each gradient direction, forms the histogram of 4 × 4 × 8=128 dimension, is i.e. SIFT is special Sign.
Step S4：Geometry feature and visual signature are merged with Method of Data with Adding Windows.To the geometric properties of different dimensional with Visual signature carries out normalization and connects to obtain feature vector, calculates feature vector autocorrelation matrix R, and feature decomposition is carried out to R, point Eigenvalue matrix U and corresponding eigenvectors matrix Λ is not obtained to decompose, each column vector of matrix contains the information of X, It can be considered the fusion feature of X, the preceding m larger corresponding feature vector tectonic transition matrix T of characteristic value, by original normalizing Change feature and carry out Karhunen-Loeve transformation：Y=T × X is to get to the scene appearance fusion feature Y after dimensionality reduction.
Step S5：Closed loop detection operation is carried out using fusion feature to operate comprising vocabulary training, similarity calculation.
S51：Vocabulary training.Using kmeans clustering method construction feature word lists, multiple feature vectors are randomly choosed As center vector, other features in characteristic set are distributed into closest center vector, are calculated by formula (10) And the average value of center vector is updated, and pass through formula (11) calculation criterion function E.Stop cluster when E meets threshold requirement.
Wherein, x represents feature vector, xiRepresent center vector, CiRepresent the characteristic set of the cluster centre, E expressiveness Function.
S52：Similarity calculation.For the scene that robot in motion process obtains, all features in scene are distributed Into the cluster centre of closest feature vocabulary, and using the corresponding word of place cluster centre as the word of this feature It represents.By comparing current scene and the word matched degree of historic scenery, you can quickly obtain the similarity of scene.If Scene similarity is more than threshold value, then it is assumed that closed loop is set up.
It is above-mentioned for the preferable embodiment of the present invention, but embodiments of the present invention and from the limitation of the above, His any Spirit Essence without departing from the present invention with made under principle change, modification, replacement, combine, simplification, should be The substitute mode of effect, is included within protection scope of the present invention.

Claims (4)

1. a kind of closed loop detecting system based on multi-feature fusion, which is characterized in that wherein：
Data acquisition module gathers scene image using laser sensor and is pre-processed；
Extraction of Geometrical Features module extracts the geometry feature of scene；
Visual feature extraction module extracts the visual signature of scene；
Fusion Features module merges geometry feature and visual signature with Method of Data with Adding Windows；
Closed loop detection module carries out closed loop detection using fusion feature.
2. a kind of method realized using closed loop detecting system based on multi-feature fusion described in claim 1, feature exist In including the following steps：
S1, gather scene image using laser sensor and pre-processed；
S2, the geometry feature for extracting scene；
S3, the visual signature for extracting scene；
S4, geometry feature and visual signature are merged with Method of Data with Adding Windows；
S5, closed loop detection is carried out using fusion feature.
3. closed loop detection method based on multi-feature fusion according to claim 2, which is characterized in that the step S2 is carried The geometric properties of scene is taken to operate, HOG features, edge feature or other suitable geometry character representation methods can be used.
4. closed loop detection method based on multi-feature fusion according to claim 2, which is characterized in that the step S3 is carried The visual structure of scene is taken to operate, SIFT feature, SUFR features or other suitable visual signature method for expressing can be used.
CN201611034155.2A 2016-11-24 2016-11-24 Closed loop detecting system and method based on multi-feature fusion Pending CN108108737A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611034155.2A CN108108737A (en) 2016-11-24 2016-11-24 Closed loop detecting system and method based on multi-feature fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611034155.2A CN108108737A (en) 2016-11-24 2016-11-24 Closed loop detecting system and method based on multi-feature fusion

Publications (1)

Publication Number Publication Date
CN108108737A true CN108108737A (en) 2018-06-01

Family

ID=62204653

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611034155.2A Pending CN108108737A (en) 2016-11-24 2016-11-24 Closed loop detecting system and method based on multi-feature fusion

Country Status (1)

Country Link
CN (1) CN108108737A (en)

Cited By (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109711245A (en) * 2018-11-05 2019-05-03 广东工业大学 A kind of closed loop detection method based on image candidate region
CN110276348A (en) * 2019-06-20 2019-09-24 腾讯科技（深圳）有限公司 A kind of image position method, device, server and storage medium
CN110659688A (en) * 2019-09-24 2020-01-07 江西慧识智能科技有限公司 Monitoring video riot and terrorist behavior identification method based on machine learning
CN112595322A (en) * 2020-11-27 2021-04-02 浙江同善人工智能技术有限公司 Laser SLAM method fusing ORB closed loop detection
CN112800833A (en) * 2019-08-09 2021-05-14 河海大学 Method for realizing overall object identification based on mechanism model for water environment monitoring
CN115496931A (en) * 2022-11-14 2022-12-20 济南奥普瑞思智能装备有限公司 Industrial robot health monitoring method and system

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN109711245A (en) * 2018-11-05 2019-05-03 广东工业大学 A kind of closed loop detection method based on image candidate region
CN110276348A (en) * 2019-06-20 2019-09-24 腾讯科技（深圳）有限公司 A kind of image position method, device, server and storage medium
CN110276348B (en) * 2019-06-20 2022-11-25 腾讯科技（深圳）有限公司 Image positioning method, device, server and storage medium
CN112800833A (en) * 2019-08-09 2021-05-14 河海大学 Method for realizing overall object identification based on mechanism model for water environment monitoring
CN112800833B (en) * 2019-08-09 2022-02-25 河海大学 Method for realizing overall object identification based on mechanism model for water environment monitoring
CN110659688A (en) * 2019-09-24 2020-01-07 江西慧识智能科技有限公司 Monitoring video riot and terrorist behavior identification method based on machine learning
CN112595322A (en) * 2020-11-27 2021-04-02 浙江同善人工智能技术有限公司 Laser SLAM method fusing ORB closed loop detection
CN112595322B (en) * 2020-11-27 2024-05-07 浙江同善人工智能技术有限公司 ORB closed loop detection fused laser SLAM method
CN115496931A (en) * 2022-11-14 2022-12-20 济南奥普瑞思智能装备有限公司 Industrial robot health monitoring method and system
CN115496931B (en) * 2022-11-14 2023-02-10 济南奥普瑞思智能装备有限公司 Industrial robot health monitoring method and system

Similar Documents

Publication Publication Date Title
Zhu et al. Single image 3d object detection and pose estimation for grasping
CN108108737A (en) Closed loop detecting system and method based on multi-feature fusion
CN106682598B (en) Multi-pose face feature point detection method based on cascade regression
US9846946B2 (en) Objection recognition in a 3D scene
CN105956582B (en) A kind of face identification system based on three-dimensional data
Lin et al. Facet segmentation-based line segment extraction for large-scale point clouds
US20170116781A1 (en) 3d scene rendering
US9189855B2 (en) Three dimensional close interactions
CN103632132B (en) Face detection and recognition method based on skin color segmentation and template matching
CN101398886B (en) Rapid three-dimensional face identification method based on bi-eye passiveness stereo vision
CN107093205A (en) A kind of three dimensions building window detection method for reconstructing based on unmanned plane image
CN109902532A (en) A kind of vision closed loop detection method
Wang et al. 3-D point cloud object detection based on supervoxel neighborhood with Hough forest framework
CN103617413B (en) Method for identifying object in image
CN105809113B (en) Three-dimensional face identification method and the data processing equipment for applying it
Zang et al. Road network extraction via aperiodic directional structure measurement
CN112116635A (en) Visual tracking method and device based on rapid human body movement
CN106778491B (en) The acquisition methods and equipment of face 3D characteristic information
Jing et al. Island road centerline extraction based on a multiscale united feature
Wang et al. Hand posture recognition from disparity cost map
CN105404883B (en) A kind of heterogeneous three-dimensional face identification method
CN110008902B (en) Finger vein recognition method and system fusing basic features and deformation features
Berretti et al. 3D partial face matching using local shape descriptors
Xu et al. MultiView-based hand posture recognition method based on point cloud
Zhou et al. Road detection based on edge feature with GAC model in aerial image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
WD01 Invention patent application deemed withdrawn after publication
WD01 Invention patent application deemed withdrawn after publication

Application publication date: 20180601