CN110322500A - Immediately optimization method and device, medium and the electronic equipment of positioning and map structuring - Google Patents
Immediately optimization method and device, medium and the electronic equipment of positioning and map structuring Download PDFInfo
- Publication number
- CN110322500A CN110322500A CN201910578527.5A CN201910578527A CN110322500A CN 110322500 A CN110322500 A CN 110322500A CN 201910578527 A CN201910578527 A CN 201910578527A CN 110322500 A CN110322500 A CN 110322500A
- Authority
- CN
- China
- Prior art keywords
- frame image
- point
- current frame
- visual signature
- information
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/50—Depth or shape recovery
- G06T7/55—Depth or shape recovery from multiple images
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
- Y02T10/00—Road transport of goods or passengers
- Y02T10/10—Internal combustion engine [ICE] based vehicles
- Y02T10/40—Engine management systems
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention discloses optimization method and device, storage medium and the electronic equipments of a kind of positioning and map structuring immediately, are related to technical field of image processing.The optimization method of the instant positioning and map structuring includes: the visual signature point for extracting current frame image, and determines characteristic point unrelated to the depth information of current frame image in visual signature point, as monocular characteristic point；If the quantity of matched visual signature point is greater than preset matching threshold value between current frame image and reference frame image, then when visual signature is aligned with inertia characteristics, the pose of current frame image is optimized using the first constraint condition constructed by monocular characteristic point information and by the second constraint condition that Inertia information constructs, obtains intermediate pose parameter；Interposition appearance parameter is optimized using the depth information of current frame image and the depth information of reference frame image, to determine the pose parameter of current frame image.The precision and robustness of SLAM can be improved in the present invention.
Description
Technical field
This disclosure relates to technical field of image processing, in particular to the optimization of a kind of positioning and map structuring immediately
Method, instant optimization device, storage medium and the electronic equipment positioned with map structuring.
Background technique
One of important technology as computer vision field, SLAM (Simultaneous Localization And
Mapping, immediately positioning and map structuring) technology has received widespread attention and obtained quick development.The technology can be with
It is applied to the every field such as unmanned plane, automatic Pilot, building high-precision map, virtual reality, augmented reality.
SLAM technology be used for construct circumstances not known map and in real time in map alignment sensor position.In base
In the monocular SLAM technology of inertia, on the one hand, the speed for carrying out map initialization using visual information is slower；On the other hand,
During generating three-dimensional information using visual information, may there are problems that failed regeneration, cause the something lost of valid data
Leakage causes to estimate pose inaccuracy, finally will affect the precision and robustness of SLAM.
It should be noted that information is only used for reinforcing the reason to the background of the disclosure disclosed in above-mentioned background technology part
Solution, therefore may include the information not constituted to the prior art known to persons of ordinary skill in the art.
Summary of the invention
Optimization method, instant positioning and the map structure for being designed to provide a kind of positioning immediately and map structuring of the disclosure
Optimization device, storage medium and the electronic equipment built, so overcome at least to a certain extent due to the relevant technologies limitation and
The not high problem of monocular SLAM scheme accuracy caused by defect based on inertia.
According to disclosure illustrative embodiments in a first aspect, providing the optimization of a kind of positioning and map structuring immediately
Method this method comprises: extracting the visual signature point of current frame image, and determines the depth in visual signature point with current frame image
The unrelated characteristic point of information is spent, as monocular characteristic point；If matched view between current frame image and reference frame image
Feel that the quantity of characteristic point is greater than preset matching threshold value, then when visual signature is aligned with inertia characteristics, using by monocular characteristic point
First constraint condition of information architecture and the second constraint condition constructed by Inertia information carry out the pose of current frame image
Optimization, obtains intermediate pose parameter；Using the depth information of current frame image and the depth information of reference frame image to interposition
Appearance parameter optimizes, to determine the pose parameter of current frame image.
According to the second aspect of disclosure illustrative embodiments, the optimization of a kind of positioning and map structuring immediately is provided
Device, which includes: the visual signature point that characteristic determination module is used to extract current frame image, and is determined in visual signature point
The characteristic point unrelated to the depth information of current frame image, as monocular characteristic point；If the first optimization module is for working as
Between prior image frame and reference frame image the quantity of matched visual signature point be greater than preset matching threshold value, then visual signature with
When inertia characteristics are aligned, using the first constraint condition constructed by monocular characteristic point information and by the second of Inertia information building
Constraint condition optimizes the pose of current frame image, obtains intermediate pose parameter；Second optimization module is used for using current
The depth information of frame image and the depth information of reference frame image optimize interposition appearance parameter, to determine current frame image
Pose parameter.
According to the third aspect of disclosure illustrative embodiments, a kind of storage medium is provided, is stored thereon with calculating
Machine program, the computer program realize the optimization method of above-mentioned instant positioning and map structuring when being executed by processor
According to the fourth aspect of disclosure illustrative embodiments, a kind of electronic equipment is provided, comprising: processor；With
And memory, the executable instruction for storage processor；Wherein, which is configured to hold via executable instruction is executed
The optimization method of row above-mentioned instant positioning and map structuring.
In the technical solution provided by some embodiments of the present disclosure, by constructed by monocular characteristic point information
One constraint condition and by Inertia information construct the second constraint condition the pose of current frame image is optimized after, recycle
Depth information optimizes optimum results further progress.Pose is optimized in conjunction with depth information, is avoided only with vision
Information, which optimizes, may cause the problem of characteristic point is omitted, and the precision and robustness of SLAM can be improved.
It should be understood that above general description and following detailed description be only it is exemplary and explanatory, not
The disclosure can be limited.
Detailed description of the invention
The drawings herein are incorporated into the specification and forms part of this specification, and shows the implementation for meeting the disclosure
Example, and together with specification for explaining the principles of this disclosure.It should be evident that the accompanying drawings in the following description is only the disclosure
Some embodiments for those of ordinary skill in the art without creative efforts, can also basis
These attached drawings obtain other attached drawings.In the accompanying drawings:
Fig. 1 diagrammatically illustrates the optimization side of instant positioning and map structuring according to an exemplary embodiment of the present disclosure
The flow chart of method；
Fig. 2 shows the processes of order mesh characteristic point and stereoscopic features point really according to an exemplary embodiment of the present disclosure
Schematic diagram；
Fig. 3 shows the schematic diagram of progress trigonometric ratio processing according to an exemplary embodiment of the present disclosure；
Fig. 4 diagrammatically illustrates the flow chart of carry out relocation process according to an exemplary embodiment of the present disclosure；
Fig. 5 is diagrammatically illustrated according to the excellent of the instant positioning of the first illustrative embodiments of the disclosure and map structuring
Disguise the block diagram set；
Fig. 6 is diagrammatically illustrated according to the excellent of the instant positioning of the second illustrative embodiments of the disclosure and map structuring
Disguise the block diagram set；
Fig. 7 diagrammatically illustrates the block diagram of characteristic determination module according to an exemplary embodiment of the present disclosure；
Fig. 8 is diagrammatically illustrated according to the excellent of the instant positioning of the third illustrative embodiments of the disclosure and map structuring
Disguise the block diagram set；
Fig. 9 is diagrammatically illustrated according to the excellent of the instant positioning of the 4th illustrative embodiments of the disclosure and map structuring
Disguise the block diagram set；
Figure 10 diagrammatically illustrates instant positioning and the map structuring of the 5th illustrative embodiments according to the disclosure
Optimize the block diagram of device；
Figure 11 diagrammatically illustrates the block diagram of electronic equipment according to an exemplary embodiment of the present disclosure.
Specific embodiment
Example embodiment is described more fully with reference to the drawings.However, example embodiment can be with a variety of shapes
Formula is implemented, and is not understood as limited to example set forth herein；On the contrary, thesing embodiments are provided so that the disclosure will more
Fully and completely, and by the design of example embodiment comprehensively it is communicated to those skilled in the art.Described feature, knot
Structure or characteristic can be incorporated in any suitable manner in one or more embodiments.In the following description, it provides perhaps
More details fully understand embodiment of the present disclosure to provide.It will be appreciated, however, by one skilled in the art that can
It is omitted with technical solution of the disclosure one or more in the specific detail, or others side can be used
Method, constituent element, device, step etc..In other cases, be not shown in detail or describe known solution to avoid a presumptuous guest usurps the role of the host and
So that all aspects of this disclosure thicken.
In addition, attached drawing is only the schematic illustrations of the disclosure, it is not necessarily drawn to scale.Identical attached drawing mark in figure
Note indicates same or similar part, thus will omit repetition thereof.Some block diagrams shown in the drawings are function
Energy entity, not necessarily must be corresponding with physically or logically independent entity.These function can be realized using software form
Energy entity, or these functional entitys are realized in one or more hardware modules or integrated circuit, or at heterogeneous networks and/or place
These functional entitys are realized in reason device device and/or microcontroller device.
Flow chart shown in the drawings is merely illustrative, it is not necessary to including all steps.For example, the step of having
It can also decompose, and the step of having can merge or part merges, therefore the sequence actually executed is possible to according to the actual situation
Change.In addition, term " first " used below, " second ", " third " etc. are merely to the purpose distinguished, should not be used as this public affairs
The limitation opened.
The instant positioning below and the optimization method of map structuring can be realized by terminal device, that is to say, that terminal
Equipment can execute each step of the instant positioning and the optimization method of map structuring of the disclosure.In this case, this public affairs
The optimization device of the instant positioning and map structuring of opening illustrative embodiments can be only fitted in the terminal device.
Terminal device can be realize SLAM scheme equipment, for example, terminal device can be mobile phone, plate, intelligence can
Wearable device (smartwatch, intelligent glasses etc.), unmanned plane, mobile robot etc..
Fig. 1 diagrammatically illustrates instant positioning and the optimization method of map structuring of the illustrative embodiments of the disclosure
Flow chart.With reference to Fig. 1, the instant positioning and the optimization method of map structuring be may comprise steps of:
S12. the visual signature point of current frame image is extracted, and determines in visual signature point and believes with the depth of current frame image
Unrelated characteristic point is ceased, as monocular characteristic point.
In the illustrative embodiments of the disclosure, visual signature point can be ORB (Oriented FAST and
Rotated BRIEF, rapid characteristic points extract and description) characteristic point.The calculating speed of ORB characteristic point is fast, is suitable in terminal
Implement in equipment.
One ORB characteristic point may include that FAST angle point and BRIER describe sub- two parts.Wherein, FAST angle point refers to this
The position of ORB characteristic point in the picture, FAST angle point predominantly detect the apparent region of local pixel grey scale change, calculating speed
Fastly, core idea is, if the pixel differences of a pixel and neighborhood are larger (that is, too dark or too bright), which is one
A angle point；It is a vector by binary representation that BRIEF, which describes son, which describes FAST in the way of artificial settings
The information of angle point surrounding pixel, that is to say, that BRIEF description son vector be made of multiple 0 and 1, characterization FAST angle point with it is attached
Pixel value size relation between nearly pixel.
Although being illustrated by taking ORB characteristic point as an example below, it being understood, however, that, other features can also be used
Point is as the point of visual signature described in the disclosure, for example, (Speeded Up Robust Features accelerates steady special SURF
Sign), Sift (Scale-invariant feature transform, Scale invariant features transform) feature, Harris's angle point
Deng the disclosure does not do the type of visual signature point specifically limited.
For the current frame image shot by camera, the ORB characteristic point that ORB algorithm extracts current frame image can use,
Visual signature point as current frame image.Furthermore it is possible to obtain the corresponding depth of current frame image by means of depth transducer
Information (depth map), in such a case, it is possible to determine whether visual signature point is associated with depth information, that is to say, that from
The corresponding visual signature point of depth information is searched in current frame image.
Specifically, can the outer ginseng matrix in advance to camera image and depth map demarcate, on this basis, for one
A given image coordinate can determine corresponding depth pixel coordinate by the transformation of outer ginseng matrix on depth map, by
This can determine whether visual signature point is corresponding in the visual signature point and depth information for getting current frame image
Depth information, that is to say, that can determine whether visual signature point is associated with depth information.
Disclosure illustrative embodiments are using characteristic point unrelated to depth information in visual signature point as monocular
Characteristic point (and being referred to alternatively as mono characteristic point), wherein monocular characteristic point can be interpreted as to vision corresponding with two-dimensional space
Characteristic point.In addition, (and being referred to alternatively as characteristic point associated with depth information in visual signature point as stereoscopic features point
Stereo characteristic point), wherein stereoscopic features point can be interpreted as to visual signature point corresponding with three-dimensional space.
Fig. 2 shows the schematic diagrames of determining monocular characteristic point and the process of stereoscopic features point.It is possible, firstly, to be set from by terminal
ORB characteristic point is extracted in the RGB image of standby camera shooting；Next, will using the depth information that depth transducer is got
ORB characteristic point is divided into monocular characteristic point and stereoscopic features point.
In addition, there may be the feelings of inaccuracy for the depth information detected for strong light environment or the object of black surface
Condition, some embodiments of the present disclosure can also include the process evaluated depth information, for example, can set for characterizing
Depth information can use the quality threshold of degree.When determining that depth information reaches the quality threshold, depth information pair is just utilized
Visual signature point is divided.
S14. if the quantity of matched visual signature point is greater than preset matching between current frame image and reference frame image
Threshold value, then when visual signature is aligned with inertia characteristics, using the first constraint condition constructed by monocular characteristic point information and
The pose of current frame image is optimized by the second constraint condition that Inertia information constructs, obtains intermediate pose parameter.
Disclosure illustrative embodiments further include the scheme of map initialization before optimizing to current frame image,
To determine to be compared the reference frame to realize tracking with current frame image.
Specifically, firstly, extracting its visual signature point, and determine the monocular of the input picture for an input picture
Characteristic point and stereoscopic features point, detailed process is similar with the monocular characteristic point of above-mentioned determining current frame image and stereoscopic features point,
Details are not described herein；Next, the quantity of the visual signature point of input picture can be compared with the first preset threshold, and
The quantity of the stereoscopic features point of input picture is compared with the second preset threshold, wherein the first preset threshold and second is in advance
If threshold value can be threshold value relevant to image resolution ratio, the disclosure does not do specific value taking human as being configured in advance
It is specifically limited.
If the quantity of the visual signature point of input picture is greater than the first preset threshold, and the stereoscopic features of input picture
The quantity of point is greater than the second preset threshold, it may be considered that map initialization success, it can be by the stereoscopic features of the input picture
The corresponding space three-dimensional point of point is as initial point cloud map.In such a case, it is possible to using the input picture as reference frame figure
Picture.
If the quantity of the visual signature point of input picture is special no more than the solid of the first preset threshold or input picture
The quantity of sign point is not more than the second preset threshold, then illustrates that the input picture is unsatisfactory for the requirement of reference frame image, can choose
It is rejected.
In the illustrative embodiments of the disclosure, the visual signature in current frame image with reference frame image can be determined
The quantity of the matched visual signature point of point, if the quantity is greater than preset matching threshold value, it may be considered that tracking successfully.Wherein,
Preset configuration threshold value can be manually set, and the disclosure does not do specific value specifically limited.
In addition.The disclosure is to current frame image and the relativeness of reference frame image in video streaming without special limit
It is fixed, for example, current frame image can be the next frame image of reference frame image, in another example, current frame image and reference frame image
Between there are several frame images.
In the case where tracking successful situation, the visual signature that terminal device can be made to obtain is aligned with inertia characteristics, realizes view
Feel the initialization procedure of feature and inertia characteristics.
Specifically, Inertia information can be by means of IMU (Inertial Measurement Unit, the inertia of terminal device
Measuring unit) device acquisition, IMU device may include gyroscope and accelerometer, can distinguish the angle speed of measuring terminals equipment
Degree and acceleration.Since the working frequency of IMU device is usually higher than the frequency that camera acquires image, IMU pre-integration can be used
Mode assess the Inertia information of corresponding frame.Wherein, IMU pre-integration is time-based integral, available correspondence two figures
The Inertia informations such as position, speed and the rotation angle of picture.
Visual information corresponds to the feature of inertia, can use SFM (Structure From Motion, exercise recovery knot
Structure) technology handles two images, determine the information such as corresponding position, speed and rotation angle.Wherein, SFM technology
Corresponding three-dimensional information can be recovered from two-dimensional image or video sequence, that is to say, that input a series of two dimensional images
Or video sequence can export the three-dimensional model information of scene by SFM technology.
Ideally, the Inertia information determined using SFM technology should be with the Inertia information measured based on IMU
Equal.However, the result of actually the two can have deviation due to circuit clock, device measurement accuracy etc., therefore,
It needs for the two to be aligned (Alignment).
In this case it is necessary in initial alignment Inertia information each parameter quantity of state, with ensure using SFM technology it is true
The result that the result made and IMU are measured as close possible to, wherein these quantity of states may include position, speed, rotation angle
Degree, acceleration biasing and offset of gyroscope etc..
If being based on above-mentioned calibration process, the deviation between the result using result and the IMU measurement of SFM technology is determined
Less than a predetermined deviation, then visual signature is aligned with inertia characteristics, that is to say, that visual signature is initialized to inertia characteristics
Function.In such a case, it is possible to optimize the pose of current frame image.
In the illustrative embodiments of the disclosure, used majorized function may include two constraints when optimizing here
Condition, that is, include two cost functions, one is the first constraint condition constructed by monocular characteristic point information, the second is by
Second constraint condition of Inertia information building.Then, it can use nonlinear optimization method to optimize cost function, so that
The value of cost function constantly reduces, and to determine this pose optimum results of current frame image, is denoted as intermediate pose parameter.Its
In, disclosure illustrative embodiments to nonlinear optimization method without particular determination, for example, it may be Gauss-Newton is calculated
Method, column Wen Baige-Ma Kuaerte algorithm etc..
For the process optimized using above-mentioned first constraint condition to current frame image, it is possible, firstly, to present frame
The monocular characteristic point information of image and the monocular characteristic point information of reference frame image carry out trigonometric ratio processing, to determine to work as respectively
The space three-dimensional information of prior image frame and reference frame image based on monocular characteristic point.
Specifically, trigonometric ratio is also referred to as triangulation (Triangulation), refer to same by being observed at two
The angle of point determines the distance of the point.It with reference to Fig. 3, is shot in two positions, camera photocentre is O1 and O2.Characteristic point
P1 character pair point P2, theoretically straight line O1P1 and O2P2 can intersect at a point P in the scene, which is two characteristic points
Position in corresponding three-dimensional scenic.However, two straight lines often can not normally intersect, this due to the influence of noise
In the case of, the position of P point can be solved using least square method.Thus, it is possible to determine present frame using Triangulation Algorithm
The space three-dimensional information of image and reference frame image based on monocular characteristic point.
Next, the weight of the space three-dimensional information of current frame image and reference frame image based on monocular characteristic point can be calculated
Projection error is optimized with the pose to current frame image.Specifically, can use PNP (Perspective-n-Point,
Perspective n point) calculation method determines re-projection error, the disclosure is to specific calculating process without particular determination.
According to other embodiments of the disclosure, if using inclined between the result of result and the IMU measurement of SFM technology
Difference is not less than predetermined deviation, then illustrates that visual signature is unjustified with inertia characteristics, that is to say, that at the beginning of visual signature and inertia characteristics
Beginningization failure.
In such a case, it is possible to the pose based on the constant model prediction current frame image of movement velocity, in the process,
Think that camera is in uniform motion.Can for example search in previous frame image with the matched visual signature point of current frame image, needle
To these visual signature points, using monocular characteristic point information and stereoscopic features point information as constraint condition to the present frame of prediction
The pose of image optimizes.
It is excellent by combining stereoscopic features point information including depth information to carry out the pose of current frame image based on this
Change, improves precision and robustness.
According to other embodiments of the disclosure, matched vision between current frame image and reference frame image is being determined
When the quantity of characteristic point is not more than preset matching threshold value, repositioning process as shown in Figure 4 can be executed.
In step S402, bag of words (Bag of Words, BoW) vector of current frame image is calculated.Specifically, can incite somebody to action
Feature on image regards word one by one as, the dictionary comprising all characteristic types can be trained in advance, accordingly, for image
Feature, can generate the set of an equivalent according to the dictionary, which is combined into bag of words.
In step s 404, the bag of words vector based on current frame image determines multiple candidate figures from key frame data library
Picture.Specifically, the bag of words vector of each key frame images in the bag of words vector and key frame data library of current frame image can be calculated
Between similarity, and determine that similarity meets image that default similarity requires as candidate image.Wherein, default similarity is wanted
Ask can taking human as being configured, for example, default similarity require can for both similarity be greater than 80%, the disclosure to this not
It does specifically limited.
In step S406, determine in each candidate image with the matched visual signature of the visual signature of current frame image point
Point.That is, being directed to each candidate image, ORB characteristic point corresponding with the point map cloud of current frame image is calculated.
In step S408, present frame figure is calculated using with the matched visual signature point of the visual signature of current frame image point
The pose of picture successively calculates each candidate image specifically, can use PNP calculation method, and uses random sampling
The unanimously iterative calculation of (RANdom SAmple Consensus, RANSAC) method, calculates the pose of current frame image.
Target image can be determined from multiple candidate images according to calculated result.Specifically, target image can be with
The most image of current frame image Feature Points Matching.
In step S410, the visual signature point optimization of visual signature point and current frame image based on target image is current
The pose of frame image, to complete repositioning process.
S16. interposition appearance parameter is carried out using the depth information of current frame image and the depth information of reference frame image
Optimization, to determine the pose parameter of current frame image.
In the first constraint condition using the building of monocular characteristic point information and by the second constraint item of Inertia information building
After part optimizes the pose of current frame image, it can use depth information and advanced optimized.
Specifically, firstly, being based on the intermediate pose parameter, determining current frame image after determining intermediate pose parameter
Corresponding cloud map of depth information as first cloud map, and determine corresponding cloud of depth information of reference frame image
Map as second point cloud map,
Next, extracting the geometrical characteristic point of first cloud map and second point cloud map.Wherein, geometrical characteristic point can be with
Including 2 category feature points, the first category feature is sharp edge feature, is denoted as edge point；Second category feature is part plan feature, is denoted as
Planar point.
Then, matched geometrical characteristic point between first cloud map and second point cloud map can be determined.Specifically, can
To cluster 2 category feature points in first cloud map and second point cloud map, each characteristic point can be corresponding with another
Closest point on a cloud map.In such a case, it is possible to the distance function of characteristic point Yu its closest point be established, at this
When the distance of two points is less than a pre-determined distance, then it is assumed that the matching of the two geometrical characteristic points.Tool of the disclosure to pre-determined distance
Body value is not done specifically limited.
It is then possible to establish third constraint condition using the matched set feature point determined, constrained using the third
Condition simultaneously combines nonlinear optimization method, further optimizes to the pose of current frame image.
In addition, some embodiments of the present disclosure can also include judge current frame image whether be key frame images side
Case.
Specifically, may determine that the pose parameter via current frame image after above-mentioned optimization process, if meet default close
Key frame Rule of judgment.For example, the default key frame Rule of judgment can be such as are as follows: the point cloud number of current frame image tracking is less than 50
The point cloud quantity of a or current frame image tracking is less than the 90% of reference frame image point cloud quantity.However, the disclosure is to default
Key frame Rule of judgment does not do specifically limited.
When the pose parameter of current frame image meets default key frame Rule of judgment, current frame image is determined as key
Frame image, and current frame image can be inserted into key frame data library.
It should be understood that the instant positioning of above example description and the optimization method of map structuring are primarily directed to
Track thread in SLAM scheme.Based on above content, on the one hand, map initialization is carried out in conjunction with depth information, compared to adopting
The scheme of map initialization is carried out with pure visual signature, calculating speed is improved；On the other hand, by the visual signature of image
Point is divided into the three-dimensional feature point (i.e. above-mentioned stereoscopic features point) based on depth information and the feature using pending trigonometric ratio
Point (i.e. above-mentioned monocular characteristic point), the two cost function is different, compared to the characteristic point only with pending trigonometric ratio, saves
Runing time；Another aspect, in conjunction with depth information, can effectively solve to omit due to valid data leads to pose estimation inaccuracy
The problem of, and then the precision and robustness of SLAM can be improved.
In addition, some embodiments of the present disclosure, which additionally provide the new part of one kind, builds drawing method, design includes: by image
Depth information as one of the constraint condition for determining local pose so as to build figure more accurate for part.
Firstly, be based on key frame data library, screen the point map cloud of current frame image, with ensure retain point cloud at least by
Other three key frames observe；Then, using light-stream adjustment (Bundle Adjustment, BA) to current frame image with
And above-mentioned associated at least other three key frames carry out local pose optimization, by the monocular feature in the ORB characteristic point of extraction
The constraint condition of point information, stereoscopic features point information and Inertia information (IMU pre-integration information) as local pose optimization, and
It is optimized using nonlinear optimization method.
In addition, the disclosure can also include the scheme for rejecting redundant image.For example, if 90% in a key frame images
Characteristic point can be observed simultaneously by other three key frame images, it is determined that the key frame images are redundant image, are picked
Except in the calculating process of SLAM.
In addition, some embodiments of the present disclosure additionally provide a kind of closed loop detection thread executed parallel.
Firstly, detecting candidate winding frame.Specifically, calculating key frame images in current frame image and key frame data library
Similarity rejects the low key frame of similitude, to determine candidate winding frame, calculates the phase of current frame image to winding key frame
Like transformation matrix, to determine the accumulated error of winding；Next, passing through position of the similitude transformation matrix to current frame image of calculating
Appearance optimizes, and this optimization form can be applied to all key frames adjacent with current frame image；Then, winding key frame
And its neighbouring key frame is capable of determining that all point map clouds both map in current frame image and its adjacent image, and is reflecting
The areas adjacent penetrated determines corresponding match point, will be effective during all matched point map clouds and calculating similarity transformation
Data are merged；Then, parent map optimization is carried out, this process ignores inertial data.Optimize program to rectify by similarity transformation
Positive scale offset, after optimization, each point map cloud is converted according to the correction of key frame；Finally, carrying out global pose
BA optimization and map rejuvenation.
It should be noted that although describing each step of method in the disclosure in the accompanying drawings with particular order, this is simultaneously
Undesired or hint must execute these steps in this particular order, or have to carry out the ability of step shown in whole
Realize desired result.Additional or alternative, it is convenient to omit multiple steps are merged into a step and executed by certain steps,
And/or a step is decomposed into execution of multiple steps etc..
Further, the optimization device of a kind of positioning and map structuring immediately is additionally provided in this example embodiment.
Fig. 5 diagrammatically illustrates the instant of the illustrative embodiments of the disclosure and positions and the optimization device of map structuring
Block diagram.With reference to Fig. 5, the optimization device 5 of instant positioning according to an exemplary embodiment of the present disclosure and map structuring can be with
Including characteristic determination module 51, the first optimization module 53 and the second optimization module 55.
Specifically, characteristic determination module 51 can be used for extracting the visual signature point of current frame image, and determine that vision is special
The characteristic point unrelated to the depth information of current frame image in sign point, as monocular characteristic point；First optimization module 53 can
If to be greater than preset matching threshold value for the quantity of visual signature point matched between current frame image and reference frame image,
When visual signature is aligned with inertia characteristics, believe using the first constraint condition constructed by monocular characteristic point information and by inertia
Second constraint condition of breath building optimizes the pose of current frame image, obtains intermediate pose parameter；Second optimization module
55 can be used for using current frame image depth information and reference frame image depth information to interposition appearance parameter carry out it is excellent
Change, to determine the pose parameter of current frame image.
According to the optimization device of the instant positioning and map structuring of disclosure illustrative embodiments, in conjunction with depth information pair
Pose optimizes, and avoids and optimizes the problem of may cause characteristic point omission only with visual information, can be improved
The precision and robustness of SLAM.
According to an exemplary embodiment of the present disclosure, with reference to Fig. 6, the optimization device 6 of positioning immediately and map structuring compared to
Immediately the optimization device 5 of positioning and map structuring can also include reorientation module 61.
Specifically, reorientation module 61 can be configured as execution: if between current frame image and reference frame image
The quantity for the visual signature point matched is not more than preset matching threshold value, then executes following repositioning process: calculating current frame image
Bag of words vector；Bag of words vector based on current frame image determines multiple candidate images from key frame data library；Determine each time
Select in image with the matched visual signature point of the visual signature of current frame image point；Using with the visual signature of current frame image point
Matched visual signature point calculates the pose of current frame image, and target figure is determined from multiple candidate images according to calculated result
Picture；The pose of the visual signature point optimization current frame image of visual signature point and current frame image based on target image.
According to an exemplary embodiment of the present disclosure, with reference to Fig. 7, characteristic determination module 51 may include feature division unit
701。
Specifically, feature division unit 701 can be configured as execution: obtaining the depth information of the current frame image；
Determine whether visual signature point is associated with depth information based on the outer ginseng matrix demarcated in advance；It wherein, will be in visual signature point
Characteristic point associated with depth information is determined as stereoscopic features point.
According to an exemplary embodiment of the present disclosure, the first optimization module 53 can be additionally configured to execute: in visual signature
When unjustified with inertia characteristics, believe based on the pose of the constant model prediction current frame image of movement velocity, and by monocular characteristic point
Breath and stereoscopic features point information are optimized as pose of the constraint condition to the current frame image of prediction.
According to an exemplary embodiment of the present disclosure, the first optimization module 53 is executed using the first constraint condition to present frame figure
The process that the pose of picture optimizes can be configured as: monocular characteristic point information and reference frame image to current frame image
Monocular characteristic point information carries out trigonometric ratio processing, to determine current frame image and reference frame image based on monocular characteristic point respectively
Space three-dimensional information；The re-projection for calculating the space three-dimensional information of current frame image and reference frame image based on monocular characteristic point misses
Difference is optimized with the pose to current frame image.
According to an exemplary embodiment of the present disclosure, with reference to Fig. 8, the optimization device 8 of positioning immediately and map structuring compared to
Immediately the optimization device 5 of positioning and map structuring, can also include reference frame determining module 81.
Specifically, reference frame determining module 81 can be configured to execute: the visual signature point of an input picture is extracted in advance,
And determine the monocular characteristic point and stereoscopic features point of input picture；The quantity of the visual signature point of input picture is preset with first
Threshold value is compared, and the quantity of the stereoscopic features of input picture point is compared with the second preset threshold；If input figure
The quantity of the visual signature point of picture is greater than the first preset threshold, and the quantity of the stereoscopic features point of input picture is greater than second in advance
If threshold value, then using the input picture as reference frame image.
According to an exemplary embodiment of the present disclosure, the second optimization module 55 can be configured as execution: be based on intermediate pose
Parameter determines that corresponding cloud map of the depth information of current frame image as first cloud map, and determines reference frame image
Corresponding cloud map of depth information as second point cloud map；Extract the geometry of first cloud map and second point cloud map
Characteristic point, and determine matched geometrical characteristic point between first cloud map and second point cloud map；It is special using matched geometry
The information architecture third constraint condition of point is levied, and interposition appearance parameter is optimized using third constraint condition.
According to an exemplary embodiment of the present disclosure, with reference to Fig. 9, the optimization device 9 of positioning immediately and map structuring compared to
Immediately the optimization device 5 of positioning and map structuring, can also include key frame determining module 91.
Specifically, key frame determining module 91 can be configured as execution: meeting in the pose parameter of current frame image pre-
If when key frame Rule of judgment, current frame image is determined as key frame images, and key frame images are inserted into key frame data
In library.
According to an exemplary embodiment of the present disclosure, with reference to Figure 10, positioning is compared with the optimization device 10 of map structuring immediately
It can also include that figure optimization module 101 is built in part in the optimization device 5 of instant positioning and map structuring.
Specifically, part builds figure optimization module 101 and can be configured as execution: being key frame images in current frame image
When, during locally building figure, believed using the monocular characteristic point information of current frame image, stereoscopic features point information and inertia
Breath is used as constraint condition, and the process of figure is built in optimization part.
Since each functional module and the above method of the program analysis of running performance device of embodiment of the present invention are invented
It is identical in embodiment, therefore details are not described herein.
In an exemplary embodiment of the disclosure, a kind of computer readable storage medium is additionally provided, energy is stored thereon with
Enough realize the program product of this specification above method.In some possible embodiments, various aspects of the invention may be used also
In the form of being embodied as a kind of program product comprising program code, when described program product is run on the terminal device, institute
Program code is stated for executing the terminal device described in above-mentioned " illustrative methods " part of this specification according to this hair
The step of bright various illustrative embodiments.
The program product for realizing the above method of embodiment according to the present invention can use portable compact disc
Read-only memory (CD-ROM) and including program code, and can be run on terminal device, such as PC.However, this
The program product of invention is without being limited thereto, in this document, readable storage medium storing program for executing can be it is any include or storage program it is tangible
Medium, the program can be commanded execution system, device or device use or in connection.
Described program product can be using any combination of one or more readable mediums.Readable medium can be readable letter
Number medium or readable storage medium storing program for executing.Readable storage medium storing program for executing for example can be but be not limited to electricity, magnetic, optical, electromagnetic, infrared ray or
System, device or the device of semiconductor, or any above combination.The more specific example of readable storage medium storing program for executing is (non exhaustive
List) include: electrical connection with one or more conducting wires, portable disc, hard disk, random access memory (RAM), read-only
Memory (ROM), erasable programmable read only memory (EPROM or flash memory), CD, portable compact disc read only memory
(CD-ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
In carry readable program code.The data-signal of this propagation can take various forms, including but not limited to electromagnetic signal,
Optical signal or above-mentioned any appropriate combination.Readable signal medium can also be any readable Jie other than readable storage medium storing program for executing
Matter, the readable medium can send, propagate or transmit for by instruction execution system, device or device use or and its
The program of combined use.
The program code for including on readable medium can transmit with any suitable medium, including but not limited to wirelessly, have
Line, optical cable, RF etc. or above-mentioned any appropriate combination.
The program for executing operation of the present invention can be write with any combination of one or more programming languages
Code, described program design language include object oriented program language-Java, C++ etc., further include conventional
Procedural programming language-such as " C " language or similar programming language.Program code can be fully in user
It calculates and executes in equipment, partly executes on a user device, being executed as an independent software package, partially in user's calculating
Upper side point is executed on a remote computing or is executed in remote computing device or server completely.It is being related to far
Journey calculates in the situation of equipment, and remote computing device can pass through the network of any kind, including local area network (LAN) or wide area network
(WAN), it is connected to user calculating equipment, or, it may be connected to external computing device (such as utilize ISP
To be connected by internet).
In an exemplary embodiment of the disclosure, a kind of electronic equipment that can be realized the above method is additionally provided.
Person of ordinary skill in the field it is understood that various aspects of the invention can be implemented as system, method or
Program product.Therefore, various aspects of the invention can be embodied in the following forms, it may be assumed that complete hardware embodiment, complete
The embodiment combined in terms of full Software Implementation (including firmware, microcode etc.) or hardware and software, can unite here
Referred to as circuit, " module " or " system ".
The electronic equipment 1100 of this embodiment according to the present invention is described referring to Figure 11.The electricity that Figure 11 is shown
Sub- equipment 1100 is only an example, should not function to the embodiment of the present invention and use scope bring any restrictions.
As shown in figure 11, electronic equipment 1100 is showed in the form of universal computing device.The component of electronic equipment 1100 can
To include but is not limited to: at least one above-mentioned processing unit 1110, connects not homologous ray at least one above-mentioned storage unit 1120
The bus 1130 of component (including storage unit 1120 and processing unit 1110), display unit 1140.
Wherein, the storage unit is stored with program code, and said program code can be held by the processing unit 1110
Row, so that various according to the present invention described in the execution of the processing unit 1110 above-mentioned " illustrative methods " part of this specification
The step of illustrative embodiments.For example, the processing unit 1110 can execute step S12 as shown in fig. 1 to step
S16。
Storage unit 1120 may include the readable medium of volatile memory cell form, such as Random Access Storage Unit
(RAM) 11201 and/or cache memory unit 11202, it can further include read-only memory unit (ROM) 11203.
Storage unit 1120 can also include program/utility with one group of (at least one) program module 11205
11204, such program module 11205 includes but is not limited to: operating system, one or more application program, other programs
It may include the realization of network environment in module and program data, each of these examples or certain combination.
Bus 1130 can be to indicate one of a few class bus structures or a variety of, including storage unit bus or storage
Cell controller, peripheral bus, graphics acceleration port, processing unit use any bus structures in a variety of bus structures
Local bus.
Electronic equipment 1100 can also be with one or more external equipments 1200 (such as keyboard, sensing equipment, bluetooth equipment
Deng) communication, can also be enabled a user to one or more equipment interact with the electronic equipment 1100 communicate, and/or with make
The electronic equipment 1100 can with it is one or more of the other calculating equipment be communicated any equipment (such as router, modulation
Demodulator etc.) communication.This communication can be carried out by input/output (I/O) interface 1150.Also, electronic equipment 1100
Network adapter 1160 and one or more network (such as local area network (LAN), wide area network (WAN) and/or public affairs can also be passed through
Common network network, such as internet) communication.As shown, network adapter 1160 passes through its of bus 1130 and electronic equipment 1100
The communication of its module.It should be understood that although not shown in the drawings, other hardware and/or software can be used in conjunction with electronic equipment 1100
Module, including but not limited to: microcode, device driver, redundant processing unit, external disk drive array, RAID system, magnetic
Tape drive and data backup storage system etc..
Through the above description of the embodiments, those skilled in the art is it can be readily appreciated that example described herein is implemented
Mode can also be realized by software realization in such a way that software is in conjunction with necessary hardware.Therefore, according to the disclosure
The technical solution of embodiment can be embodied in the form of software products, which can store non-volatile at one
Property storage medium (can be CD-ROM, USB flash disk, mobile hard disk etc.) in or network on, including some instructions are so that a calculating
Equipment (can be personal computer, server, terminal installation or network equipment etc.) is executed according to disclosure embodiment
Method.
In addition, above-mentioned attached drawing is only the schematic theory of processing included by method according to an exemplary embodiment of the present invention
It is bright, rather than limit purpose.It can be readily appreciated that the time that above-mentioned processing shown in the drawings did not indicated or limited these processing is suitable
Sequence.In addition, be also easy to understand, these processing, which can be, for example either synchronously or asynchronously to be executed in multiple modules.
It should be noted that although being referred to several modules or list for acting the equipment executed in the above detailed description
Member, but this division is not enforceable.In fact, according to embodiment of the present disclosure, it is above-described two or more
Module or the feature and function of unit can embody in a module or unit.Conversely, an above-described mould
The feature and function of block or unit can be to be embodied by multiple modules or unit with further division.
Those skilled in the art after considering the specification and implementing the invention disclosed here, will readily occur to its of the disclosure
His embodiment.This application is intended to cover any variations, uses, or adaptations of the disclosure, these modifications, purposes or
Adaptive change follow the general principles of this disclosure and including the undocumented common knowledge in the art of the disclosure or
Conventional techniques.The description and examples are only to be considered as illustrative, and the true scope and spirit of the disclosure are by claim
It points out.
It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and
And various modifications and changes may be made without departing from the scope thereof.The scope of the present disclosure is only limited by the attached claims.
Claims (12)
1. a kind of optimization method of positioning and map structuring immediately characterized by comprising
The visual signature point of current frame image is extracted, and determines in the visual signature point and believes with the depth of the current frame image
Unrelated characteristic point is ceased, as monocular characteristic point；
If the quantity of matched visual signature point is greater than preset matching threshold value between the current frame image and reference frame image,
Then when visual signature is aligned with inertia characteristics, using the first constraint condition constructed by monocular characteristic point information and by inertia
Second constraint condition of information architecture optimizes the pose of the current frame image, obtains intermediate pose parameter；
Using the depth information of the current frame image and the depth information of the reference frame image to the intermediate pose parameter
It optimizes, with the pose parameter of the determination current frame image.
2. the optimization method of instant positioning and map structuring according to claim 1, which is characterized in that the instant positioning
With the optimization method of map structuring further include:
If the quantity of matched visual signature point is no more than described default between the current frame image and reference frame image
With threshold value, then following repositioning process is executed:
Calculate the bag of words vector of the current frame image；
Bag of words vector based on the current frame image determines multiple candidate images from key frame data library；
Determine in each candidate image with the matched visual signature point of the visual signature point of the current frame image；
Using the pose for calculating the current frame image with the matched visual signature point of the visual signature of current frame image point,
And target image is determined from the multiple candidate image according to calculated result；
The visual signature point of visual signature point and the current frame image based on the target image optimizes the present frame figure
The pose of picture.
3. the optimization method of instant positioning and map structuring according to claim 1, which is characterized in that the instant positioning
With the optimization method of map structuring further include:
Obtain the depth information of the current frame image；
Determine whether the visual signature point is associated with the depth information based on the outer ginseng matrix demarcated in advance；
Wherein, characteristic point associated with the depth information in the visual signature point is determined as stereoscopic features point.
4. the optimization method of instant positioning and map structuring according to claim 3, which is characterized in that the instant positioning
With the optimization method of map structuring further include:
When visual signature and inertia characteristics are unjustified, the position based on current frame image described in the constant model prediction of movement velocity
Appearance, and using monocular characteristic point information and stereoscopic features point information as constraint condition to the position of the current frame image of prediction
Appearance optimizes.
5. the optimization method of instant positioning and map structuring according to claim 1, which is characterized in that using by monocular spy
First constraint condition of sign point information architecture, which optimizes the pose of the current frame image, includes:
The monocular characteristic point information of monocular characteristic point information and the reference frame image to the current frame image carries out triangle
Change processing, to determine the space three-dimensional information of the current frame image and the reference frame image based on monocular characteristic point respectively；
The re-projection for calculating the space three-dimensional information of the current frame image and the reference frame image based on monocular characteristic point misses
Difference is optimized with the pose to the current frame image.
6. the optimization method of instant positioning and map structuring according to claim 3, which is characterized in that the instant positioning
With the optimization method of map structuring further include:
The visual signature point of an input picture is extracted in advance, and determines the monocular characteristic point and stereoscopic features of the input picture
Point；
The quantity of the visual signature point of the input picture is compared with the first preset threshold, and by the input picture
The quantity of stereoscopic features point is compared with the second preset threshold；
If the quantity of the visual signature point of the input picture is greater than first preset threshold, and the input picture
The quantity of stereoscopic features point is greater than second preset threshold, then using the input picture as the reference frame image.
7. the optimization method of instant positioning and map structuring according to claim 1, which is characterized in that utilize described current
The depth information of frame image and the depth information of the reference frame image, which optimize the intermediate pose parameter, includes:
Based on the intermediate pose parameter, determine corresponding cloud map of the depth information of the current frame image as first point
Cloud map, and determine corresponding cloud map of depth information of the reference frame image as second point cloud map；
The geometrical characteristic point of first cloud map and the second point cloud map is extracted, and determines first cloud map
The matched geometrical characteristic point between the second point cloud map；
Using the information architecture third constraint condition of the matched geometrical characteristic point, and using the third constraint condition to institute
Intermediate pose parameter is stated to optimize.
8. the optimization method of instant positioning and map structuring according to any one of claim 1 to 7, which is characterized in that
The optimization method of the instant positioning and map structuring further include:
When the pose parameter of the current frame image meets default key frame Rule of judgment, the current frame image is determined as
Key frame images, and the key frame images are inserted into key frame data library.
9. the optimization method of instant positioning and map structuring according to claim 8, which is characterized in that the instant positioning
With the optimization method of map structuring further include:
When the current frame image is key frame images, during locally building figure, the list of the current frame image is utilized
Mesh characteristic point information, stereoscopic features point information and Inertia information optimize the process that figure is built in the part as constraint condition.
10. a kind of optimization device of positioning and map structuring immediately characterized by comprising
Characteristic determination module, for extracting the visual signature point of current frame image, and determine in the visual signature point with it is described
The unrelated characteristic point of the depth information of current frame image, as monocular characteristic point；
First optimization module, if the quantity for visual signature point matched between the current frame image and reference frame image
Greater than preset matching threshold value, then when visual signature is aligned with inertia characteristics, first constructed by monocular characteristic point information is utilized
Constraint condition and the pose of the current frame image optimized by the second constraint condition that Inertia information constructs, is obtained
Between pose parameter；
Second optimization module, the depth information pair for depth information and the reference frame image using the current frame image
The intermediate pose parameter optimizes, with the pose parameter of the determination current frame image.
11. a kind of storage medium, is stored thereon with computer program, which is characterized in that the computer program is held by processor
The optimization method positioned immediately described in any one of claims 1 to 9 with map structuring is realized when row.
12. a kind of electronic equipment characterized by comprising
Processor；And
Memory, for storing the executable instruction of the processor；
Wherein, the processor is configured to come described in any one of perform claim requirement 1 to 9 via the execution executable instruction
Instant positioning and map structuring optimization method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910578527.5A CN110322500B (en) | 2019-06-28 | 2019-06-28 | Optimization method and device for instant positioning and map construction, medium and electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910578527.5A CN110322500B (en) | 2019-06-28 | 2019-06-28 | Optimization method and device for instant positioning and map construction, medium and electronic equipment |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110322500A true CN110322500A (en) | 2019-10-11 |
| CN110322500B CN110322500B (en) | 2023-08-15 |
Family
ID=68120757
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910578527.5A Active CN110322500B (en) | 2019-06-28 | 2019-06-28 | Optimization method and device for instant positioning and map construction, medium and electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110322500B (en) |
Cited By (30)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110796706A (en) * | 2019-11-08 | 2020-02-14 | 四川长虹电器股份有限公司 | Visual positioning method and system |
| CN110866953A (en) * | 2019-10-31 | 2020-03-06 | Oppo广东移动通信有限公司 | Map construction method and device, and positioning method and device |
| CN111127584A (en) * | 2019-11-19 | 2020-05-08 | 奇点汽车研发中心有限公司 | Method and device for establishing visual map, electronic equipment and storage medium |
| CN111311684A (en) * | 2020-04-01 | 2020-06-19 | 亮风台（上海）信息科技有限公司 | Method and equipment for initializing SLAM |
| CN111337947A (en) * | 2020-05-18 | 2020-06-26 | 深圳市智绘科技有限公司 | Instant mapping and positioning method, device, system and storage medium |
| CN111462029A (en) * | 2020-03-27 | 2020-07-28 | 北京百度网讯科技有限公司 | Visual point cloud and high-precision map fusion method and device and electronic equipment |
| CN111649724A (en) * | 2020-06-04 | 2020-09-11 | 百度在线网络技术（北京）有限公司 | Visual positioning method and device based on mobile edge calculation |
| CN111652933A (en) * | 2020-05-06 | 2020-09-11 | Oppo广东移动通信有限公司 | Monocular camera-based repositioning method and device, storage medium and electronic equipment |
| CN111724439A (en) * | 2019-11-29 | 2020-09-29 | 中国科学院上海微系统与信息技术研究所 | Visual positioning method and device in dynamic scene |
| CN111998846A (en) * | 2020-07-24 | 2020-11-27 | 中山大学 | Unmanned system rapid relocation method based on environment geometry and topological characteristics |
| CN112102400A (en) * | 2020-09-15 | 2020-12-18 | 上海云绅智能科技有限公司 | Distance-based closed loop detection method and device, electronic equipment and storage medium |
| CN112258647A (en) * | 2020-11-12 | 2021-01-22 | Oppo广东移动通信有限公司 | Map reconstruction method and device, computer readable medium and electronic device |
| CN112446951A (en) * | 2020-11-06 | 2021-03-05 | 杭州易现先进科技有限公司 | Three-dimensional reconstruction method and device, electronic equipment and computer storage medium |
| CN112819860A (en) * | 2021-02-18 | 2021-05-18 | Oppo广东移动通信有限公司 | Visual inertial system initialization method and device, medium and electronic equipment |
| CN112880675A (en) * | 2021-01-22 | 2021-06-01 | 京东数科海益信息科技有限公司 | Pose smoothing method and device for visual positioning, terminal and mobile robot |
| CN112884840A (en) * | 2021-03-29 | 2021-06-01 | 湖南国科微电子股份有限公司 | Visual positioning method, device, equipment and storage medium |
| CN113034594A (en) * | 2021-03-16 | 2021-06-25 | 浙江商汤科技开发有限公司 | Pose optimization method and device, electronic equipment and storage medium |
| CN113034582A (en) * | 2021-03-25 | 2021-06-25 | 浙江商汤科技开发有限公司 | Pose optimization device and method, electronic device and computer readable storage medium |
| CN113034538A (en) * | 2019-12-25 | 2021-06-25 | 杭州海康威视数字技术股份有限公司 | Pose tracking method and device of visual inertial navigation equipment and visual inertial navigation equipment |
| CN113096185A (en) * | 2021-03-29 | 2021-07-09 | Oppo广东移动通信有限公司 | Visual positioning method, visual positioning device, storage medium and electronic equipment |
| CN113129366A (en) * | 2020-01-10 | 2021-07-16 | 北京字节跳动网络技术有限公司 | Monocular SLAM (simultaneous localization and mapping) initialization method and device and electronic equipment |
| CN113124854A (en) * | 2019-12-31 | 2021-07-16 | 杭州海康机器人技术有限公司 | Visual positioning method, map construction method and map construction device |
| CN113269876A (en) * | 2021-05-10 | 2021-08-17 | Oppo广东移动通信有限公司 | Map point coordinate optimization method and device, electronic equipment and storage medium |
| CN113326769A (en) * | 2021-05-28 | 2021-08-31 | 北京三快在线科技有限公司 | High-precision map generation method, device, equipment and storage medium |
| CN113409368A (en) * | 2020-03-16 | 2021-09-17 | 北京京东乾石科技有限公司 | Drawing method and device, computer readable storage medium and electronic equipment |
| CN113689485A (en) * | 2021-08-25 | 2021-11-23 | 北京三快在线科技有限公司 | Method and device for determining depth information of unmanned aerial vehicle, unmanned aerial vehicle and storage medium |
| CN113776530A (en) * | 2020-09-11 | 2021-12-10 | 北京京东乾石科技有限公司 | Point cloud map construction method and device, electronic equipment and storage medium |
| CN115982399A (en) * | 2023-03-16 | 2023-04-18 | 北京集度科技有限公司 | Image searching method, mobile device, electronic device and computer program product |
| CN116030136A (en) * | 2023-03-29 | 2023-04-28 | 中国人民解放军国防科技大学 | Cross-view visual positioning method and device based on geometric features and computer equipment |
| WO2023151251A1 (en) * | 2022-02-09 | 2023-08-17 | 上海商汤智能科技有限公司 | Map construction method and apparatus, pose determination method and apparatus, and device and computer program product |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019043233A1 (en) * | 2017-09-04 | 2019-03-07 | Universität Zürich | Visual-inertial odometry with an event camera |
| CN109816696A (en) * | 2019-02-01 | 2019-05-28 | 西安全志科技有限公司 | A kind of robot localization and build drawing method, computer installation and computer readable storage medium |
-
2019
- 2019-06-28 CN CN201910578527.5A patent/CN110322500B/en active Active
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2019043233A1 (en) * | 2017-09-04 | 2019-03-07 | Universität Zürich | Visual-inertial odometry with an event camera |
| CN109816696A (en) * | 2019-02-01 | 2019-05-28 | 西安全志科技有限公司 | A kind of robot localization and build drawing method, computer installation and computer readable storage medium |
Non-Patent Citations (1)
| Title |
|---|
| 林辉灿等: "鲁棒的非线性优化的立体视觉-惯导SLAM", 《机器人》 * |
Cited By (47)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110866953A (en) * | 2019-10-31 | 2020-03-06 | Oppo广东移动通信有限公司 | Map construction method and device, and positioning method and device |
| CN110866953B (en) * | 2019-10-31 | 2023-12-29 | Oppo广东移动通信有限公司 | Map construction method and device, and positioning method and device |
| CN110796706A (en) * | 2019-11-08 | 2020-02-14 | 四川长虹电器股份有限公司 | Visual positioning method and system |
| CN111127584A (en) * | 2019-11-19 | 2020-05-08 | 奇点汽车研发中心有限公司 | Method and device for establishing visual map, electronic equipment and storage medium |
| CN111724439B (en) * | 2019-11-29 | 2024-05-17 | 中国科学院上海微系统与信息技术研究所 | Visual positioning method and device under dynamic scene |
| CN111724439A (en) * | 2019-11-29 | 2020-09-29 | 中国科学院上海微系统与信息技术研究所 | Visual positioning method and device in dynamic scene |
| CN113034538A (en) * | 2019-12-25 | 2021-06-25 | 杭州海康威视数字技术股份有限公司 | Pose tracking method and device of visual inertial navigation equipment and visual inertial navigation equipment |
| CN113034538B (en) * | 2019-12-25 | 2023-09-05 | 杭州海康威视数字技术股份有限公司 | Pose tracking method and device of visual inertial navigation equipment and visual inertial navigation equipment |
| CN113124854A (en) * | 2019-12-31 | 2021-07-16 | 杭州海康机器人技术有限公司 | Visual positioning method, map construction method and map construction device |
| CN113129366A (en) * | 2020-01-10 | 2021-07-16 | 北京字节跳动网络技术有限公司 | Monocular SLAM (simultaneous localization and mapping) initialization method and device and electronic equipment |
| CN113129366B (en) * | 2020-01-10 | 2024-04-30 | 北京字节跳动网络技术有限公司 | Monocular SLAM initialization method and device and electronic equipment |
| CN113409368B (en) * | 2020-03-16 | 2023-11-03 | 北京京东乾石科技有限公司 | Mapping method and device, computer readable storage medium and electronic equipment |
| CN113409368A (en) * | 2020-03-16 | 2021-09-17 | 北京京东乾石科技有限公司 | Drawing method and device, computer readable storage medium and electronic equipment |
| CN111462029B (en) * | 2020-03-27 | 2023-03-03 | 阿波罗智能技术(北京)有限公司 | Visual point cloud and high-precision map fusion method and device and electronic equipment |
| CN111462029A (en) * | 2020-03-27 | 2020-07-28 | 北京百度网讯科技有限公司 | Visual point cloud and high-precision map fusion method and device and electronic equipment |
| CN111311684A (en) * | 2020-04-01 | 2020-06-19 | 亮风台（上海）信息科技有限公司 | Method and equipment for initializing SLAM |
| CN111652933A (en) * | 2020-05-06 | 2020-09-11 | Oppo广东移动通信有限公司 | Monocular camera-based repositioning method and device, storage medium and electronic equipment |
| CN111652933B (en) * | 2020-05-06 | 2023-08-04 | Oppo广东移动通信有限公司 | Repositioning method and device based on monocular camera, storage medium and electronic equipment |
| CN111337947B (en) * | 2020-05-18 | 2020-09-22 | 深圳市智绘科技有限公司 | Instant mapping and positioning method, device, system and storage medium |
| CN111337947A (en) * | 2020-05-18 | 2020-06-26 | 深圳市智绘科技有限公司 | Instant mapping and positioning method, device, system and storage medium |
| CN111649724A (en) * | 2020-06-04 | 2020-09-11 | 百度在线网络技术（北京）有限公司 | Visual positioning method and device based on mobile edge calculation |
| US11380012B2 (en) | 2020-06-04 | 2022-07-05 | Baidu Online Network Technology (Beijing) Co., Ltd. | Method and apparatus for visual positioning based on mobile edge computing |
| CN111998846B (en) * | 2020-07-24 | 2023-05-05 | 中山大学 | Unmanned system rapid repositioning method based on environment geometry and topological characteristics |
| CN111998846A (en) * | 2020-07-24 | 2020-11-27 | 中山大学 | Unmanned system rapid relocation method based on environment geometry and topological characteristics |
| CN113776530A (en) * | 2020-09-11 | 2021-12-10 | 北京京东乾石科技有限公司 | Point cloud map construction method and device, electronic equipment and storage medium |
| CN112102400A (en) * | 2020-09-15 | 2020-12-18 | 上海云绅智能科技有限公司 | Distance-based closed loop detection method and device, electronic equipment and storage medium |
| CN112102400B (en) * | 2020-09-15 | 2022-08-02 | 上海云绅智能科技有限公司 | Distance-based closed loop detection method and device, electronic equipment and storage medium |
| CN112446951A (en) * | 2020-11-06 | 2021-03-05 | 杭州易现先进科技有限公司 | Three-dimensional reconstruction method and device, electronic equipment and computer storage medium |
| CN112446951B (en) * | 2020-11-06 | 2024-03-26 | 杭州易现先进科技有限公司 | Three-dimensional reconstruction method, three-dimensional reconstruction device, electronic equipment and computer storage medium |
| CN112258647A (en) * | 2020-11-12 | 2021-01-22 | Oppo广东移动通信有限公司 | Map reconstruction method and device, computer readable medium and electronic device |
| CN112258647B (en) * | 2020-11-12 | 2024-04-02 | Oppo广东移动通信有限公司 | Map reconstruction method and device, computer readable medium and electronic equipment |
| CN112880675A (en) * | 2021-01-22 | 2021-06-01 | 京东数科海益信息科技有限公司 | Pose smoothing method and device for visual positioning, terminal and mobile robot |
| CN112819860A (en) * | 2021-02-18 | 2021-05-18 | Oppo广东移动通信有限公司 | Visual inertial system initialization method and device, medium and electronic equipment |
| CN112819860B (en) * | 2021-02-18 | 2023-12-22 | Oppo广东移动通信有限公司 | Visual inertial system initialization method and device, medium and electronic equipment |
| WO2022193508A1 (en) * | 2021-03-16 | 2022-09-22 | 浙江商汤科技开发有限公司 | Method and apparatus for posture optimization, electronic device, computer-readable storage medium, computer program, and program product |
| CN113034594A (en) * | 2021-03-16 | 2021-06-25 | 浙江商汤科技开发有限公司 | Pose optimization method and device, electronic equipment and storage medium |
| CN113034582A (en) * | 2021-03-25 | 2021-06-25 | 浙江商汤科技开发有限公司 | Pose optimization device and method, electronic device and computer readable storage medium |
| CN113096185B (en) * | 2021-03-29 | 2023-06-06 | Oppo广东移动通信有限公司 | Visual positioning method, visual positioning device, storage medium and electronic equipment |
| CN112884840A (en) * | 2021-03-29 | 2021-06-01 | 湖南国科微电子股份有限公司 | Visual positioning method, device, equipment and storage medium |
| CN113096185A (en) * | 2021-03-29 | 2021-07-09 | Oppo广东移动通信有限公司 | Visual positioning method, visual positioning device, storage medium and electronic equipment |
| CN113269876A (en) * | 2021-05-10 | 2021-08-17 | Oppo广东移动通信有限公司 | Map point coordinate optimization method and device, electronic equipment and storage medium |
| CN113326769A (en) * | 2021-05-28 | 2021-08-31 | 北京三快在线科技有限公司 | High-precision map generation method, device, equipment and storage medium |
| CN113689485A (en) * | 2021-08-25 | 2021-11-23 | 北京三快在线科技有限公司 | Method and device for determining depth information of unmanned aerial vehicle, unmanned aerial vehicle and storage medium |
| CN113689485B (en) * | 2021-08-25 | 2022-06-07 | 北京三快在线科技有限公司 | Method and device for determining depth information of unmanned aerial vehicle, unmanned aerial vehicle and storage medium |
| WO2023151251A1 (en) * | 2022-02-09 | 2023-08-17 | 上海商汤智能科技有限公司 | Map construction method and apparatus, pose determination method and apparatus, and device and computer program product |
| CN115982399A (en) * | 2023-03-16 | 2023-04-18 | 北京集度科技有限公司 | Image searching method, mobile device, electronic device and computer program product |
| CN116030136A (en) * | 2023-03-29 | 2023-04-28 | 中国人民解放军国防科技大学 | Cross-view visual positioning method and device based on geometric features and computer equipment |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110322500B (en) | 2023-08-15 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110322500A (en) | Immediately optimization method and device, medium and the electronic equipment of positioning and map structuring | |
| Memo et al. | Head-mounted gesture controlled interface for human-computer interaction | |
| CN110349212A (en) | Immediately optimization method and device, medium and the electronic equipment of positioning and map structuring | |
| CN109947886B (en) | Image processing method, image processing device, electronic equipment and storage medium | |
| KR102292028B1 (en) | Gesture recognition method, device, electronic device, and storage medium | |
| US8467596B2 (en) | Method and apparatus for object pose estimation | |
| JP4972201B2 (en) | Man-machine interaction based on sign | |
| EP3811284A1 (en) | Object-based localization | |
| CN110986969B (en) | Map fusion method and device, equipment and storage medium | |
| CN110335316A (en) | Method, apparatus, medium and electronic equipment are determined based on the pose of depth information | |
| US10580148B2 (en) | Graphical coordinate system transform for video frames | |
| WO2020228643A1 (en) | Interactive control method and apparatus, electronic device and storage medium | |
| JP6571108B2 (en) | Real-time 3D gesture recognition and tracking system for mobile devices | |
| EP4006847A1 (en) | Virtual object processing method and apparatus, and storage medium and electronic device | |
| Hernández-Vela et al. | BoVDW: Bag-of-Visual-and-Depth-Words for gesture recognition | |
| US11842514B1 (en) | Determining a pose of an object from rgb-d images | |
| CN111094895A (en) | System and method for robust self-repositioning in pre-constructed visual maps | |
| Wang et al. | A LINE-MOD-based markerless tracking approachfor AR applications | |
| US20160210761A1 (en) | 3d reconstruction | |
| US11263818B2 (en) | Augmented reality system using visual object recognition and stored geometry to create and render virtual objects | |
| CN110070578B (en) | Loop detection method | |
| Kerdvibulvech | Hand tracking by extending distance transform and hand model in real-time | |
| Akman et al. | Multi-cue hand detection and tracking for a head-mounted augmented reality system | |
| CN107944337A (en) | A kind of low target intelligent-tracking method and system, storage medium and electric terminal | |
| US10796435B2 (en) | Image processing method and image processing apparatus |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |