CN109816726A - A kind of visual odometry map updating method and system based on depth filter - Google Patents
A kind of visual odometry map updating method and system based on depth filter Download PDFInfo
- Publication number
- CN109816726A CN109816726A CN201910083820.4A CN201910083820A CN109816726A CN 109816726 A CN109816726 A CN 109816726A CN 201910083820 A CN201910083820 A CN 201910083820A CN 109816726 A CN109816726 A CN 109816726A
- Authority
- CN
- China
- Prior art keywords
- seed
- depth
- frame image
- depth filter
- map
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Image Analysis (AREA)
- Length Measuring Devices By Optical Means (AREA)
Abstract
The invention discloses a kind of visual odometry map updating method, system, computer readable storage medium and computer equipment based on depth filter, the update method includes: to judge whether to need to add key frame images according to the depth information characteristic value of the fisrt feature point of the current frame image of acquisition, updates the seed of the existing depth filter according to the fisrt feature point of the current frame image using existing depth filter if not needing addition；If desired it adds, establishes and initialize new depth filter, determine key frame images and update the seed of the new depth filter according to the key frame images using the new depth filter；The map is added when the convergence of the estimation of Depth of the seed to update the map in the probability distribution for calculating and updating the seed.Embodiment provided by the invention can dramatically speed up the convergence rate of depth filter, effectively promote the robustness of visual odometry map rejuvenation.
Description
Technical field
The present invention relates to visual odometry technical fields, more particularly to a kind of visual odometry based on depth filter
Map updating method, system, computer readable storage medium and computer equipment.
Background technique
Currently, SLAM (Simultaneous localization and mapping, simultaneous localization and mapping) makees
To establish environmental model and estimating a cutting edge technology of displacement, it is able to solve in the case where no environment priori, essence
True ground position location, posture and building map, are the important component of VR, AR and the fields such as unmanned.
Current VR, AR product generally require can movement for itself and position assess, to match it
The scene content of rendering provides good viewing experience and interactive perception for user.Therefore it is produced for carrying the related of sensor
Product carry out accurate locomotion evaluation and model of place calculating is very important for application effect of products.Visual odometry
One of the nucleus module of (Visual Odometry) as SLAM can satisfy current VR, AR product for the demand of positioning.
But current visual odometry method convergence rate in estimation of Depth is slower, is often not possible to after having obtained bulk information
Accurate estimation of Depth is obtained, causes final positioning result anti-interference ability not strong, robustness is not high.
Summary of the invention
At least one to solve the above-mentioned problems, first aspect present invention is provided in a kind of vision based on depth filter
Journey meter map updating method, comprising:
Judge whether to need to add according to the depth information characteristic value of the fisrt feature of the current frame image of acquisition point crucial
Frame image,
If not needing to add, institute is updated according to the fisrt feature point of the current frame image using existing depth filter
State the seed of existing depth filter；
If desired it adds, then establishes and initialize new depth filter, it is special that second is extracted from the current frame image
Seed of the sign point as the new depth filter, determines key frame images and using the new depth filter according to institute
State the seed that key frame images update the new depth filter；
The probability distribution for calculating and updating the seed, when the seed estimation of Depth convergence when be added the map with
Update the map.
Further, the method also includes
It obtains in real time and stores frame image；
Multiple fisrt feature points of current frame image are extracted, the depth information of each fisrt feature point is calculated and determine all the
The depth information characteristic value of one characteristic point depth information.
Further, the depth information characteristic value of the fisrt feature point of the current frame image according to acquisition judges whether
Needing to add key frame images further comprises:
The depth information characteristic value is compared with the eigenvalue threshold pre-seted, if the depth information characteristic value
Greater than eigenvalue threshold, then need to add key frame images.
Further, described if desired to add, then new depth filter is established and initializes, from the current frame image
The middle seed for extracting second feature point as the new depth filter determines key frame images and uses the new depth
Filter further comprises according to the seed that the key frame images update the new depth filter:
New depth filter is established and initialized, second feature point is extracted from the current frame image as described new
Depth filter seed；
The frame image of preset quantity is selected as key frame images according to the time from the frame image of the storage；
The new depth is updated according to the key frame images of the preset quantity using the new depth filter to filter
The depth information of seed in wave device.
Further, the frame image of preset quantity is selected as pass according to the time in the frame image from the storage
Key frame image further comprises:
Key point is selected from the second feature point of the current frame image, multiple frame images are chosen according to the time and is passed through
The key point judges each frame image and the current frame image with the presence or absence of overlapping region respectively, and if it exists, then determines
The frame image is key frame images, otherwise gives up the frame image.
Further, if described do not need to add, using existing depth filter according to the of the current frame image
The seed that one characteristic point updates the existing depth filter further comprises:
The fisrt feature point of the current frame image is subjected to characteristic matching with point map corresponding in the map to obtain
Obtain the depth information of the point map；
The existing depth filter updates the existing depth filtering according to the fisrt feature point of the current frame image
Corresponding seed in device.
Further, the probability distribution calculated and update the seed, when the convergence of the estimation of Depth of the seed
The map, which is added, to update the map further comprises:
The probability distribution of the seed is updated using bayesian probability model；
The seed estimation of Depth convergence is then determined when the probability of the seed is greater than the probability threshold value pre-seted, then by it
The characteristic point of corresponding picture frame is converted to point map and is added to the map to update the map.
Second aspect of the present invention provides a kind of visual odometry map updating system based on depth filter, comprising:
Key frame extraction device, for judging whether system needs to add key frame images；
Key frame depth updating device is described new for establishing and initializing new depth filter and determine seed
Depth filter updates the new depth filter according to the key frame images with overlapped view that the system stores
Seed；
It is existing deep according to current frame image update to have depth filter for the system for depth filter updating device
Spend the seed of filter；
Probability distribution updating device, for calculating and updating the probability distribution of the seed, when the depth of the seed is estimated
It collects and the map is added when holding back to update the map.
Third aspect present invention provides a kind of computer readable storage medium, is stored thereon with computer program, the program
Method as described in relation to the first aspect is realized when being executed by processor.
Fourth aspect present invention provides a kind of computer equipment, including memory, processor and storage are on a memory simultaneously
The computer program that can be run on a processor, the processor realize side as described in relation to the first aspect when executing described program
Method.
Beneficial effects of the present invention are as follows:
The present invention formulates one for the problem that existing depth filter convergence time is slow, map rejuvenation is slow at present
Visual odometry map updating method and system of the kind based on depth filter, extract second feature while adding key frame
It puts and carries out depth update, to make full use of present frame and there is the overlapping frame of overlapped view with present frame, dramatically speed up depth
It spends the convergence rate of filter, improve the efficiency that depth updates, to compensate for the problem of existing in the prior art, effectively promote view
Feel the robustness of odometer map rejuvenation.
Detailed description of the invention
Specific embodiments of the present invention will be described in further detail with reference to the accompanying drawing.
Fig. 1 shows the flow chart of map updating method described in one embodiment of the present of invention；
Fig. 2 shows the flow charts for needing to add key frame described in one embodiment of the present of invention；
Fig. 3 shows the schematic diagram of key point described in one embodiment of the present of invention；
Fig. 4 shows the flow chart that addition key frame is not needed described in one embodiment of the present of invention；
Fig. 5 shows the structural schematic diagram of map updating system described in one embodiment of the present of invention；
Fig. 6 shows a kind of structural schematic diagram of computer equipment described in one embodiment of the present of invention.
Specific embodiment
In order to illustrate more clearly of the present invention, the present invention is done further below with reference to preferred embodiments and drawings
It is bright.Similar component is indicated in attached drawing with identical appended drawing reference.It will be appreciated by those skilled in the art that institute is specific below
The content of description is illustrative and be not restrictive, and should not be limited the scope of the invention with this.
In the prior art, during building map in real time, depth update method is generally passed through using depth filter
The image obtained to camera is handled and is updated map, is specifically updated using existing dichotomy method, crucial in addition
New characteristic point is extracted when frame as seed, carries out the operation of depth update until next frame normal frames arrive, however this mistake
Journey also includes the processing such as pose estimation and the optimization of camera, and time loss is excessive for depth update, is easy to cause
The case where tracking failure, it often cannot achieve the real-time building of map.
Based on the above situation, An embodiment provides a kind of visual odometry based on depth filter
Figure update method, comprising: needs are judged whether according to the depth information characteristic value of the fisrt feature of the current frame image of acquisition point
Key frame images are added, if not needing to add, using existing depth filter according to the fisrt feature of the current frame image
Point updates the seed of the existing depth filter；If desired it adds, then establishes and initialize new depth filter, from described
Seed of the second feature point as the new depth filter is extracted in current frame image, determines key frame images and uses institute
State the seed that new depth filter updates the new depth filter according to the key frame images；It calculates and updates described
The map is added when the convergence of the estimation of Depth of the seed to update the map in the probability distribution of seed.
In a specific example, as shown in Figure 1, acquisition is schemed in real time during actual motion using monocular cam
Picture simultaneously constructs map, and the monocular cam obtains according to the time interval pre-seted and store frame image, such as 1 second acquires
Image is handled and is stored sequentially in time by 30 frame images, the application to using monocular or more mesh cameras with no restrictions.It will
Time immediate current frame image is described in acquisition image:
Firstly, extracting the fisrt feature point of the current frame image, and the depth information of each characteristic point is calculated separately, determined
The depth information characteristic value of all fisrt feature point depth informations.Depth information is determined i.e. from the depth information of multiple characteristic points
Characteristic value, the characteristic value can be the statistical natures such as extreme value, depth information mean value and the depth information intermediate value of depth information
Value is used as judgment basis, and the present embodiment is using the depth information intermediate value of characteristic point as judgment basis.It is worth noting that, about feature
There are the gray differences of 16 pixels around various ways, such as one pixel of detection for the extraction of point to extract characteristic point, right
With no restrictions, those skilled in the art can be according to the requirement extract characteristic point of true resolution and sensitivity by this application.
Secondly, judging whether to need to add key frame images according to the depth information characteristic value.It specifically includes: will be described
Depth information characteristic value is compared with the eigenvalue threshold pre-seted, if the depth information characteristic value is greater than characteristic value threshold
Value, then need to add key frame images.The depth information intermediate value is carried out with the intermediate value threshold value pre-seted in the present embodiment
It compares, thinks to need to add key frame images if the depth information intermediate value is greater than intermediate value threshold value.The intermediate value threshold value is root
According to the determining threshold value of the acquisition of actual frame image, processing and analysis, such as fisrt feature point and the map according to present frame
In the corresponding picture frame of newest point map with the fisrt feature point at a distance from corresponding characteristic point, when in the distance
Value is more than that pre-determined distance then thinks that camera has moved farther out, then needs to select tool from the frame image stored before present frame
There is the frame image of overlapping region to be added as key frame images, it can be according to described in the case where only considering distance feature
The intermediate value threshold value is determined apart from intermediate value；The frame of storage can also be for example calculated again on the basis of being compared using distance again
The ratio of coordinate of the image under camera coordinates system and the depth intermediate value, if the ratio meets the ratio range pre-seted
Think not needing addition key frame images, key frame images is otherwise added, to further increase visual odometry map rejuvenation
Sensitivity.
If desired key frame images are added, then new depth filter are established and initialize, from the current frame image
Seed of the second feature point as the new depth filter is extracted, determine key frame images and is filtered using the new depth
Wave device updates the seed of the new depth filter according to the key frame images.As shown in Fig. 2, specifically including:
New depth filter is established and initialized, second feature point is extracted from the current frame image as described new
Depth filter seed.A new depth filter is re-established, and the depth filter is initialized, except normal
It further include being arranged in the depth filter for calculating the parameter of probability distribution other than rule initialization；Then again to current
Frame image carry out the extraction of second feature point, the second feature point be different from fisrt feature point, using second feature point as newly
The seed of depth filter, i.e., each second feature point correspond to a seed of new depth filter, and each seed storage is still
The character pair point of depth information is not obtained for subsequent carry out estimation of Depth.It is worth noting that the application is to the ginseng
Several settings calculate probability distribution as design criteria with no restrictions, can be realized.The design parameter ginseng being arranged in the present embodiment
See and " is proposed in Vogiatzis G, Hernandez C.Video-based, real-time multi-view stereo [J] "
Bayesian probability model.
The frame image of preset quantity is selected as key frame images according to the time from the frame image of the storage.Due to
Present frame with point map newest in the map at a distance from corresponding picture frame farther out, i.e., camera moved farther out, because
This needs chooses key frame images from the frame image stored before the present frame, in the present embodiment from the frame image of storage
The time immediate frame image for choosing preset quantity is selected, i.e., choosing from the frame image of storage has weight with present frame
The frame image in folded region (i.e. overlapped view).It specifically includes and selects key point from the characteristic point of the current frame image, lead to
It crosses the key point and judges that the frame image and the current frame image with the presence or absence of overlapping region, then determine the frame figure if it exists
As being key frame images, otherwise give up the frame image.Wherein, the preset quantity can be a frame image, or more
A frame image, the application are without limitation.
As shown in figure 3,5 points are chosen in the present embodiment from the characteristic point that present frame extracts as key point, 5 passes
Key o'clock is made of 1 central point and 4 boundary points, and central point is the point nearest with image central distance in all characteristic points, and 4
Boundary point is respectively point nearest with four angular distance of image in all characteristic points.It can be from storage according to above-mentioned 5 key points
The frame image with current frame image there are overlapping region is picked out in frame image, and the frame image is determined as key frame images.
Since the key frame images and current frame image are apart from close, then the seed maximum probability in new depth filter is selected
Key frame images observed, therefore with these key frame images more new seed, to make full use of current frame image and add
The convergence rate of the estimation of Depth of seed in fast new depth filter, to improve the robust of visual odometry map rejuvenation
Property.It is worth noting that the application to the selection of the key point with no restrictions, those skilled in the art should be according to practical need
Seek selection key point.
The new depth is updated according to the key frame images of the preset quantity using the new depth filter to filter
The depth information of seed in wave device.The i.e. described new depth filter is directly updated described in acquisition stored seed
The depth information of seed does not need to wait a normal frames image arrival just seed according to the normal frames image update again
Depth information, thus effectively reduce seed update consumed by the time.
If not needing addition key frame images, using existing depth filter according to the first spy of the current frame image
Sign point updates the seed of the existing depth filter.When not needing addition key frame images, the current frame image is as general
Logical frame image, is updated the seed stored in existing depth filter.As shown in figure 4, specifically including:
First, the fisrt feature point of the present frame is subjected to characteristic matching with point map corresponding in the map to obtain
Obtain the depth information of the point map.Specifically include: the existing depth filter is special according to the first of the current frame image
Sign point updates the depth information for being stored in seed in the existing depth filter, i.e., by point map corresponding to each seed
On projection to present frame, the point after projection is located at camera rear or skips these seeds if not projecting on present frame, because
The depth information of these seeds can not be updated and be had an impact for present frame.Calculate the corresponding picture frame of each seed and present frame
Between affine transformation matrix, that is, be stored in each seed in the depth filter come it is affine between source frame and present frame
Transformation matrix.The determinant for calculating affine transformation matrix, being found in image pyramid by the value of the determinant has most
The horizontal pyramidal layer of good matching.
Second, the existing depth filter updates the existing depth according to the fisrt feature point of the current frame image
Corresponding seed in filter.The i.e. described existing depth filter is believed using the depth that present frame assesses the stored seed
Breath, the existing depth filter use bilinear interpolation, the affine transformation matrix and pyramidal layer by current frame image
Image block, which transforms to, obtains transformed image block to obtain polar curve in reference frame image, the interval sampling on grade line, and calculates
Difference and Jacobian matrix between the image block of present frame after the corresponding image block of sampled result and variation, thus accurately
The feature locations of prediction and matching obtain accurate estimation of Depth value using triangulation.Wherein, the reference frame image is
The previous normal frames image of the current frame image；Described image block is characterized a pixel for surrounding preset quantity, described pre-
If quantity is configured according to practical application request.Then the depth for calculating each seed using trigonometric sine, the cosine law is not true
It is qualitative, method used in the present embodiment referring specifically to " Pizzoli M, Forster C, Scaramuzza D.REMODE:
Probablistic, Monocular dense reconstruction in real time [C] ", details are not described herein.
Finally, the probability distribution of the seed is calculated and updates, when the convergence of the estimation of Depth of the seed described in addition
Map is to update the map.New depth filter and new at this is either established when needing to add key frame images
Increase seed in depth filter and utilize key frame images more new seed, or present frame is used to deposit as normal frames update
The seed in existing depth filter is stored up, updates the probability point of each seed using bayesian probability model after seed update
Cloth.It specifically includes: updating the probability distribution of each seed using bayesian probability model；When the probability of each seed is greater than
The probability threshold value pre-seted then determines the seed estimation of Depth convergence, then is converted to ground for the characteristic point of its corresponding picture frame
Figure point is added to the map to update the map.I.e. when the seed estimation of Depth is restrained by its corresponding world coordinates
The update of completing map is added in map as point map.Bayesian probability model used in the present embodiment referring specifically to
" Vogiatzis G, Hernandez C.Video-based, real-time multi-view stereo [J] ", specific table
It is now that a series of histogram distribution of measurements of the depth of each seed using Gaussian Profile and is uniformly distributed and carrys out association list
Show, the solution that the solution of the depth of seed can be converted to Gaussian Profile and Beta distribution by introducing latent variable, i.e., by repeatedly
For formula after new observed quantity is added, the Posterior probability distribution of more new seed, when the seed probability in Posterior probability distribution is big
When the probability threshold value pre-seted, then it is assumed that the convergence of seed estimation of Depth, so that its corresponding point map is added in map, with
Realize the update of cartographic information.
The present embodiment directly carries out the depth that assessment obtains seed to stored seed using the new depth filter
Degree estimation does not need that a normal frames image is waited to arrive just according to normal frames image progress pose estimation again, effectively subtracts
Lack the time consumed by seed estimation of Depth, to accelerate the convergence rate of seed estimation of Depth, improves the effect that depth updates
Rate effectively promotes the robustness of visual odometry map rejuvenation to compensate for problems of the prior art.
Corresponding with map updating method provided by the above embodiment, one embodiment of the application also provides one kind and is based on
The visual odometry map updating system of depth filter, due to map updating system provided by the embodiments of the present application with it is above-mentioned several
The map updating method that kind of embodiment provides is corresponding, therefore is also applied for that the present embodiment provides maps more in aforementioned embodiments
New system is not described in detail in the present embodiment.
As shown in figure 5, one embodiment of the application also provides a kind of visual odometry map based on depth filter
More new system, comprising: key frame extraction device, for judging whether system needs to add key frame images；Key frame depth is more
New equipment, for establishing and initializing new depth filter and determine seed, the new depth filter is according to the system
The key frame images with overlapped view of system storage update the seed of the new depth filter；Depth filter more new clothes
It sets, has depth filter for the system and update the seed for having depth filter according to current frame image；Probability distribution
Updating device, for calculating and updating the probability distribution of the seed, when the convergence of the estimation of Depth of the seed described in addition
Map is to update the map.
In a specific example, key frame extraction device is according to the multiple characteristic points extracted from current frame image
Depth information characteristic value judges whether to need to add key frame images, that is, judges whether the camera has moved farther out, be
No needs select key frame images from the frame image stored before current frame image；If desired it adds and then uses key frame depth
New depth filter is established and initialized to updating device, using the characteristic point extracted in current frame image as new depth filtering
The seed of device, and the more new seed of the key frame images with overlapped view stored using system；If not needing addition key frame
Depth filter updating device is then used, using the existing depth filter of the system according to the feature of the current frame image
Point updates the seed of the existing depth filter storage；Finally, the seed that updates of either new depth filter is still
The seed for having depth filter to update, the probability distribution of the seed is calculated using probability distribution updating device, when the seed
Probability distribution be greater than probability threshold value when think seed estimation of Depth restrain, then by the seed be added map, to realize ground
The update of figure.
The system also includes feature point extraction devices and point map updating device, and wherein feature point extraction device, is used for
Extract multiple characteristic points of current frame image；Point map updating device, when do not need addition key frame when, depth filter more
Before the seed that new equipment is updated storage using existing depth filter according to current frame image, for estimating the map roughly
The depth information of point map, i.e., when the described current frame image is normal frames image, characteristic point that will be extracted from the present frame
Characteristic matching is carried out with point map corresponding in the map to obtain the depth information of the point map.
The map updating system updates depth filter by selecting key frame images from stored frame image
Seed to complete the estimation of Depth of seed using a small amount of existing information does not need that a normal frames image is waited to arrive just again
Pose estimation is carried out according to the normal frames image, the time consumed by seed estimation of Depth is effectively reduced, to accelerate to plant
The convergence rate of sub- estimation of Depth improves the efficiency that depth updates, to compensate for problems of the prior art, have compared with
Strong anti-interference ability, significantly improve system operation robustness, can be widely applied for VR, AR equipment pose calculate,
Figure building and scene interactivity.
Another embodiment of the present invention provides a kind of computer readable storage mediums, are stored thereon with computer journey
Sequence, realization when which is executed by processor: according to the depth information characteristic value of the fisrt feature of the current frame image of acquisition point
Judge whether to need to add key frame images, if not needing to add, using existing depth filter according to the present frame figure
The fisrt feature point of picture updates the seed of the existing depth filter；If desired it adds, then establish and initializes new depth
Filter extracts seed of the second feature point as the new depth filter from the current frame image, determines crucial
Frame image and the seed for updating the new depth filter according to the key frame images using the new depth filter；
The probability distribution for calculating and updating the seed, it is described to update when the convergence of the estimation of Depth of the seed map to be added
Map.
In practical applications, the computer readable storage medium can be using one or more computer-readable media
Any combination.Computer-readable medium can be computer-readable signal media or computer readable storage medium.It calculates
Machine readable storage medium storing program for executing can for example be but not limited to system, device or the device of electricity, magnetic, optical, electromagnetic, infrared ray or semiconductor
Part, or any above combination.The more specific example (non exhaustive list) of computer readable storage medium includes: to have
The electrical connection of one or more conducting wires, portable computer diskette, hard disk, random access memory (RAM), read-only memory
(ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, portable compact disc read-only memory (CD-
ROM), light storage device, magnetic memory device or above-mentioned any appropriate combination.It is computer-readable to deposit in this in real time example
Storage media can be any tangible medium for including or store program, which can be commanded execution system, device or device
Part use or in connection.
Computer-readable signal media may include in a base band or as carrier wave a part propagate data-signal,
Wherein carry computer-readable program code.The data-signal of this propagation can take various forms, including but unlimited
In electromagnetic signal, optical signal or above-mentioned any appropriate combination.Computer-readable signal media can also be that computer can
Any computer-readable medium other than storage medium is read, which can send, propagates or transmit and be used for
By the use of instruction execution system, device or device or program in connection.
The program code for including on computer-readable medium can transmit with any suitable medium, including but not limited to without
Line, electric wire, optical cable, RF etc. or above-mentioned any appropriate combination.
The computer for executing operation of the present invention can be write with one or more programming languages or combinations thereof
Program code, described program design language include object oriented program language-such as Java, Smalltalk, C++,
It further include conventional procedural programming language-such as " C " language or similar programming language.Program code can be with
It fully executes, partly execute on the user computer on the user computer, being executed as an independent software package, portion
Divide and partially executes or executed on a remote computer or server completely on the remote computer on the user computer.?
Be related in the situation of remote computer, remote computer can pass through the network of any kind --- including local area network (LAN) or
Wide area network (WAN)-be connected to subscriber computer, or, it may be connected to outer computer (such as mentioned using Internet service
It is connected for quotient by internet).
As shown in fig. 6, another embodiment of the present invention provides a kind of computer equipment structural schematic diagram.Fig. 6 is aobvious
The computer equipment 12 shown is only an example, should not function to the embodiment of the present invention and use scope bring any limit
System.
As shown in fig. 6, computer equipment 12 is showed in the form of universal computing device.The component of computer equipment 12 can be with
Including but not limited to: one or more processor or processing unit 16, system storage 28 connect different system components
The bus 18 of (including system storage 28 and processing unit 16).
Bus 18 indicates one of a few class bus structures or a variety of, including memory bus or Memory Controller,
Peripheral bus, graphics acceleration port, processor or the local bus using any bus structures in a variety of bus structures.It lifts
For example, these architectures include but is not limited to industry standard architecture (ISA) bus, microchannel architecture (MAC)
Bus, enhanced isa bus, Video Electronics Standards Association (VESA) local bus and peripheral component interconnection (PCI) bus.
Computer equipment 12 typically comprises a variety of computer system readable media.These media can be it is any can be by
The usable medium that computer equipment 12 accesses, including volatile and non-volatile media, moveable and immovable medium.
System storage 28 may include the computer system readable media of form of volatile memory, such as arbitrary access
Memory (RAM) 30 and/or cache memory 32.Computer equipment 12 may further include it is other it is removable/can not
Mobile, volatile/non-volatile computer system storage medium.Only as an example, storage system 34 can be used for reading and writing not
Movably, non-volatile magnetic media (Fig. 6 do not show, commonly referred to as " hard disk drive ").It, can be with although being not shown in Fig. 6
The disc driver for reading and writing to removable non-volatile magnetic disk (such as " floppy disk ") is provided, and non-volatile to moving
The CD drive of CD (such as CD-ROM, DVD-ROM or other optical mediums) read-write.In these cases, each driving
Device can be connected by one or more data media interfaces with bus 18.Memory 28 may include that at least one program produces
Product, the program product have one group of (for example, at least one) program module, these program modules are configured to perform of the invention each
The function of embodiment.
Program/utility 40 with one group of (at least one) program module 42 can store in such as memory 28
In, such program module 42 include but is not limited to operating system, one or more application program, other program modules and
It may include the realization of network environment in program data, each of these examples or certain combination.Program module 42 is usual
Execute the function and/or method in embodiment described in the invention.
Computer equipment 12 can also be with one or more external equipments 14 (such as keyboard, sensing equipment, display 24
Deng) communication, can also be enabled a user to one or more equipment interact with the computer equipment 12 communicate, and/or with make
The computer equipment 12 any equipment (such as network interface card, the modulatedemodulate that can be communicated with one or more of the other calculating equipment
Adjust device etc.) communication.This communication can be carried out by input/output (I/O) interface 22.Also, computer equipment 12 may be used also
To pass through network adapter 20 and one or more network (such as local area network (LAN), wide area network (WAN) and/or public network
Network, such as internet) communication.As shown in fig. 6, network adapter 20 is logical by other modules of bus 18 and computer equipment 12
Letter.It should be understood that other hardware and/or software module, packet can be used in conjunction with computer equipment 12 although being not shown in Fig. 6
It includes but is not limited to: microcode, device driver, redundant processing unit, external disk drive array, RAID system, magnetic tape drive
Device and data backup storage system etc..
Processor unit 16 by the program that is stored in system storage 28 of operation, thereby executing various function application with
And data processing, such as realize a kind of visual odometry map rejuvenation based on depth filter provided by the embodiment of the present invention
Method.
The present invention formulates one for the problem that existing depth filter convergence time is slow, map rejuvenation is slow at present
Visual odometry map updating method and system of the kind based on depth filter, extract second feature while adding key frame
It puts and carries out depth update, to make full use of present frame and there is the overlapping frame of overlapped view with present frame, dramatically speed up depth
It spends the convergence rate of filter, improve the efficiency that depth updates, to compensate for the problem of existing in the prior art, effectively promote view
Feel the robustness of odometer map rejuvenation.
Obviously, the above embodiment of the present invention be only to clearly illustrate example of the present invention, and not be pair
The restriction of embodiments of the present invention may be used also on the basis of the above description for those of ordinary skill in the art
To make other variations or changes in different ways, all embodiments can not be exhaustive here, it is all to belong to this hair
The obvious changes or variations that bright technical solution is extended out are still in the scope of protection of the present invention.
Claims (10)
1. a kind of visual odometry map updating method based on depth filter characterized by comprising
Judge whether to need to add key frame figure according to the depth information characteristic value of the fisrt feature of the current frame image of acquisition point
Picture,
It is described according to the update of the fisrt feature point of the current frame image using existing depth filter if not needing to add
There is the seed of depth filter；
If desired it adds, then establishes and initialize new depth filter, second feature point is extracted from the current frame image
As the seed of the new depth filter, key frame images are determined and using the new depth filter according to the pass
The seed of new depth filter described in key frame image update；
The map is added when the convergence of the estimation of Depth of the seed to update in the probability distribution for calculating and updating the seed
The map.
2. the method according to claim 1, wherein the method also includes
It obtains in real time and stores frame image；
Multiple fisrt feature points of current frame image are extracted, the depth information of each fisrt feature point is calculated and determine that all first is special
The depth information characteristic value of sign point depth information.
3. method according to claim 1 or 2, which is characterized in that the first of the current frame image according to acquisition is special
The depth information characteristic value of sign point judges whether that needing to add key frame images further comprises:
The depth information characteristic value is compared with the eigenvalue threshold pre-seted, if the depth information characteristic value is greater than
Eigenvalue threshold then needs to add key frame images.
4. according to the method described in claim 2, then establishing and initializing new depth it is characterized in that, if desired described add
Filter is spent, seed of the second feature point as the new depth filter is extracted from the current frame image, determines and closes
Key frame image and the kind for updating the new depth filter according to the key frame images using the new depth filter
Son further comprises:
New depth filter is established and initialized, second feature point is extracted from the current frame image as the new depth
Spend the seed of filter；
The frame image of preset quantity is selected as key frame images according to the time from the frame image of the storage；
The new depth filter is updated according to the key frame images of the preset quantity using the new depth filter
The depth information of middle seed.
5. according to the method described in claim 4, it is characterized in that, being chosen in the frame image from the storage according to the time
The frame image of preset quantity further comprises as key frame images out:
Key point is selected from the second feature point of the current frame image, multiple frame images are chosen according to the time and by described
Key point judges each frame image and the current frame image with the presence or absence of overlapping region respectively, and if it exists, then determines the frame
Image is key frame images, otherwise gives up the frame image.
6. the method according to claim 1, wherein if described do not need to add, use has depth filtering
Device further comprises according to the seed that the fisrt feature point of the current frame image updates the existing depth filter:
The fisrt feature point of the current frame image is subjected to characteristic matching with point map corresponding in the map to obtain
State the depth information of point map；
The existing depth filter updates in the existing depth filter according to the fisrt feature point of the current frame image
Corresponding seed.
7. the method according to claim 1, wherein the calculating and update the probability distribution of the seed, when
The map, which is added, to update the map when estimation of Depth convergence of the seed further comprises:
The probability distribution of the seed is updated using bayesian probability model；
The seed estimation of Depth convergence is then determined when the probability of the seed is greater than the probability threshold value pre-seted, then is corresponded to
The characteristic point of picture frame be converted to point map and be added to the map to update the map.
8. a kind of visual odometry map updating system based on depth filter characterized by comprising
Key frame extraction device, for judging whether system needs to add key frame images；
Key frame depth updating device, for establishing and initializing new depth filter and determine seed, the new depth
Filter updates the seed of the new depth filter according to the key frame images with overlapped view that the system stores；
Depth filter updating device has depth filter for the system and updates existing depth filter according to current frame image
The seed of wave device；
Probability distribution updating device, for calculating and updating the probability distribution of the seed, when the estimation of Depth of the seed is received
The map is added when holding back to update the map.
9. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the program is held by processor
Such as method of any of claims 1-7 is realized when row.
10. a kind of computer equipment including memory, processor and stores the meter that can be run on a memory and on a processor
Calculation machine program, which is characterized in that the processor realizes the side as described in any in claim 1-7 when executing described program
Method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910083820.4A CN109816726B (en) | 2019-01-29 | 2019-01-29 | Visual odometer map updating method and system based on depth filter |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910083820.4A CN109816726B (en) | 2019-01-29 | 2019-01-29 | Visual odometer map updating method and system based on depth filter |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109816726A true CN109816726A (en) | 2019-05-28 |
| CN109816726B CN109816726B (en) | 2021-10-01 |
Family
ID=66605525
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910083820.4A Active CN109816726B (en) | 2019-01-29 | 2019-01-29 | Visual odometer map updating method and system based on depth filter |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109816726B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110309330A (en) * | 2019-07-01 | 2019-10-08 | 北京百度网讯科技有限公司 | The treating method and apparatus of vision map |
| CN111288989A (en) * | 2020-02-25 | 2020-06-16 | 浙江大学 | Visual positioning method for small unmanned aerial vehicle |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107687850A (en) * | 2017-07-26 | 2018-02-13 | 哈尔滨工业大学深圳研究生院 | A kind of unmanned vehicle position and orientation estimation method of view-based access control model and Inertial Measurement Unit |
| CN108151728A (en) * | 2017-12-06 | 2018-06-12 | 华南理工大学 | A kind of half dense cognitive map creation method for binocular SLAM |
| CN108615246A (en) * | 2018-04-19 | 2018-10-02 | 浙江大承机器人科技有限公司 | It improves visual odometry system robustness and reduces the method that algorithm calculates consumption |
| CN108986037A (en) * | 2018-05-25 | 2018-12-11 | 重庆大学 | Monocular vision odometer localization method and positioning system based on semi-direct method |
-
2019
- 2019-01-29 CN CN201910083820.4A patent/CN109816726B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN107687850A (en) * | 2017-07-26 | 2018-02-13 | 哈尔滨工业大学深圳研究生院 | A kind of unmanned vehicle position and orientation estimation method of view-based access control model and Inertial Measurement Unit |
| CN108151728A (en) * | 2017-12-06 | 2018-06-12 | 华南理工大学 | A kind of half dense cognitive map creation method for binocular SLAM |
| CN108615246A (en) * | 2018-04-19 | 2018-10-02 | 浙江大承机器人科技有限公司 | It improves visual odometry system robustness and reduces the method that algorithm calculates consumption |
| CN108986037A (en) * | 2018-05-25 | 2018-12-11 | 重庆大学 | Monocular vision odometer localization method and positioning system based on semi-direct method |
Non-Patent Citations (1)
| Title |
|---|
| CHRISTIAN FORSTER ET AL.: "SVO: Fast Semi-Direct Monocular Visual Odometry", 《2014 IEEE INTERNATIONAL CONFERENCE ON ROBOTICS & AUTOMATION》 * |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110309330A (en) * | 2019-07-01 | 2019-10-08 | 北京百度网讯科技有限公司 | The treating method and apparatus of vision map |
| CN110309330B (en) * | 2019-07-01 | 2021-08-31 | 北京百度网讯科技有限公司 | Processing method and device of visual map |
| CN111288989A (en) * | 2020-02-25 | 2020-06-16 | 浙江大学 | Visual positioning method for small unmanned aerial vehicle |
| CN111288989B (en) * | 2020-02-25 | 2021-11-05 | 浙江大学 | Visual positioning method for small unmanned aerial vehicle |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109816726B (en) | 2021-10-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11394950B2 (en) | Augmented reality-based remote guidance method and apparatus, terminal, and storage medium | |
| CN109087359B (en) | Pose determination method, pose determination apparatus, medium, and computing device | |
| CN109003325B (en) | Three-dimensional reconstruction method, medium, device and computing equipment | |
| CN108447097A (en) | Depth camera scaling method, device, electronic equipment and storage medium | |
| CN109887003B (en) | Method and equipment for carrying out three-dimensional tracking initialization | |
| WO2019161813A1 (en) | Dynamic scene three-dimensional reconstruction method, apparatus and system, server, and medium | |
| CN109242913A (en) | Scaling method, device, equipment and the medium of collector relative parameter | |
| CN108895981B (en) | Three-dimensional measurement method, device, server and storage medium | |
| US20200134903A1 (en) | Smooth Draping Layer for Rendering Vector Data on Complex Three Dimensional Objects | |
| CN108898676B (en) | Method and system for detecting collision and shielding between virtual and real objects | |
| EP2915140B1 (en) | Fast initialization for monocular visual slam | |
| CN110246147A (en) | Vision inertia odometer method, vision inertia mileage counter device and mobile device | |
| CN109084746A (en) | Monocular mode for the autonomous platform guidance system with aiding sensors | |
| SG189284A1 (en) | Rapid 3d modeling | |
| CN109191554B (en) | Super-resolution image reconstruction method, device, terminal and storage medium | |
| TW201308252A (en) | Depth measurement quality enhancement | |
| CN115578433B (en) | Image processing method, device, electronic equipment and storage medium | |
| CN114519772A (en) | Three-dimensional reconstruction method and system based on sparse point cloud and cost aggregation | |
| CN109816726A (en) | A kind of visual odometry map updating method and system based on depth filter | |
| CN115578432B (en) | Image processing method, device, electronic equipment and storage medium | |
| CN115619986B (en) | Scene roaming method, device, equipment and medium | |
| CN116843754A (en) | Visual positioning method and system based on multi-feature fusion | |
| KR20210050997A (en) | Method and apparatus for estimating pose, computer-readable storage medium and computer program for controlling the holder device | |
| CN114266876A (en) | Positioning method, visual map generation method and device | |
| CN110211239B (en) | Augmented reality method, apparatus, device and medium based on label-free recognition |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |