CN109191526A - Three-dimensional environment method for reconstructing and system based on RGBD camera and optical encoder - Google Patents
Three-dimensional environment method for reconstructing and system based on RGBD camera and optical encoder Download PDFInfo
- Publication number
- CN109191526A CN109191526A CN201811052260.8A CN201811052260A CN109191526A CN 109191526 A CN109191526 A CN 109191526A CN 201811052260 A CN201811052260 A CN 201811052260A CN 109191526 A CN109191526 A CN 109191526A
- Authority
- CN
- China
- Prior art keywords
- frame
- pose
- optical encoder
- key frame
- present frame
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
- G06T7/74—Determining position or orientation of objects or cameras using feature-based methods involving reference images or patches
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Computer Graphics (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Image Analysis (AREA)
- Studio Devices (AREA)
Abstract
The invention discloses a kind of three-dimensional environment method for reconstructing and system based on RGBD camera and optical encoder, when choosing key frame, if former frame cannot function as key frame, the interframe pose change information between present frame and former frame then is calculated using the pose of the optical encoder of the pose and present frame of the optical encoder of former frame, obtains the pose estimation of present frame；In reconstruction process, when the present frame that view-based access control model method is estimated is more than the first given threshold relative to the pose and its error of nearest key frame, using the output pose of optical encoder as the initial pose of camera, pose and its error of the present frame relative to nearest key frame are reevaluated.It is by estimating camera pose using vision and optical encoder, it is possible to reduce ambient lighting changes the influence estimated pose；When key frame initializes failure, the estimation of pose variation is provided using optical encoder, the robustness to the environment rebuilt for lacking texture can be enhanced, can be more widely used.
Description
Technical field
The present invention relates to technical field of image processing more particularly to a kind of three-dimensionals based on RGBD camera and optical encoder
Environment rebuilt method and system.
Background technique
In computer vision, three-dimensional reconstruction refers to the mistake of the image reconstruction three-dimensional information according to single-view or multiple view
Journey.Due to the INFORMATION OF INCOMPLETE of single video, therefore three-dimensional reconstruction is relatively difficult.And (similar people's is double for the three-dimensional reconstruction of multiple view
Mesh positioning) relatively it is easy, substantially method is first to demarcate to video camera, that is, calculates the image coordinate system of video camera
With the relationship of world coordinate system, the information reconstruction in multiple two dimensional images is then recycled to go out three-dimensional information.
Currently, it is based purely on the three-dimensional environment method for reconstructing of RGBD (RGB+Depth, cromogram+depth map) camera,
Input picture is color image and depth image.Wherein, color image is easy to become by ambient lightings such as bright, excessively dark, overexposures
The influence of change, so as to cause image deterioration or even unavailable.In addition, in the environment for lacking texture, it can not from color image
Extract the edge and characteristic point of object.And the edge and characteristic point of object are extremely important to the pose estimation of camera, if can not
The edge and characteristic point of object are extracted, then the pose evaluated error that can directly result in camera is excessive, and then influences to rebuild effect.
Therefore, the existing three-dimensional environment method for reconstructing based on RGBD camera is easy by environment and its illumination variation
It influences, it is lower to the robustness of environment and its illumination variation in reconstruction process, using limited.
Summary of the invention
Based on this, rebuild the purpose of the present invention is to provide a kind of based on the three-dimensional environment of RGBD camera and optical encoder
Method and system cause color image to degrade or not can be used as key frame in use, benefit in ambient lighting or environment texture variations
The pose estimation that camera is compensated with the output of optical encoder, to increase reconstruction process to the Shandong of environment and its illumination variation
Stick can be more widely used.
A kind of three-dimensional environment method for reconstructing based on RGBD camera and optical encoder provided by the invention, including following step
It is rapid:
When choosing key frame, if former frame cannot function as key frame, using former frame optical encoder pose and
The pose of the optical encoder of present frame calculates the interframe pose change information between present frame and former frame, obtains the position of present frame
Appearance estimation；
In reconstruction process, when the present frame that view-based access control model method is estimated relative to the pose of nearest key frame and its
When error is more than the first given threshold, using the output pose of optical encoder as the initial pose of camera, present frame is reevaluated
Pose and its error relative to nearest key frame.
As an embodiment, according to the edge of image and characteristic point, key frame is chosen.
As an embodiment, when choosing key frame, if former frame cannot function as key frame, former frame is utilized
The pose of optical encoder and the pose of optical encoder of present frame calculate the interframe pose variation between present frame and former frame
Information obtains the pose estimation of present frame, comprising the following steps:
When choosing key frame, if former frame cannot function as key frame, the pose using the optical encoder of former frame is believed
The posture information of the optical encoder of breath and present frame, calculates the interframe pose change information between present frame and former frame；
According to the pose of the former frame and the interframe pose change information, the pose estimation of present frame is obtained.
As an embodiment, the three-dimensional environment reconstruction side of the invention based on RGBD camera and optical encoder
Method, further comprising the steps of:
Initialize key frame；
If initializing successfully, according to present frame and nearest key frame, the pose estimation of present frame is obtained；
If initialization failure, judges whether former frame can be used as key frame；
If former frame can be used as key frame, former frame is saved as to new key frame, and former frame pose is initial
Unit matrix is turned to, label initializes successfully.
As an embodiment, according to present frame and nearest key frame, the pose for obtaining present frame estimates it
Afterwards, further comprising the steps of:
Calculate pose and error of the present frame relative to nearest key frame；
If error is less than the second given threshold, present frame is saved as into former frame；
If error is greater than the second given threshold, initialization is marked to fail.
As an embodiment, in reconstruction process, when the present frame that view-based access control model method is estimated relative to
When the pose and its error of nearest key frame are more than the first given threshold, using the output pose of optical encoder as the initial of camera
Pose reevaluates pose and its error of the present frame relative to nearest key frame, comprising the following steps:
The pose and its error that obtain present frame relative to nearest key frame are estimated based on general visible sensation method；
If the error is greater than the first given threshold, according to the output pose and present frame pumped FIR laser of former frame optical encoder
The output pose of device, obtains interframe pose；
According to the pose of interframe pose and former frame, the initial estimation of the pose of present frame is obtained；
According to the initial estimation of nearest key frame and the pose of present frame, the pose of present frame is obtained.
Correspondingly, a kind of three-dimensional environment reconstructing system based on RGBD camera and optical encoder provided by the invention, packet
It includes and chooses module and reconstruction module；
The selection module, for if former frame cannot function as key frame, utilizing former frame when choosing key frame
The pose of the optical encoder of the posture information and present frame of optical encoder calculates the interframe pose between present frame and former frame and becomes
Change, obtains the pose estimation of present frame；
The reconstruction module is used in reconstruction process, when the present frame that view-based access control model method is estimated is relative to most
When the pose and its error of nearly key frame are more than the first given threshold, using the output pose of optical encoder as the initial bit of camera
Appearance reevaluates pose and its error of the present frame relative to nearest key frame.
As an embodiment, according to the edge of image and characteristic point, key frame is chosen.
As an embodiment, the selection module includes the first computing unit and the second computing unit；
First computing unit, for if former frame cannot function as key frame, utilizing previous when choosing key frame
The posture information of the optical encoder of the posture information and present frame of the optical encoder of frame calculates the frame between present frame and former frame
Between pose change information；
Second computing unit is obtained for the pose and the interframe pose change information according to the former frame
The pose of present frame is estimated.
Three-dimensional environment reconstructing system provided by the invention based on RGBD camera and optical encoder further includes initialization mould
Block；
The initialization module, for initializing key frame；If initializing successfully, according to present frame and nearest key
Frame obtains the pose estimation of present frame；If initialization failure, judges whether former frame can be used as key frame, if former frame
It can be used as key frame, then former frame saved as to new key frame, and former frame pose is initialized as unit matrix, label is just
Begin chemical conversion function.
As an embodiment, the selection module further includes third computing unit；
The third computing unit is for calculating pose and error of the present frame relative to nearest key frame；If error is less than
When the second given threshold, present frame is saved as into former frame；If error is greater than the second given threshold, initialization is marked to lose
It loses.
As an embodiment, the reconstruction module includes pose estimation unit, comparing unit, the 4th computing unit
With the 5th computing unit；
The pose estimation unit, for estimating to obtain present frame relative to nearest key frame based on general visible sensation method
Pose and its error；
The comparing unit, if being greater than the first given threshold for the error, according to the output of former frame optical encoder
The output pose of pose and present frame optical encoder, obtains interframe pose；
4th computing unit obtains the first of the pose of present frame for the pose according to interframe pose and former frame
Begin to estimate；
5th computing unit obtains current for the initial estimation according to nearest key frame and the pose of present frame
The pose of frame.
Compared with prior art, the technical program has the advantage that
Three-dimensional environment method for reconstructing and system provided by the invention based on RGBD camera and optical encoder is closed choosing
When key frame, if former frame cannot function as key frame, the pose of the optical encoder of former frame and the optical encoder of present frame are utilized
Pose calculate the interframe pose change information between present frame and former frame, obtain the pose estimation of present frame；It was rebuilding
Cheng Zhong, when the present frame that view-based access control model method is estimated is more than the first setting relative to the pose and its error of nearest key frame
When threshold value, using the output pose of optical encoder as the initial pose of camera, present frame is reevaluated relative to nearest key frame
Pose and its error.It is by estimating camera pose using vision and optical encoder, it is possible to reduce ambient lighting variation contraposition
The influence of appearance estimation；When key frame initializes failure, the estimation of pose variation is provided using optical encoder, can be enhanced to scarce
The robustness of the environment rebuilt of few texture, can be more widely used.
Detailed description of the invention
Fig. 1 is in the three-dimensional environment method for reconstructing based on RGBD camera and optical encoder that the embodiment of the present invention one provides
Previous key frame, nearest key frame, the position view of former frame and present frame on a timeline；
Fig. 2 is the three-dimensional environment method for reconstructing based on RGBD camera and optical encoder that the embodiment of the present invention one provides
Flow diagram；
Fig. 3 is in the three-dimensional environment method for reconstructing based on RGBD camera and optical encoder that the embodiment of the present invention one provides
Key frame initialization process schematic diagram；
Fig. 4 is in the three-dimensional environment method for reconstructing based on RGBD camera and optical encoder that the embodiment of the present invention one provides
Reconstruction flow diagram；
Fig. 5 is in the three-dimensional environment method for reconstructing based on RGBD camera and optical encoder that the embodiment of the present invention one provides
The current pose of estimation flow diagram；
Fig. 6 is the three-dimensional environment reconstructing system provided by Embodiment 2 of the present invention based on RGBD camera and optical encoder
Structural schematic diagram.
Specific embodiment
Below in conjunction with attached drawing, the technical characteristic and advantage above-mentioned and other to the present invention are clearly and completely described,
Obviously, described embodiment is only section Example of the invention, rather than whole embodiments.
Each of present invention normal frames include following information: the cromogram and depth map, pumped FIR laser of camera output
The pose of device output and the pose of camera.Previous key frame, nearest key frame, the sequence of former frame and present frame on a timeline
Position, as shown in Figure 1.The letter of present frame is generally saved between two neighboring key frame in several normal frames, former frame
Breath.
Referring to fig. 2, a kind of three-dimensional environment weight based on RGBD camera and optical encoder that the embodiment of the present invention one provides
Construction method, comprising the following steps:
S100, when choosing key frame, if former frame cannot function as key frame, using former frame optical encoder position
The pose of the optical encoder of appearance and present frame calculates the interframe pose change information between present frame and former frame, obtains present frame
Pose estimation.
S200, in reconstruction process, when position of the present frame that view-based access control model method is estimated relative to nearest key frame
When appearance and its error are more than the first given threshold, using the output pose of optical encoder as the initial pose of camera, reevaluate
Pose and its error of the present frame relative to nearest key frame.
In step S100, generally in three-dimensional environment reconstruction process, what the first step was done is all initialization key frame.Key frame
It is to pick out from normal frames, formed according to the sequencing of time.The initialization of key frame, is exactly selected from normal frames
One frame is as first key frame.In order to estimate the pose of normal frames below, key frame is needed to have edge abundant enough or spy
Sign point.
The initialization of key frame in the present embodiment, as shown in figure 3, the specific implementation process is as follows:
S010 receives the normal frames newly to arrive, saves as present frame；
S011, judges whether present frame can be used as key frame；Judge present frame whether have edge abundant enough or
Characteristic point.
Present frame is saved as first key frame, label is crucial if present frame can be used as key frame by S012
Frame initializes successfully, and present frame is saved as former frame；It is then possible to start reconstruction process, S021 is entered step.
S013, if present frame cannot be used as key frame, receiving the normal frames newly to arrive is present frame, continues to sentence
Whether disconnected present frame can be used as key frame；Meanwhile if present frame cannot be used as key frame, key frame initialization is marked to lose
It loses, enters step S022.
S021, according to present frame and nearest key frame, obtains the pose estimation of present frame if initializing successfully；
S022, if initialization failure, judges whether former frame can be used as key frame；
Former frame is saved as new key frame if former frame can be used as key frame by S023, and by former frame pose
It is initialized as unit matrix, label initializes successfully.
Specifically, step S100 can be realized by following steps:
Firstly, if former frame cannot function as key frame, utilizing the position of the optical encoder of former frame when choosing key frame
The posture information of the optical encoder of appearance information and present frame calculates the interframe pose change information between present frame and former frame；
Then, according to the pose of former frame and interframe pose change information, the pose estimation of present frame is obtained.
Next, calculating pose and error of the present frame relative to nearest key frame；If error is less than the second given threshold
When, present frame is saved as into former frame；If error is greater than the second given threshold, initialization is marked to fail.
The reconstruction process that above-mentioned camera and optical encoder combine, by asking present frame relative to the position of nearest key frame
Appearance estimation and its error indicate that pose estimation is correctly, so present frame is protected when error is less than the second given threshold
Former frame is saved as, and receives new normal frames and saves as present frame；When error is greater than the second given threshold, the pose is indicated
Estimation is incorrect, so needing to generate new key frame.
Further, if former frame meets the condition of key frame, preservation former frame be new key frame, will before
The pose of one frame is initialized as the unit matrix of 4X4, and label key frame initializes successfully；Then, estimation present frame is relative to nearest
Key frame pose.
If former frame is unsatisfactory for the condition of key frame, compiled using the pose and present frame light of former frame optical encoder
The pose of code device, obtains the interframe pose change information between present frame and former frame, may further obtain the position of present frame
Appearance estimation；Since former frame fails to become key frame, so label key frame initialization is unsuccessful.
More than, when key frame initializes failure, the estimation of pose variation is provided using optical encoder, can be enhanced to scarce
The robustness of the environment rebuilt of few texture.
As an embodiment, the reconstruction process that above-mentioned camera and optical encoder combine, as shown in Figure 4:
S41 receives the normal frames newly to arrive, saves as present frame；
S42 estimates pose and its error of the present frame relative to nearest key frame after key frame initializes successfully；If
Error is greater than given threshold, then initialization is marked to fail, and executes step S43；If error is less than given threshold, indicate that the pose is estimated
Meter is accurately, to execute step S47；
S43 judges whether former frame can be used as key frame after key frame initialization failure；
Former frame is saved as new key frame if former frame can be used as key frame by S44, by previous framing bit
Appearance estimates that the unit matrix for being initialized as 4X4, label key frame initialize successfully；
S45 estimates the pose between present frame and nearest key frame, present frame is saved as former frame；
S46 utilizes the optical encoder posture information and present frame of former frame if former frame cannot be used as key frame
Optical encoder information obtains the pose between present frame and former frame, and then obtains the pose estimation of present frame, marks key frame
It initializes unsuccessful；Then, present frame is saved to former frame (S47), receives new normal frames to present frame (S41)；
S47 saves present frame to former frame, receives new normal frames and save as present frame (S41)；
Next, step S200 can be realized by following steps referring to Fig. 5:
S210, the pose and its error that obtain present frame relative to nearest key frame are estimated based on general visible sensation method；
If S220, the error are greater than the first given threshold, according to the output pose and present frame of former frame optical encoder
The output pose of optical encoder, obtains interframe pose；
S230, according to the pose of interframe pose and former frame, obtain the initial estimation of the pose of present frame；
S240, according to the initial estimation of nearest key frame and the pose of present frame, obtain the pose of present frame.
Wherein, the pose and its error based on general visible sensation method estimation present frame relative to nearest key frame, when accidentally
When difference is greater than the first given threshold, indicate that the position and attitude error of visible sensation method estimation is larger；So the pose exported with optical encoder
As initial value, the pose estimation and its error with visible sensation method estimation present frame relative to nearest key frame again.
If error is less than the first given threshold, indicate that pose estimated value at this time is more acurrate, estimates to replace with the pose
The evaluated error, is replaced original error, and export by present frame pose estimation originally.
A kind of three-dimensional environment method for reconstructing based on RGBD camera and optical encoder provided in an embodiment of the present invention, is being closed
After key frame initializes successfully, start reconstruction process；In the output of RGBD camera, pose is obtained with general visible sensation method and is estimated
Meter and its error；When error is excessive, uses the output of optical encoder as pose initial value, reevaluate pose and its error；
After error falls into setting error range, new pose is taken to be estimated as exporting.When generating key frame, if previous frame image is not
It can be used as key frame, then asking interframe pose to change with the output of the optical encoder of former frame and present frame, obtain present frame
Pose estimation.
In the present embodiment, by estimating camera pose using vision and optical encoder, it is possible to reduce ambient lighting variation pair
The influence of pose estimation；When key frame initializes failure, the estimation of pose variation is provided using optical encoder, can be enhanced pair
Lack the robustness of the environment rebuilt of texture.
Based on the same inventive concept, the embodiment of the present invention two additionally provides a kind of based on RGBD camera and optical encoder
Three-dimensional environment reconstructing system, the system is identical as the principle of preceding method, therefore the implementation of the system, can refer to the stream of preceding method
Cheng Shixian repeats place, no longer redundant later.
Referring to Fig. 6, a kind of three-dimensional environment weight based on RGBD camera and optical encoder provided by Embodiment 2 of the present invention
System is built, including chooses module 100 and rebuilds module 200.
Wherein, it chooses module 100 to be used for when choosing key frame, if former frame cannot function as key frame, utilize previous
The pose of the optical encoder of the posture information and present frame of the optical encoder of frame calculates the interframe position between present frame and former frame
Appearance variation obtains the pose estimation of present frame；
It rebuilds module 200 to be used in reconstruction process, when the present frame that view-based access control model method is estimated is relative to nearest
When the pose and its error of key frame are more than the first given threshold, using the output pose of optical encoder as the initial bit of camera
Appearance reevaluates pose and its error of the present frame relative to nearest key frame.
Three-dimensional environment reconstructing system provided in this embodiment based on RGBD camera and optical encoder, according to the side of image
Edge and characteristic point choose key frame.
Wherein, choosing module 100 includes the first computing unit and the second computing unit.First computing unit is for choosing
When key frame, if former frame cannot function as key frame, the posture information of the optical encoder of former frame and the light of present frame are utilized
The posture information of encoder calculates the interframe pose change information between present frame and former frame；Second computing unit is used for root
According to the pose and interframe pose change information of former frame, the pose estimation of present frame is obtained.
Choosing module further includes third computing unit；Third computing unit is for calculating present frame relative to nearest key frame
Pose and error；If error is less than the second given threshold, present frame is saved as into former frame；If error is greater than the second setting
When threshold value, then initialization is marked to fail.
Further, the three-dimensional environment reconstructing system provided in this embodiment based on RGBD camera and optical encoder, also
Including initialization module 010；
Initialization module 010 is for initializing key frame；If initializing successfully, according to present frame and nearest key
Frame obtains the pose estimation of present frame；If initialization failure, judges whether former frame can be used as key frame, if former frame
It can be used as key frame, then former frame saved as to new key frame, and former frame pose is initialized as unit matrix, label is just
Begin chemical conversion function.
Rebuilding module 200 includes pose estimation unit, comparing unit, the 4th computing unit and the 5th computing unit.Pose
Estimation unit is used to estimate the pose and its error that obtain present frame relative to nearest key frame based on general visible sensation method；Than
If being greater than the first given threshold for the error compared with unit, compiled according to the output pose of former frame optical encoder and present frame light
The output pose of code device, obtains interframe pose；4th computing unit is used for the pose according to interframe pose and former frame, is worked as
The initial estimation of the pose of previous frame；5th computing unit is used for the initial estimation according to nearest key frame and the pose of present frame,
Obtain the pose of present frame.
Three-dimensional environment method for reconstructing and system provided in this embodiment based on RGBD camera and optical encoder, is working as ring
Border illumination or environment texture variations cause color image to degrade or not can be used as key frame in use, using the defeated of optical encoder
The pose estimation for compensating camera out, increases reconstruction process to the robustness of environment and its illumination variation, makes the three-dimensional reconstruction
Method and system can be more widely used.
Although the invention has been described by way of example and in terms of the preferred embodiments, but it is not for limiting the present invention, any this field
Technical staff without departing from the spirit and scope of the present invention, may be by the methods and technical content of the disclosure above to this hair
Bright technical solution makes possible variation and modification, therefore, anything that does not depart from the technical scheme of the invention, and according to the present invention
Technical spirit any simple modifications, equivalents, and modifications to the above embodiments, belong to technical solution of the present invention
Protection scope.
Claims (12)
1. a kind of three-dimensional environment method for reconstructing based on RGBD camera and optical encoder, which comprises the following steps:
When choosing key frame, if former frame cannot function as key frame, the pose of the optical encoder of former frame and current is utilized
The pose of the optical encoder of frame calculates the interframe pose change information between present frame and former frame, and the pose for obtaining present frame is estimated
Meter；
In reconstruction process, when pose and its error of the present frame that view-based access control model method is estimated relative to nearest key frame
When more than the first given threshold, using the output pose of optical encoder as the initial pose of camera, it is opposite to reevaluate present frame
In the pose and its error of nearest key frame.
2. the three-dimensional environment method for reconstructing according to claim 1 based on RGBD camera and optical encoder, feature exist
According to the edge and characteristic point of image, selection key frame.
3. the three-dimensional environment method for reconstructing according to claim 1 based on RGBD camera and optical encoder, feature exist
In, when choosing key frame, if former frame cannot function as key frame, using former frame optical encoder pose and present frame
The pose of optical encoder calculate the interframe pose change information between present frame and former frame, the pose for obtaining present frame is estimated
Meter, comprising the following steps:
When choosing key frame, if former frame cannot function as key frame, using former frame optical encoder posture information and
The posture information of the optical encoder of present frame calculates the interframe pose change information between present frame and former frame；
According to the pose of the former frame and the interframe pose change information, the pose estimation of present frame is obtained.
4. the three-dimensional environment method for reconstructing according to claim 1 based on RGBD camera and optical encoder, feature exist
In further comprising the steps of:
Initialize key frame；
If initializing successfully, according to present frame and nearest key frame, the pose estimation of present frame is obtained；
If initialization failure, judges whether former frame can be used as key frame；
If former frame can be used as key frame, former frame is saved as to new key frame, and former frame pose is initialized as
Unit matrix, label initialize successfully.
5. the three-dimensional environment method for reconstructing according to claim 3 based on RGBD camera and optical encoder, feature exist
In further comprising the steps of after according to present frame and nearest key frame, obtaining the pose estimation of present frame:
Calculate pose and error of the present frame relative to nearest key frame；
If error is less than the second given threshold, present frame is saved as into former frame；
If error is greater than the second given threshold, initialization is marked to fail.
6. the three-dimensional environment method for reconstructing according to any one of claims 1 to 5 based on RGBD camera and optical encoder,
It is characterized in that, in reconstruction process, when pose of the present frame that view-based access control model method is estimated relative to nearest key frame
And its error be more than the first given threshold when, using the output pose of optical encoder as the initial pose of camera, reevaluate and work as
Pose and its error of the previous frame relative to nearest key frame, comprising the following steps:
The pose and its error that obtain present frame relative to nearest key frame are estimated based on general visible sensation method；
If the error is greater than the first given threshold, according to the output pose and present frame optical encoder of former frame optical encoder
Pose is exported, interframe pose is obtained；
According to the pose of interframe pose and former frame, the initial estimation of the pose of present frame is obtained；
According to the initial estimation of nearest key frame and the pose of present frame, the pose of present frame is obtained.
7. a kind of three-dimensional environment reconstructing system based on RGBD camera and optical encoder, which is characterized in that including choosing module
With reconstruction module；
The selection module, for if former frame cannot function as key frame, being compiled using the light of former frame when choosing key frame
The pose of the optical encoder of the posture information and present frame of code device calculates the interframe pose variation between present frame and former frame, obtains
Pose to present frame is estimated；
The reconstruction module is used in reconstruction process, when the present frame that view-based access control model method is estimated is closed relative to nearest
The pose and its error of key frame be more than the first given threshold when, using optical encoder output pose as camera initial pose,
Reevaluate pose and its error of the present frame relative to nearest key frame.
8. the three-dimensional environment reconstructing system according to claim 7 based on RGBD camera and optical encoder, feature exist
According to the edge and characteristic point of image, selection key frame.
9. the three-dimensional environment reconstructing system according to claim 7 based on RGBD camera and optical encoder, feature exist
In the selection module includes the first computing unit and the second computing unit；
First computing unit, for if former frame cannot function as key frame, utilizing former frame when choosing key frame
The posture information of the optical encoder of the posture information and present frame of optical encoder calculates the interframe position between present frame and former frame
Appearance change information；
Second computing unit obtains current for the pose and the interframe pose change information according to the former frame
The pose of frame is estimated.
10. the three-dimensional environment reconstructing system according to claim 7 based on RGBD camera and optical encoder, feature exist
In further including initialization module；
The initialization module, for initializing key frame；If initializing successfully, according to present frame and nearest key frame,
Obtain the pose estimation of present frame；If initialization failure, judges whether former frame can be used as key frame, if former frame can be with
As key frame, then former frame is saved as to new key frame, and former frame pose is initialized as unit matrix, label initialization
Success.
11. the three-dimensional environment reconstructing system according to claim 9 based on RGBD camera and optical encoder, feature exist
In the selection module further includes third computing unit；
The third computing unit is for calculating pose and error of the present frame relative to nearest key frame；If error is less than second
When given threshold, present frame is saved as into former frame；If error is greater than the second given threshold, initialization is marked to fail.
12. being according to described in any item rebuild based on the three-dimensional environment of RGBD camera and optical encoder of claim 7 to 11
System, which is characterized in that the reconstruction module includes that pose estimation unit, comparing unit, the 4th computing unit and the 5th calculating are single
Member；
The pose estimation unit obtains position of the present frame relative to nearest key frame for estimating based on general visible sensation method
Appearance and its error；
The comparing unit, if being greater than the first given threshold for the error, according to the output pose of former frame optical encoder
With the output pose of present frame optical encoder, interframe pose is obtained；
4th computing unit obtains initially estimating for the pose of present frame for the pose according to interframe pose and former frame
Meter；
5th computing unit obtains present frame for the initial estimation according to nearest key frame and the pose of present frame
Pose.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811052260.8A CN109191526B (en) | 2018-09-10 | 2018-09-10 | Three-dimensional environment reconstruction method and system based on RGBD camera and optical encoder |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811052260.8A CN109191526B (en) | 2018-09-10 | 2018-09-10 | Three-dimensional environment reconstruction method and system based on RGBD camera and optical encoder |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109191526A true CN109191526A (en) | 2019-01-11 |
| CN109191526B CN109191526B (en) | 2020-07-07 |
Family
ID=64915950
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811052260.8A Active CN109191526B (en) | 2018-09-10 | 2018-09-10 | Three-dimensional environment reconstruction method and system based on RGBD camera and optical encoder |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109191526B (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103247075A (en) * | 2013-05-13 | 2013-08-14 | 北京工业大学 | Variational mechanism-based indoor scene three-dimensional reconstruction method |
| US9460513B1 (en) * | 2015-06-17 | 2016-10-04 | Mitsubishi Electric Research Laboratories, Inc. | Method for reconstructing a 3D scene as a 3D model using images acquired by 3D sensors and omnidirectional cameras |
| CN106875437A (en) * | 2016-12-27 | 2017-06-20 | 北京航空航天大学 | A kind of extraction method of key frame towards RGBD three-dimensional reconstructions |
| CN108124489A (en) * | 2017-12-27 | 2018-06-05 | 深圳前海达闼云端智能科技有限公司 | Information processing method and device, cloud processing equipment and computer program product |
| CN108364344A (en) * | 2018-02-08 | 2018-08-03 | 重庆邮电大学 | A kind of monocular real-time three-dimensional method for reconstructing based on loopback test |
-
2018
- 2018-09-10 CN CN201811052260.8A patent/CN109191526B/en active Active
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103247075A (en) * | 2013-05-13 | 2013-08-14 | 北京工业大学 | Variational mechanism-based indoor scene three-dimensional reconstruction method |
| US9460513B1 (en) * | 2015-06-17 | 2016-10-04 | Mitsubishi Electric Research Laboratories, Inc. | Method for reconstructing a 3D scene as a 3D model using images acquired by 3D sensors and omnidirectional cameras |
| CN106875437A (en) * | 2016-12-27 | 2017-06-20 | 北京航空航天大学 | A kind of extraction method of key frame towards RGBD three-dimensional reconstructions |
| CN108124489A (en) * | 2017-12-27 | 2018-06-05 | 深圳前海达闼云端智能科技有限公司 | Information processing method and device, cloud processing equipment and computer program product |
| CN108364344A (en) * | 2018-02-08 | 2018-08-03 | 重庆邮电大学 | A kind of monocular real-time three-dimensional method for reconstructing based on loopback test |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109191526B (en) | 2020-07-07 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN110490928A (en) | A kind of camera Attitude estimation method based on deep neural network | |
| CN105631861A (en) | Method of restoring three-dimensional human body posture from unmarked monocular image in combination with height map | |
| CN104615234B (en) | Message processing device and information processing method | |
| CN108010081B (en) | RGB-D visual odometer method based on Census transformation and local graph optimization | |
| CN110807364B (en) | Modeling and capturing method and system for three-dimensional face and eyeball motion | |
| CN109671120A (en) | A kind of monocular SLAM initial method and system based on wheel type encoder | |
| CN108428249A (en) | A kind of initial position and orientation estimation method based on optical flow tracking and double geometrical models | |
| CN103182172B (en) | The diagnostic method of golf | |
| CN106548519A (en) | Augmented reality method based on ORB SLAM and the sense of reality of depth camera | |
| CN113706699B (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN103914820B (en) | Image haze removal method and system based on image layer enhancement | |
| CN103002309B (en) | Depth recovery method for time-space consistency of dynamic scene videos shot by multi-view synchronous camera | |
| CN112001859A (en) | Method and system for repairing face image | |
| CN109472830A (en) | A kind of monocular visual positioning method based on unsupervised learning | |
| CN111462210A (en) | Monocular line feature map construction method based on epipolar constraint | |
| CN112308918A (en) | Unsupervised monocular vision odometer method based on pose decoupling estimation | |
| CN107844742A (en) | Facial image glasses minimizing technology, device and storage medium | |
| CN110210426A (en) | Method for estimating hand posture from single color image based on attention mechanism | |
| CN109191366A (en) | Multi-angle of view human body image synthetic method and device based on human body attitude | |
| CN106778576B (en) | Motion recognition method based on SEHM characteristic diagram sequence | |
| CN111784775A (en) | Identification-assisted visual inertia augmented reality registration method | |
| CN107204015A (en) | Instant positioning based on color image and infrared image fusion is with building drawing system | |
| CN114202472A (en) | High-precision underwater imaging method and device | |
| CN106909904A (en) | It is a kind of based on the face front method that can learn Deformation Field | |
| CN109191526A (en) | Three-dimensional environment method for reconstructing and system based on RGBD camera and optical encoder |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |