CN109658507A - Information processing method and device, electronic equipment - Google Patents
Information processing method and device, electronic equipment Download PDFInfo
- Publication number
- CN109658507A CN109658507A CN201811430680.5A CN201811430680A CN109658507A CN 109658507 A CN109658507 A CN 109658507A CN 201811430680 A CN201811430680 A CN 201811430680A CN 109658507 A CN109658507 A CN 109658507A
- Authority
- CN
- China
- Prior art keywords
- information
- dimensional image
- image information
- multiple images
- acquisition unit
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 230000010365 information processing Effects 0.000 title claims abstract description 22
- 238000003672 processing method Methods 0.000 title claims abstract description 15
- 238000000034 method Methods 0.000 claims description 46
- 238000005457 optimization Methods 0.000 claims description 29
- 238000012545 processing Methods 0.000 claims description 23
- 238000010276 construction Methods 0.000 claims description 8
- 238000004590 computer program Methods 0.000 description 16
- 238000010586 diagram Methods 0.000 description 15
- 238000005516 engineering process Methods 0.000 description 14
- 230000006870 function Effects 0.000 description 12
- 230000001133 acceleration Effects 0.000 description 6
- 230000005284 excitation Effects 0.000 description 6
- 238000013519 translation Methods 0.000 description 5
- 238000005259 measurement Methods 0.000 description 4
- 230000005484 gravity Effects 0.000 description 3
- 238000005538 encapsulation Methods 0.000 description 2
- 238000011423 initialization method Methods 0.000 description 2
- 239000011159 matrix material Substances 0.000 description 2
- 239000000758 substrate Substances 0.000 description 2
- 238000007476 Maximum Likelihood Methods 0.000 description 1
- 230000005611 electricity Effects 0.000 description 1
- 238000002474 experimental method Methods 0.000 description 1
- 238000000605 extraction Methods 0.000 description 1
- 210000003205 muscle Anatomy 0.000 description 1
- 238000013139 quantization Methods 0.000 description 1
- 238000005096 rolling process Methods 0.000 description 1
- 230000000007 visual effect Effects 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/04—Context-preserving transformations, e.g. by using an importance map
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
- G06T17/05—Geographic models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/70—Determining position or orientation of objects or cameras
- G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Geometry (AREA)
- Software Systems (AREA)
- Remote Sensing (AREA)
- Computer Graphics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Processing Or Creating Images (AREA)
Abstract
Present disclose provides a kind of information processing methods, comprising: during goal systems operation, obtains the two-dimensional image information of multiple images acquisition unit acquisition, wherein goal systems is for constructing map and positioning to target object；3-D image is constructed according to the parameter information of multiple images acquisition unit and two-dimensional image information；And initialization assignment is carried out according to the parameter that two-dimensional image information and three-dimensional image information are included to goal systems, goal systems is initialized with realizing.The disclosure additionally provides a kind of information processing unit and a kind of electronic equipment.
Description
Technical field
This disclosure relates to a kind of information processing method, device and electronic equipment.
Background technique
Positioning simultaneously with build drawing system be according on mobile terminal camera and the sensors such as Inertial Measurement Unit acquire
Data building mobile terminal ambient enviroment map, and sterically defined technology is carried out to mobile terminal.Currently, positioning simultaneously
It usually requires first to carry out initialization procedure at runtime with drawing system is built, i.e., to positioning simultaneously and the series of parameters for building drawing system
Initial assignment is carried out, this initialization procedure requires the image of camera acquisition that there is certain parallax to need in practical applications
User is motivated using preceding by gently shaking equipment to provide certain rotation and translation, but the initialization procedure is complicated, leads
Cause poor user experience.
Summary of the invention
An aspect of this disclosure provides a kind of information processing method, during being included in goal systems operation, obtains
The two-dimensional image information for taking multiple images acquisition unit to acquire, wherein above-mentioned goal systems is for constructing map and to target pair
As being positioned；3-D image is constructed according to the parameter information of above-mentioned multiple images acquisition unit and above-mentioned two-dimensional image information；
And it is carried out according to the parameter that above-mentioned two-dimensional image information and above-mentioned three-dimensional image information are included to above-mentioned goal systems initial
Change assignment, above-mentioned goal systems is initialized with realizing.
In accordance with an embodiment of the present disclosure, according to above-mentioned two-dimensional image information and above-mentioned three-dimensional image information to above-mentioned target system
Included parameter of uniting carries out initialization assignment, to realize that carrying out initialization to above-mentioned goal systems includes: according to above-mentioned two dimension
Image information and above-mentioned three-dimensional image information determine pose letter when above-mentioned multiple images acquisition unit acquisition two-dimensional image information
Breath；The posture information of above-mentioned multiple images acquisition unit is optimized；And according to the posture information after optimization to above-mentioned mesh
The parameter that mark system is included carries out initialization assignment.
In accordance with an embodiment of the present disclosure, optimizing to the posture information of above-mentioned multiple images acquisition unit includes: use
Posture information when bundle adjustment method acquires two-dimensional image information to above-mentioned multiple images acquisition unit carries out global optimization, obtains
To the posture information of optimization.
In accordance with an embodiment of the present disclosure, believed according to the parameter information of above-mentioned multiple images acquisition unit and above-mentioned two dimensional image
Breath building 3-D image includes carrying out characteristic matching to the two-dimensional image information of above-mentioned multiple images acquisition unit acquisition, and determination is more
Frame associated images；And according to the parameter information of above-mentioned multiple images acquisition unit and above-mentioned multiframe associated images building above-mentioned three
Tie up image.
In accordance with an embodiment of the present disclosure, the parameter information of above-mentioned multiple images acquisition unit includes above-mentioned multiple images acquisition
Evolution information between unit.
Another aspect of the present disclosure provides a kind of information processing unit, comprising: module is obtained, for transporting in goal systems
In capable process, the two-dimensional image information of multiple images acquisition unit acquisition is obtained, wherein above-mentioned goal systems is for constructing ground
Scheme and target object is positioned；Module is constructed, for according to the parameter information of above-mentioned multiple images acquisition unit and above-mentioned
Two-dimensional image information constructs 3-D image；And processing module, for according to above-mentioned two-dimensional image information and above-mentioned 3-D image
The parameter that information is included to above-mentioned goal systems carries out initialization assignment, is initialized with realizing to above-mentioned goal systems.
In accordance with an embodiment of the present disclosure, above-mentioned processing module comprises determining that unit, for according to above-mentioned two-dimensional image information
Posture information when above-mentioned multiple images acquisition unit acquisition two-dimensional image information is determined with above-mentioned three-dimensional image information；Optimization is single
Member is optimized for the posture information to above-mentioned multiple images acquisition unit；And processing unit, after according to optimization
The parameter that posture information is included to above-mentioned goal systems carries out initialization assignment.
In accordance with an embodiment of the present disclosure, above-mentioned optimization unit is used to acquire above-mentioned multiple images using bundle adjustment method
Posture information when unit acquires two-dimensional image information carries out global optimization, the posture information optimized.
In accordance with an embodiment of the present disclosure, above-mentioned building module includes: matching unit, single for acquiring to above-mentioned multiple images
The two-dimensional image information of member acquisition carries out characteristic matching, determines multiframe associated images；And construction unit, for according to above-mentioned more
The parameter information of a image acquisition units and above-mentioned multiframe associated images construct above-mentioned 3-D image.
In accordance with an embodiment of the present disclosure, the parameter information of above-mentioned multiple images acquisition unit includes above-mentioned multiple images acquisition
Evolution information between unit.
Another aspect of the present disclosure provides a kind of electronic equipment, comprising: one or more processors；Memory is used for
Store one or more programs, wherein when one or more of programs are executed by one or more of processors, so that
One or more of processors realize information processing method as described above.
Another aspect of the present disclosure provides a kind of computer readable storage medium, is stored thereon with executable instruction, should
Instruction makes processor realize information processing method as described above when being executed by processor.
Another aspect of the present disclosure provides a kind of computer program, and the computer program, which includes that computer is executable, to be referred to
It enables, described instruction is when executed for realizing information processing method as described above.
Detailed description of the invention
In order to which the disclosure and its advantage is more fully understood, referring now to being described below in conjunction with attached drawing, in which:
Fig. 1 diagrammatically illustrates the application scenarios of information processing method and device according to the embodiment of the present disclosure；
Fig. 2 diagrammatically illustrates the flow chart of system initialization method in the related technology；
Fig. 3 diagrammatically illustrates the flow chart of the information processing method according to the embodiment of the present disclosure；
Fig. 4 is diagrammatically illustrated according to the embodiment of the present disclosure according to two-dimensional image information and three-dimensional image information to target
The parameter that system is included carries out the flow chart of initialization assignment；
Fig. 5 diagrammatically illustrates the parameter information and two dimension according to multiple images acquisition unit according to the embodiment of the present disclosure
The flow chart of image information building 3-D image；
Fig. 6 diagrammatically illustrates the block diagram of the information processing unit according to the embodiment of the present disclosure；
Fig. 7 diagrammatically illustrates the block diagram of the processing module according to the embodiment of the present disclosure；
Fig. 8 diagrammatically illustrates the block diagram of the building module according to the embodiment of the present disclosure；And
Fig. 9 diagrammatically illustrates the side of the electronic equipment for being adapted for carrying out method as described above according to the embodiment of the present disclosure
Block diagram.
Specific embodiment
Hereinafter, will be described with reference to the accompanying drawings embodiment of the disclosure.However, it should be understood that these descriptions are only exemplary
, and it is not intended to limit the scope of the present disclosure.In the following detailed description, to elaborate many specific thin convenient for explaining
Section is to provide the comprehensive understanding to the embodiment of the present disclosure.It may be evident, however, that one or more embodiments are not having these specific thin
It can also be carried out in the case where section.In addition, in the following description, descriptions of well-known structures and technologies are omitted, to avoid
Unnecessarily obscure the concept of the disclosure.
Term as used herein is not intended to limit the disclosure just for the sake of description specific embodiment.It uses herein
The terms "include", "comprise" etc. show the presence of the feature, step, operation and/or component, but it is not excluded that in the presence of
Or add other one or more features, step, operation or component.
There are all terms (including technical and scientific term) as used herein those skilled in the art to be generally understood
Meaning, unless otherwise defined.It should be noted that term used herein should be interpreted that with consistent with the context of this specification
Meaning, without that should be explained with idealization or excessively mechanical mode.
It, in general should be according to this using statement as " at least one in A, B and C etc. " is similar to
Field technical staff is generally understood the meaning of the statement to make an explanation (for example, " system at least one in A, B and C "
Should include but is not limited to individually with A, individually with B, individually with C, with A and B, with A and C, have B and C, and/or
System etc. with A, B, C).Using statement as " at least one in A, B or C etc. " is similar to, generally come
Saying be generally understood the meaning of the statement according to those skilled in the art to make an explanation (for example, " having in A, B or C at least
One system " should include but is not limited to individually with A, individually with B, individually with C, with A and B, have A and C, have
B and C, and/or the system with A, B, C etc.).
Shown in the drawings of some block diagrams and/or flow chart.It should be understood that some sides in block diagram and/or flow chart
Frame or combinations thereof can be realized by computer program instructions.These computer program instructions can be supplied to general purpose computer,
The processor of special purpose computer or other programmable data processing units, so that these instructions are when executed by this processor can be with
Creation is for realizing function/operation device illustrated in these block diagrams and/or flow chart.The technology of the disclosure can be hard
The form of part and/or software (including firmware, microcode etc.) is realized.In addition, the technology of the disclosure, which can be taken, is stored with finger
The form of computer program product on the computer readable storage medium of order, the computer program product is for instruction execution system
System uses or instruction execution system is combined to use.
It includes running in goal systems that embodiment of the disclosure, which provides a kind of information processing method and device, this method,
In the process, the two-dimensional image information of multiple images acquisition unit acquisition is obtained, wherein goal systems is for constructing map and to mesh
Mark object is positioned；3-D image is constructed according to the parameter information of multiple images acquisition unit and two-dimensional image information；And
Initialization assignment is carried out according to the parameter that two-dimensional image information and three-dimensional image information are included to goal systems, to realize to mesh
Mark system is initialized.
Fig. 1 diagrammatically illustrates the application scenarios of information processing method and device according to the embodiment of the present disclosure.It needs to infuse
Meaning, being only shown in Fig. 1 can be using the example of the scene of the embodiment of the present disclosure, to help skilled in the art to understand this
Disclosed technology contents, but it is not meant to that the embodiment of the present disclosure may not be usable for other equipment, system, environment or scene.
As shown in Figure 1, mobile terminal 100 may include image acquisition units 102 and Image Acquisition in the application scenarios
Unit 103, image acquisition units 102 and image acquisition units 103 are separated by a certain distance on spatial position, for constructing map
It may operate in mobile terminal 100 with the goal systems 101 positioned to target object, it can be pre- in goal systems 101
First it is configured with the device parameter of image acquisition units 102 and image acquisition units 103.
In accordance with an embodiment of the present disclosure, surrounding can be acquired by image acquisition units 102 and image acquisition units 103
Two dimensional image can construct 3-D image by related algorithm.
In accordance with an embodiment of the present disclosure, mobile terminal 100 can also include Inertial Measurement Unit (not shown), inertia
Measuring unit (inertial measurement unit or IMU) for measure object triaxial attitude angle (or angular speed) and
The device of acceleration generally can be equipped with the gyroscope of three axis and the accelerometer in three directions in an inertia sensing unit,
The angular speed and acceleration of object in three dimensions are measured, and calculates the posture of object with this.
In accordance with an embodiment of the present disclosure, can be included to goal systems according to two-dimensional image information and three-dimensional image information
Parameter carry out initialization assignment.For example, may not need big parallax excitation, in conjunction with existing two dimensional image in initialization step
Information and three-dimensional image information observation, carry out the initial of degree of precision using Pespective-n-Point (abbreviation PnP) algorithm
Pose estimation and parameter assignment.Specifically, for example, assignment, initialization weight can be carried out to the acceleration of gravity in goal systems
Power acceleration, or assignment etc. is carried out to the relevant parameter of Inertial Measurement Unit.
By embodiment of the disclosure, two-dimensional image information is acquired by multiple images acquisition unit, and establish three-dimensional figure
Picture can carry out initialization assignment to the parameter that goal systems is included according to two-dimensional image information and three-dimensional image information, this
Application reduces initialization process without motivating deterministic process, solves that System Initialization Procedure in the related technology is complicated to ask
Topic, and because constructing three-dimensional map, prior information is obtained, the initialization probability of success is improved.
In order to more clearly describe the application, illustrates the difference of the application and the relevant technologies, letter is carried out to the relevant technologies
Illustrate.For example, Fig. 2 diagrammatically illustrates the flow chart of system initialization method in the related technology.
In the related art, in order to realize system initialization, need user that the preceding gently rolling equipment that passes through is used to provide centainly
Rotation and translation excitation.As shown in Fig. 2, needing to judge whether motion-activated is enough after system operation.Can generally it pass through
Many experiments take an empirical value, and threshold value is less than after motion-activated quantization can consider that excitation is inadequate.Motion-activated generally needs
User is wanted to provide certain rotation and translation excitation by gently shaking equipment using preceding, this process usually requires tens of seconds, especially
When some parameter tunings are improper, it may be necessary to which repeatedly initialization could succeed, if excitation is inadequate or initializes not
Success, requires to re-start excitation, this can undoubtedly reduce user experience.As it can be seen that being carried out using the relevant technologies to system initial
The process of change is complicated, leads to poor user experience.
Fig. 3 diagrammatically illustrates the flow chart of the information processing method according to the embodiment of the present disclosure.
As shown in figure 3, this method includes operation S210~S230.
The two dimensional image of multiple images acquisition unit acquisition is obtained during goal systems operation in operation S210
Information, wherein goal systems is for constructing map and positioning to target object.
In accordance with an embodiment of the present disclosure, goal systems can be while position and build drawing system, i.e. SLAM system, or
VSLAM system (visual-SLAM), for carrying out the three-dimensional reconstruction of " no true scale " and positioned to target object, or
Person VISLAM system (visual-inertial-SLAM), for carrying out the three-dimensional reconstruction of " having true scale " and to target object
It is positioned.
In operation S220,3-D image is constructed according to the parameter information of multiple images acquisition unit and two-dimensional image information.
In accordance with an embodiment of the present disclosure, the parameter information of multiple images acquisition unit includes between multiple images acquisition unit
Evolution information, for example, it may be the spin matrix and translation vector of the relatively right mesh camera of left mesh camera.According to the disclosure
Embodiment, parameter calibration can be carried out to multiple images acquisition unit in advance, the method for parameter calibration can be using more often
The mode of rule, for example, process can be summarized simply as follows it is as follows: 1) print a gridiron pattern, it paste in one plane, as mark
Earnest；2) by adjusting the direction of calibration object, the photo of some different directions is shot for calibration object；3) chessboard is extracted from photo
Each angle point；4) in ideal distortionless situation, find out video camera intrinsic parameter and outer parameter；5) real using least square method estimation
There are the distortion factors under radial distortion on border；6) maximum-likelihood method, optimal estimating promote estimated accuracy.Calibrated multiple figures
As acquisition unit can provide accurate dimensional information, therefore the positioning accuracy of system also has certain promotion.
In accordance with an embodiment of the present disclosure, the parameter information of image acquisition units can also include focal length, parallax range etc..
Two-dimensional image information can be the pixel of characteristic point, the pose (pose) of image acquisition units when acquiring each frame image,
Velocity (speed) etc..When constructing three-dimensional map, suitable three-dimensional reconstruction algorithm can choose, including but not limited to
LSD-SLAM algorithm etc..
In operation S230, carried out just according to the parameter that two-dimensional image information and three-dimensional image information are included to goal systems
Beginningization assignment initializes goal systems with realizing.
In accordance with an embodiment of the present disclosure, three-dimensional image information can be the three-dimensional point cloud with depth, can use
Pespective-n-Point (abbreviation PnP) algorithm carries out the estimation of initial pose and the parameter assignment of degree of precision, and PnP algorithm can
To be spin matrix R and the translation for calculating corresponding two interframe in the case where known three-dimensional point cloud and two-dimensional projection's point information
Vector T, and available dimensional information.The estimation of initial pose and the parameter assignment of degree of precision can be carried out using PnP algorithm.
Specifically, for example, assignment can be carried out to the acceleration of gravity in goal systems, acceleration of gravity is initialized, or to inertia
The relevant parameter of measuring unit carries out assignment etc..
In accordance with an embodiment of the present disclosure, the dimensional information and three-dimensional point cloud information of priori can be obtained, so that initialization is wanted
It asks and substantially reduces, initialization can be completed without motion-activated, improve user experience, be binocular in multiple images acquisition unit
In the case where camera, more observations and dimensional information are provided compared to monocular, initial parameter estimates more robust and accurate, initialization
Quality greatly improves.
By embodiment of the disclosure, two-dimensional image information is acquired by multiple images acquisition unit, and establish three-dimensional figure
Picture can carry out initialization assignment to the parameter that goal systems is included according to two-dimensional image information and three-dimensional image information, this
Application reduces initialization process without motivating deterministic process, solves that System Initialization Procedure in the related technology is complicated to ask
Topic, and because constructing three-dimensional map, prior information is obtained, the initialization probability of success is improved.
Below with reference to Fig. 4~Fig. 5, method shown in Fig. 3 is described further in conjunction with specific embodiments.
Fig. 4 is diagrammatically illustrated according to the embodiment of the present disclosure according to two-dimensional image information and three-dimensional image information to target
The parameter that system is included carries out the flow chart of initialization assignment.
As shown in figure 4, being carried out just according to the parameter that two-dimensional image information and three-dimensional image information are included to goal systems
Beginningization assignment, to realize that carrying out initialization to goal systems includes operation S231~S233.
In operation S231, multiple images acquisition unit acquisition two dimension is determined according to two-dimensional image information and three-dimensional image information
Posture information when image information.
In accordance with an embodiment of the present disclosure, posture information may include position and posture (direction), such as: it is general in two dimension
It is (x, y, yaw), is usually in three-dimensional (x, y, z, aw, pitch, roll), rear three elements describes the posture of object.Wherein
Yaw is course angle, is rotated about the z axis, and pitch is pitch angle, is rotated around Y-axis, and roll is roll angle, is rotated around X-axis.
In operation S232, the posture information of multiple images acquisition unit is optimized.
In accordance with an embodiment of the present disclosure, optimizing to the posture information of multiple images acquisition unit includes: using light beam
Posture information when error compensation method acquires two-dimensional image information to multiple images acquisition unit carries out global optimization, is optimized
Posture information.
Bundle adjustment (bundle adjustment or BA), which refers to, gives multiple images for having common visual field, with a certain frame
The pose of image is reference frame, and feature point feature is extracted from two dimensional image, optimizes this feature in reference frame
Three-dimensional position optimizes three-dimensional pose (pose) of the two dimensional image in reference frame.
By embodiment of the disclosure, all observations and existing frame information is combined to carry out after completing the estimation of initial frame pose
Global B undle adjustment (BA) optimization, once global optimization is completed to initialize successfully, and it is available more accurate first
Beginningization improves the initialization probability of success as a result, stability is high.
In operation S233, initialization assignment is carried out according to the parameter that the posture information after optimization is included to goal systems.
By embodiment of the disclosure, carried out according to the parameter that the posture information after optimization is included to goal systems initial
Change assignment, provide more observations and dimensional information compared to monocular, initial parameter estimates more robust and accurate, and initialization quality is big
It is big to improve.
Fig. 5 diagrammatically illustrates the parameter information and two dimension according to multiple images acquisition unit according to the embodiment of the present disclosure
The flow chart of image information building 3-D image.
As shown in figure 5, constructing 3-D image packet according to the parameter information of multiple images acquisition unit and two-dimensional image information
Include operation S221~S222.
In operation S221, characteristic matching is carried out to the two-dimensional image information of multiple images acquisition unit acquisition, determines multiframe
Associated images.
In accordance with an embodiment of the present disclosure, feature extraction can be carried out to multiframe two dimensional image, in every frame two dimensional image really
Fixed multiple characteristic points, then calculate the similarity between the multiple characteristic points determined, find out similarity greater than d (similarity threshold
Value) characteristic point, based on similarity greater than the characteristic point of d (similarity threshold) can determine between different two dimensional images whether
For similar image, so that it is determined that multiframe associated images out, and tracking association is carried out to associated images.
In operation S222,3-D image is constructed according to the parameter information of multiple images acquisition unit and multiframe associated images.
In accordance with an embodiment of the present disclosure, can according to similarity in multiframe associated images greater than d characteristic point and because
Image difference caused by image acquisition units move calculates the pose of camera and the three-dimensional coordinate of characteristic point in turn, and building is three-dimensional
Image.
By embodiment of the disclosure, 3-D image is constructed by characteristic matching, more observations and scale letter can be provided
Breath, initial parameter estimate more robust and accurate, improve initialization quality.
Fig. 6 diagrammatically illustrates the block diagram of the information processing unit according to the embodiment of the present disclosure.
As shown in fig. 6, information processing unit 300 includes obtaining module 310, building module 320 and processing module 330.
Module 310 is obtained to be used to obtain the two dimension of multiple images acquisition unit acquisition during goal systems operation
Image information, wherein goal systems is for constructing map and positioning to target object.
Module 320 is constructed to be used to construct three-dimensional figure according to the parameter information and two-dimensional image information of multiple images acquisition unit
Picture.
Processing module 330 be used for the parameter for being included to goal systems according to two-dimensional image information and three-dimensional image information into
Row initialization assignment, initializes goal systems with realizing.
By embodiment of the disclosure, two-dimensional image information is acquired by multiple images acquisition unit, and establish three-dimensional figure
Picture can carry out initialization assignment to the parameter that goal systems is included according to two-dimensional image information and three-dimensional image information, this
Application reduces initialization process without motivating deterministic process, solves that System Initialization Procedure in the related technology is complicated to ask
Topic, and because constructing three-dimensional map, prior information is obtained, the initialization probability of success is improved.
Fig. 7 diagrammatically illustrates the block diagram of the processing module according to the embodiment of the present disclosure.
As shown in fig. 7, in accordance with an embodiment of the present disclosure, processing module 330 includes determination unit 331, optimization 332 and of unit
Processing unit 333.
Determination unit 331 is used to determine that multiple images acquisition unit is acquired according to two-dimensional image information and three-dimensional image information
Posture information when two-dimensional image information.
Optimization unit 332 is for optimizing the posture information of multiple images acquisition unit.
Processing unit 333 is used to carry out initialization tax according to the parameter that the posture information after optimization is included to goal systems
Value.
By embodiment of the disclosure, carried out according to the parameter that the posture information after optimization is included to goal systems initial
Change assignment, provide more observations and dimensional information compared to monocular, initial parameter estimates more robust and accurate, and initialization quality is big
It is big to improve.
In accordance with an embodiment of the present disclosure, optimization unit 332 is used for using bundle adjustment method to multiple images acquisition unit
Posture information when acquiring two-dimensional image information carries out global optimization, the posture information optimized.
By embodiment of the disclosure, all observations and existing frame information is combined to carry out after completing the estimation of initial frame pose
Global B undle adjustment (BA) optimization, once global optimization is completed to initialize successfully, and it is available more accurate first
Beginningization improves the initialization probability of success as a result, stability is high.
Fig. 8 diagrammatically illustrates the block diagram of the building module according to the embodiment of the present disclosure.
As shown in figure 8, building module 320 includes matching unit 321 and construction unit 322.
The two-dimensional image information that matching unit 321 is used to acquire multiple images acquisition unit carries out characteristic matching, determines
Multiframe associated images.
Construction unit 322 is used to construct three-dimensional figure according to the parameter information and multiframe associated images of multiple images acquisition unit
Picture.
By embodiment of the disclosure, 3-D image is constructed by characteristic matching, more observations and scale letter can be provided
Breath, initial parameter estimate more robust and accurate, improve initialization quality.
In accordance with an embodiment of the present disclosure, the parameter information of multiple images acquisition unit includes between multiple images acquisition unit
Evolution information.
It is module according to an embodiment of the present disclosure, submodule, unit, any number of or in which any more in subelement
A at least partly function can be realized in a module.It is single according to the module of the embodiment of the present disclosure, submodule, unit, son
Any one or more in member can be split into multiple modules to realize.According to the module of the embodiment of the present disclosure, submodule,
Any one or more in unit, subelement can at least be implemented partly as hardware circuit, such as field programmable gate
Array (FPGA), programmable logic array (PLA), system on chip, the system on substrate, the system in encapsulation, dedicated integrated electricity
Road (ASIC), or can be by the hardware or firmware for any other rational method for integrate or encapsulate to circuit come real
Show, or with any one in three kinds of software, hardware and firmware implementations or with wherein any several appropriately combined next reality
It is existing.Alternatively, can be at least by part according to one or more of the module of the embodiment of the present disclosure, submodule, unit, subelement
Ground is embodied as computer program module, when the computer program module is run, can execute corresponding function.
For example, obtain module 310, building module 320 and processing module 330, matching unit 321, construction unit 322, really
Order member 331, optimization unit 332 and processing unit 333 in any number of may be incorporated in a module realize, Huo Zheqi
In any one module can be split into multiple modules.Alternatively, at least portion of one or more modules in these modules
Point function can be combined at least partly function of other modules, and be realized in a module.According to the implementation of the disclosure
Example, obtain module 310, building module 320 and processing module 330, matching unit 321, construction unit 322, determination unit 331,
Optimization at least one of unit 332 and processing unit 333 can at least be implemented partly as hardware circuit, such as scene can
It programs gate array (FPGA), programmable logic array (PLA), system on chip, the system on substrate, the system in encapsulation, dedicated
Integrated circuit (ASIC), or can be by carrying out hardware or the firmwares such as any other rational method that is integrated or encapsulating to circuit
It realizes, or with any one in three kinds of software, hardware and firmware implementations or with wherein any several appropriately combined
To realize.Alternatively, obtaining module 310, building module 320 and processing module 330, matching unit 321, construction unit 322, determining
Unit 331, optimization at least one of unit 332 and processing unit 333 can at least be implemented partly as computer program
Module can execute corresponding function when the computer program module is run.
In accordance with an embodiment of the present disclosure, electronic equipment includes one or more processors；Memory, for store one or
Multiple programs, wherein when one or more programs are executed by one or more processors, so that one or more processors are real
The information processing method of the existing embodiment of the present disclosure.
Fig. 9 diagrammatically illustrates the side of the electronic equipment for being adapted for carrying out method as described above according to the embodiment of the present disclosure
Block diagram.Electronic equipment shown in Fig. 9 is only an example, should not function to the embodiment of the present disclosure and use scope bring and appoint
What is limited.
As shown in figure 9, electronic equipment 400 includes processor 410 and memory 420.The electronic equipment 400 can execute root
According to the method for the embodiment of the present disclosure.
Specifically, processor 410 for example may include general purpose microprocessor, instruction set processor and/or related chip group
And/or special microprocessor (for example, specific integrated circuit (ASIC)), etc..Processor 410 can also include using for caching
The onboard storage device on way.Processor 410 can be the different movements for executing the method flow according to the embodiment of the present disclosure
Single treatment unit either multiple processing units.
Memory 420, such as can be non-volatile memory, specific example includes but is not limited to: magnetic memory apparatus,
Such as tape or hard disk (HDD)；Light storage device, such as CD (CD-ROM)；Memory, such as random access memory (RAM) or sudden strain of a muscle
It deposits；Etc..
Memory 420 may include computer program 421, which may include that code/computer can be held
Row instruction, executes processor 410 according to the method for the embodiment of the present disclosure or its any change
Shape.
Computer program 421 can be configured to have the computer program code for example including computer program module.Example
Such as, in the exemplary embodiment, the code in computer program 421 may include one or more program modules, for example including
421A, module 421B ....It should be noted that the division mode and number of module are not fixation, those skilled in the art can
To be combined according to the actual situation using suitable program module or program module, when these program modules are combined by processor 410
When execution, processor 410 is executed according to the method for the embodiment of the present disclosure or its any deformation.
In accordance with an embodiment of the present disclosure, processor 410 can be handed over sender unit 430 and signal receiver 440
Mutually, it executes according to the method for the embodiment of the present disclosure or its any deformation.
According to an embodiment of the invention, obtain module 310, building module 320 and processing module 330, matching unit 321,
Construction unit 322, determination unit 331, optimization at least one of unit 332 and processing unit 333 can be implemented as with reference to Fig. 9
Corresponding operating described above may be implemented when being executed by processor 410 in the computer program module of description.
The disclosure additionally provides a kind of computer readable storage medium, which can be above-mentioned reality
It applies included in equipment/device/system described in example；Be also possible to individualism, and without be incorporated the equipment/device/
In system.Above-mentioned computer readable storage medium carries one or more program, when said one or multiple program quilts
When execution, the method according to the embodiment of the present disclosure is realized.
In accordance with an embodiment of the present disclosure, computer readable storage medium can be non-volatile computer-readable storage medium
Matter, such as can include but is not limited to: portable computer diskette, hard disk, random access storage device (RAM), read-only memory
(ROM), erasable programmable read only memory (EPROM or flash memory), portable compact disc read-only memory (CD-ROM), light
Memory device, magnetic memory device or above-mentioned any appropriate combination.In the disclosure, computer readable storage medium can
With to be any include or the tangible medium of storage program, the program can be commanded execution system, device or device use or
Person is in connection.
Flow chart and block diagram in attached drawing are illustrated according to the system of the various embodiments of the disclosure, method and computer journey
The architecture, function and operation in the cards of sequence product.In this regard, each box in flowchart or block diagram can generation
A part of one module, program segment or code of table, a part of above-mentioned module, program segment or code include one or more
Executable instruction for implementing the specified logical function.It should also be noted that in some implementations as replacements, institute in box
The function of mark can also occur in a different order than that indicated in the drawings.For example, two boxes succeedingly indicated are practical
On can be basically executed in parallel, they can also be executed in the opposite order sometimes, and this depends on the function involved.Also it wants
It is noted that the combination of each box in block diagram or flow chart and the box in block diagram or flow chart, can use and execute rule
The dedicated hardware based systems of fixed functions or operations is realized, or can use the group of specialized hardware and computer instruction
It closes to realize.
It will be understood by those skilled in the art that the feature recorded in each embodiment and/or claim of the disclosure can
To carry out multiple combinations and/or combination, even if such combination or combination are not expressly recited in the disclosure.Particularly, exist
In the case where not departing from disclosure spirit or teaching, the feature recorded in each embodiment and/or claim of the disclosure can
To carry out multiple combinations and/or combination.All these combinations and/or combination each fall within the scope of the present disclosure.
Although the disclosure, art technology has shown and described referring to the certain exemplary embodiments of the disclosure
Personnel it should be understood that in the case where the spirit and scope of the present disclosure limited without departing substantially from the following claims and their equivalents,
A variety of changes in form and details can be carried out to the disclosure.Therefore, the scope of the present disclosure should not necessarily be limited by above-described embodiment,
But should be not only determined by appended claims, also it is defined by the equivalent of appended claims.
Claims (10)
1. a kind of information processing method, comprising:
During goal systems operation, the two-dimensional image information of multiple images acquisition unit acquisition is obtained, wherein the mesh
Mark system is for constructing map and positioning to target object；
3-D image is constructed according to the parameter information of described multiple images acquisition unit and the two-dimensional image information；And
It is carried out according to the parameter that the two-dimensional image information and the three-dimensional image information are included to the goal systems initial
Change assignment, the goal systems is initialized with realizing.
2. according to the method described in claim 1, wherein, according to the two-dimensional image information and the three-dimensional image information to institute
It states the parameter that goal systems is included and carries out initialization assignment, to realize that carrying out initialization to the goal systems includes:
Determine that described multiple images acquisition unit acquires X-Y scheme according to the two-dimensional image information and the three-dimensional image information
Posture information when as information；
The posture information of described multiple images acquisition unit is optimized；And
Initialization assignment is carried out according to the parameter that the posture information after optimization is included to the goal systems.
3. according to the method described in claim 2, wherein, optimizing packet to the posture information of described multiple images acquisition unit
It includes:
Posture information when acquiring two-dimensional image information to described multiple images acquisition unit using bundle adjustment method carries out complete
Office's optimization, the posture information optimized.
4. according to the method described in claim 1, wherein, according to the parameter information of described multiple images acquisition unit and described two
Dimension image information constructs 3-D image
Characteristic matching is carried out to the two-dimensional image information of described multiple images acquisition unit acquisition, determines multiframe associated images；With
And
The 3-D image is constructed according to the parameter information of described multiple images acquisition unit and the multiframe associated images.
5. according to the method described in claim 1, wherein, the parameter information of described multiple images acquisition unit includes the multiple
Evolution information between image acquisition units.
6. a kind of information processing unit, comprising:
Module is obtained, for during goal systems operation, obtaining the two dimensional image letter of multiple images acquisition unit acquisition
Breath, wherein the goal systems is for constructing map and positioning to target object；
Module is constructed, it is three-dimensional for being constructed according to the parameter information of described multiple images acquisition unit and the two-dimensional image information
Image；And
Processing module, for being included to the goal systems according to the two-dimensional image information and the three-dimensional image information
Parameter carries out initialization assignment, is initialized with realizing to the goal systems.
7. device according to claim 6, wherein the processing module includes:
Determination unit, for determining that described multiple images acquisition is single according to the two-dimensional image information and the three-dimensional image information
Posture information when member acquisition two-dimensional image information；
Optimize unit, is optimized for the posture information to described multiple images acquisition unit；And
Processing unit, the parameter for being included to the goal systems according to the posture information after optimization carry out initialization tax
Value.
8. device according to claim 7, wherein the optimization unit is used for using bundle adjustment method to the multiple
Posture information when image acquisition units acquire two-dimensional image information carries out global optimization, the posture information optimized.
9. device according to claim 6, wherein the building module includes:
Matching unit, the two-dimensional image information for acquiring to described multiple images acquisition unit carry out characteristic matching, and determination is more
Frame associated images；And
Construction unit, for according to the parameter information of described multiple images acquisition unit and multiframe associated images building
3-D image.
10. a kind of electronic equipment, comprising:
One or more processors；
Memory, for storing one or more programs, wherein when one or more of programs are by one or more of places
When managing device execution, so that one or more of processors realize information processing side described in any one of claims 1 to 5
Method.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811430680.5A CN109658507A (en) | 2018-11-27 | 2018-11-27 | Information processing method and device, electronic equipment |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811430680.5A CN109658507A (en) | 2018-11-27 | 2018-11-27 | Information processing method and device, electronic equipment |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN109658507A true CN109658507A (en) | 2019-04-19 |
Family
ID=66111627
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811430680.5A Pending CN109658507A (en) | 2018-11-27 | 2018-11-27 | Information processing method and device, electronic equipment |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109658507A (en) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113034367A (en) * | 2021-03-29 | 2021-06-25 | 京东数字科技控股股份有限公司 | Information processing method, device, computer system and readable storage medium |
| WO2022267781A1 (en) * | 2021-06-26 | 2022-12-29 | 华为技术有限公司 | Modeling method and related electronic device, and storage medium |
| CN115661493A (en) * | 2022-12-28 | 2023-01-31 | 航天云机(北京)科技有限公司 | Object pose determination method and device, equipment and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103297486A (en) * | 2012-03-05 | 2013-09-11 | 联想(北京)有限公司 | Method and device used for simultaneous localization and mapping |
| US20140327792A1 (en) * | 2013-05-02 | 2014-11-06 | Qualcomm Incorporated | Methods for facilitating computer vision application initialization |
| CN106485744A (en) * | 2016-10-10 | 2017-03-08 | 成都奥德蒙科技有限公司 | A kind of synchronous superposition method |
| CN107833245A (en) * | 2017-11-28 | 2018-03-23 | 北京搜狐新媒体信息技术有限公司 | SLAM method and system based on monocular vision Feature Points Matching |
| CN108665540A (en) * | 2018-03-16 | 2018-10-16 | 浙江工业大学 | Robot localization based on binocular vision feature and IMU information and map structuring system |
| CN108827315A (en) * | 2018-08-17 | 2018-11-16 | 华南理工大学 | Vision inertia odometer position and orientation estimation method and device based on manifold pre-integration |
-
2018
- 2018-11-27 CN CN201811430680.5A patent/CN109658507A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103297486A (en) * | 2012-03-05 | 2013-09-11 | 联想(北京)有限公司 | Method and device used for simultaneous localization and mapping |
| US20140327792A1 (en) * | 2013-05-02 | 2014-11-06 | Qualcomm Incorporated | Methods for facilitating computer vision application initialization |
| CN106485744A (en) * | 2016-10-10 | 2017-03-08 | 成都奥德蒙科技有限公司 | A kind of synchronous superposition method |
| CN107833245A (en) * | 2017-11-28 | 2018-03-23 | 北京搜狐新媒体信息技术有限公司 | SLAM method and system based on monocular vision Feature Points Matching |
| CN108665540A (en) * | 2018-03-16 | 2018-10-16 | 浙江工业大学 | Robot localization based on binocular vision feature and IMU information and map structuring system |
| CN108827315A (en) * | 2018-08-17 | 2018-11-16 | 华南理工大学 | Vision inertia odometer position and orientation estimation method and device based on manifold pre-integration |
Non-Patent Citations (3)
| Title |
|---|
| 吴伟仁等: "《深空探测器自主导航原理与技术》", 31 May 2011 * |
| 张广军: "《光电测试技术与系统》", 31 March 2010 * |
| 王耀兵等: "《空间机器人》", 31 May 2018 * |
Cited By (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113034367A (en) * | 2021-03-29 | 2021-06-25 | 京东数字科技控股股份有限公司 | Information processing method, device, computer system and readable storage medium |
| CN113034367B (en) * | 2021-03-29 | 2024-04-16 | 京东科技控股股份有限公司 | Information processing method, information processing device, computer system and readable storage medium |
| WO2022267781A1 (en) * | 2021-06-26 | 2022-12-29 | 华为技术有限公司 | Modeling method and related electronic device, and storage medium |
| CN115661493A (en) * | 2022-12-28 | 2023-01-31 | 航天云机(北京)科技有限公司 | Object pose determination method and device, equipment and storage medium |
| CN115661493B (en) * | 2022-12-28 | 2023-07-04 | 航天云机(北京)科技有限公司 | Method, device, equipment and storage medium for determining object pose |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20210012520A1 (en) | Distance measuring method and device | |
| US9922422B2 (en) | Mobile imaging platform calibration | |
| US11610373B2 (en) | Method of generating three-dimensional model data of object | |
| US20190370983A1 (en) | System and method for real-time location tracking of a drone | |
| CN111436216A (en) | Method and system for color point cloud generation | |
| JP2020506487A (en) | Apparatus and method for obtaining depth information from a scene | |
| JP6782903B2 (en) | Self-motion estimation system, control method and program of self-motion estimation system | |
| US11082633B2 (en) | Method of estimating the speed of displacement of a camera | |
| CN106908046A (en) | Measurement subsystem and measuring system | |
| WO2020133172A1 (en) | Image processing method, apparatus, and computer readable storage medium | |
| JP2013232195A5 (en) | ||
| CN108780577A (en) | Image processing method and equipment | |
| CN109658507A (en) | Information processing method and device, electronic equipment | |
| CN113767264A (en) | Parameter calibration method, device, system and storage medium | |
| KR20200056613A (en) | System for correcting geometry of mobile platform with sensor based on an orthophoto | |
| KR20200037502A (en) | Method and apparatus of outputting pose information | |
| CN113240813A (en) | Three-dimensional point cloud information determination method and device | |
| CN110825079A (en) | Map construction method and device | |
| US11758100B2 (en) | Portable projection mapping device and projection mapping system | |
| KR102249381B1 (en) | System for generating spatial information of mobile device using 3D image information and method therefor | |
| CN109891188A (en) | Mobile platform, camera paths generation method, program and recording medium | |
| US20210256732A1 (en) | Image processing method and unmanned aerial vehicle | |
| CN115560744A (en) | Robot, multi-sensor-based three-dimensional mapping method and storage medium | |
| CN113433566B (en) | Map construction system and map construction method | |
| CN111161357B (en) | Information processing method and device, augmented reality device and readable storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |