CN106548466A - The method and apparatus of three-dimensional reconstruction object - Google Patents
The method and apparatus of three-dimensional reconstruction object Download PDFInfo
- Publication number
- CN106548466A CN106548466A CN201510591091.5A CN201510591091A CN106548466A CN 106548466 A CN106548466 A CN 106548466A CN 201510591091 A CN201510591091 A CN 201510591091A CN 106548466 A CN106548466 A CN 106548466A
- Authority
- CN
- China
- Prior art keywords
- voxel
- tsdf
- tsdf values
- values
- value
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Abstract
The invention discloses a kind of method and apparatus of three-dimensional reconstruction object.The method of the three-dimensional reconstruction object includes：The initial local for obtaining voxel in three dimensions blocks signed distance function TSDF values；Optimization problem is solved, to obtain an overall situation TSDF value of voxel；And based on resulting global TSDF values, object described in three-dimensional reconstruction；Wherein, in the optimization problem, the global TSDF values of one voxel are worth to based on final local T SDF of the voxel, the final local T SDF value of one voxel is equal to the initial local TSDF values of the corresponding voxel of voxel Jing rigid transformations, variable is the global TSDF value and the parameter of rigid transformation of voxel, and cost function is related to following factors：The quadratic sum of the difference of the initial local TSDF values of the global TSDF values voxel corresponding with voxel Jing rigid transformations of voxel.
Description
Technical field
This invention relates generally to three-dimensional imaging field.Specifically, the present invention relates to one kind can be accurate
The method and apparatus for really carrying out the three-dimensional reconstruction of object.
Background technology
In recent years, the application of many correlations with the development of 3 Dimension Image Technique, has been emerged in large numbers, has such as been strengthened
Reality, digital museum, 3 D-printing etc..The importance of 3 Dimension Image Technique is three-dimensional reconstruction
Technology.The three-dimensional rebuilding method of main flow rebuilds three dimensional object based on one group of depth image.
Traditional three-dimensional rebuilding method focuses mainly on how more accurately carrying out the parameter of camera
Estimate, for example, deformation information is introduced to estimate the distortion of camera.But, traditional three-dimensional reconstruction
Method, (blocks signed distance function, Truncated Signed Distance overall situation TSDF is calculated
When Function) characterizing, simply simply all of local T SDF is characterized carries out arithmetic average,
Fully excavate local T SDF to characterize and the relation between overall situation TSDF signs.Change due to only relying on
Enter the estimation to camera parameter to lift the effect of three-dimensional reconstruction, so traditional three-dimensional rebuilding method
Effect improved limited, the accuracy of three-dimensional reconstruction object needs further to be improved.
Therefore, it is desirable to a kind of method and apparatus of three-dimensional reconstruction object, which can carry out object exactly
Three-dimensional reconstruction.
The content of the invention
The brief overview with regard to the present invention is given below, to provide with regard to some of the invention
The basic comprehension of aspect.It should be appreciated that this general introduction is not the exhaustive general introduction with regard to the present invention.
It is not intended to the crucial or pith for determining the present invention, nor the model of the intended limitation present invention
Enclose.Its purpose only provides some concepts in simplified form, more detailed in this, as what is discussed after a while
The preamble of thin description.
The purpose of the present invention is the problems referred to above for prior art, it is proposed that one kind lays particular emphasis on excavation office
The method and apparatus of the three-dimensional reconstruction object of the relation between portion TSDF is characterized and overall situation TSDF is characterized.
To achieve these goals, according to an aspect of the invention, there is provided a kind of three-dimensional of object
Method for reconstructing, including：The initial local for obtaining voxel in three dimensions blocks signed distance function TSDF
Value；Optimization problem is solved, to obtain an overall situation TSDF value of voxel；And based on resulting
Global TSDF values, object described in three-dimensional reconstruction；Wherein, in the optimization problem, one
The global TSDF values of voxel are worth to based on final local T SDF of the voxel, a voxel it is final
Local T SDF value is equal to the initial local TSDF values of the corresponding voxel of voxel Jing rigid transformations, variable
It is the global TSDF values and the parameter of rigid transformation of voxel, cost function is related to following factors：Body
The difference of the initial local TSDF values of the global TSDF values voxel corresponding with voxel Jing rigid transformations of element
Quadratic sum.
According to another aspect of the present invention, there is provided a kind of three-dimensional reconstruction equipment of object, including：
Device is obtained, is configured to：The initial local for obtaining voxel in three dimensions blocks signed distance function
TSDF values；Solving device, is configured to：Optimization problem is solved, to obtain voxel is complete
Office's TSDF values；And reconstructing device, it is configured to：It is based on resulting global TSDF values, three-dimensional
Rebuild the object；Wherein, in the optimization problem, the global TSDF values base of a voxel
It is worth in final local T SDF of the voxel, the final local T SDF value of a voxel is equal to the body
The initial local TSDF values of the corresponding voxel of plain Jing rigid transformations, variable is the global TSDF of voxel
The parameter of value and rigid transformation, cost function are related to following factors：The global TSDF values of voxel with
The quadratic sum of the difference of the initial local TSDF values of the corresponding voxel of voxel Jing rigid transformations.
In addition, according to a further aspect in the invention, additionally provide a kind of storage medium.The storage is situated between
Matter includes machine-readable program code, when described program code is performed on message processing device,
Described program code causes described information processing equipment to perform said method of the invention.
Additionally, in accordance with a further aspect of the present invention, additionally provide a kind of program product.Described program is produced
Product include the executable instruction of machine, when the instruction is performed on message processing device, the finger
Order causes described information processing equipment to perform said method of the invention.
Description of the drawings
With reference to explanation below in conjunction with the accompanying drawings to embodiments of the invention, this can be more readily understood that
Bright above and other objects, features and advantages.Part in accompanying drawing is intended merely to illustrate the present invention's
Principle.In the accompanying drawings, same or similar technical characteristic or part will be using same or similar attached
Icon is remembered to represent.In accompanying drawing：
The flow chart that Fig. 1 shows three-dimensional reconstruction object method according to an embodiment of the invention；
Fig. 2 shows the flow chart for obtaining initial local TSDF methods according to an embodiment of the invention；
Fig. 3 shows the flow chart for calculating initial local TSDF methods according to an embodiment of the invention；
Fig. 4 shows the structure square frame of the equipment of three-dimensional reconstruction object according to an embodiment of the invention
Figure；And
Fig. 5 shows the computer that can be used to implementing method and apparatus according to an embodiment of the invention
Schematic block diagram.
Specific embodiment
The one exemplary embodiment of the present invention is described in detail hereinafter in connection with accompanying drawing.In order to clear
Chu and it is simple and clear for the sake of, all features of actual embodiment are not described in the description.However, should
The understanding, must make many specific to enforcement during any this actual embodiment is developed
The decision of mode, to realize the objectives of developer, for example, meets and system and business phase
Those restrictive conditions for closing, and these restrictive conditions may be with the different of embodiment
Change.Additionally, it also should be appreciated that, although development is likely to be extremely complex and time-consuming, but
For the those skilled in the art for having benefited from present disclosure, this development is only routine
Task.
Here, in addition it is also necessary to which explanation is a bit, in order to avoid having obscured this because of unnecessary details
It is bright, illustrate only in the accompanying drawings the apparatus structure closely related with scheme of the invention and/or
Process step, and eliminate the other details little with relation of the present invention.In addition, it is also stated that
It is that the element and feature described in an a kind of accompanying drawing or embodiment of the present invention can be with one
Or the element that illustrates in more other accompanying drawings or embodiment and feature combine.
The three-dimensional rebuilding method of object according to an embodiment of the invention is described below with reference to Fig. 1
Flow process.
The flow chart that Fig. 1 shows three-dimensional reconstruction object method according to an embodiment of the invention.As schemed
Shown in 1, three-dimensional reconstruction object method comprises the steps according to an embodiment of the invention：Obtain three
In dimension space, the initial local of voxel blocks signed distance function TSDF values (step S1)；Solve most
Optimization problem, to obtain overall situation TSDF values (step S2) of voxel；And based on resulting
Global TSDF values, object (step S3) described in three-dimensional reconstruction；Wherein, ask in the optimization
In topic, the global TSDF values of a voxel are worth to based on final local T SDF of the voxel, one
The final local T SDF value of voxel is equal to the initial local of the corresponding voxel of voxel Jing rigid transformations
TSDF values, variable is the global TSDF value and the parameter of rigid transformation of voxel, cost function with it is following
Factor is related：The initial office of the global TSDF values voxel corresponding with voxel Jing rigid transformations of voxel
The quadratic sum of the difference of portion's TSDF values.
In step sl, the initial local for obtaining voxel in three dimensions blocks signed distance function
TSDF values.
Specifically, three dimensions are evenly divided into some voxels, it is believed that voxel is three-dimensional
The elementary cell in space.Three dimensional object is rebuild due to based on one group of depth map, so three-dimensional reconstruction pair
As the input of method is one group of depth map.This group of depth map is shot by multiple cameras and is obtained, each
Camera corresponds to a depth map.Again as input does not include the parameter of camera, so needing based on deep
Degree figure is estimating the parameter of corresponding camera, and and then obtains the initial local TSDF values of associated voxels.
In addition, it should be noted here that " the multiple cameras " mentioned herein is included in different positions
With the situation of multiple cameras physically of direction, also including same camera by adjustment position and/
Or towards and form the situation of multiple cameras in logic, also including the feelings of above-mentioned two situations mixing
Condition.
The initial local TSDF values of voxel are defined as what is calculated in three dimensions based on camera parameter
Voxel relative between the depth of the voxel correspondence position in the depth and depth map of camera it is oriented away from
From the result Jing after blocking.Such TSDF values are referred to as into initial local TSDF values.From defined above
Understand, a voxel can calculate a directed distance defined above relative to a camera, one
Individual voxel can calculate multiple directed distances defined above relative to multiple cameras, each it is oriented away from
From corresponding to a camera.Directed distance is due to will be through break-in operation, so a voxel may be right
Should in one or more initial locals TSDF values, the corresponding initial local TSDF values of voxel it is upper
Limit is the number of camera/depth map.Jing after blocking, some voxels can no initial local TSDF values,
Such voxel is no longer considered.That is, being calculated initial local TSDF values in step sl
Voxel just can be continued with follow-up step S2, S3.
Due to camera capture depth map when, acquisition be subject surface information, it is possible to
Understand based on camera parameter to be the voxel relative to the depth of camera in the voxel that three dimensions are calculated
With the projection in the direction of the optical axis of the distance between the imaging point of camera, and voxel correspondence in depth map
The depth of position is in the subject surface passed through by the light between the voxel and the imaging point of camera
Projection of the point with the distance between the imaging point of camera in the direction of the optical axis.As the voxel may be relative
The point in the subject surface is farther or closer to so distance projection in the direction of the optical axis apart from camera
Difference be directed distance.Distance in the direction of the optical axis be projected as zero, show the voxel and subject surface
On the point overlap, namely the voxel is the point in subject surface.
In theory, the null all voxels of initial local TSDF values constitute subject surface.But,
As the calculating of the value of initial local TSDF depends on the estimation of camera parameter, the estimation of camera parameter
Value might not be completely the same with the actual value of camera parameter, so initial local TSDF values can not
It is used directly to rebuild the surface of three dimensional object.A kind of possible method is that have initial local to each
All initial local TSDF values of the voxel of TSDF values seek arithmetic mean of instantaneous value, using arithmetic mean of instantaneous value as
The global TSDF values of the voxel, are then based on the global TSDF values and the voxel with the value to rebuild
The surface of three dimensional object.However, so generate the problem that have cured initial local TSDF values with it is complete
Relation between office's TSDF values.
As camera has multiple, so the situation of the parameter estimation of multiple cameras there may be difference, because
Different cameral corresponding initial local TSDF values are converted to global TSDF by the way of unified by this
Value is obviously not accurate enough.
Through foregoing description, it will be understood that step S1 can pass through the method shown in Fig. 2 and realize.
As shown in Fig. 2 in the step s 21, estimate corresponding with multiple depth maps multiple magazine
The camera parameter of each camera, camera parameter include but is not limited to the position and orientation of camera.For example,
Can be carried out using the matching algorithm between the depth map and threedimensional model used in Kinect Fusion
Camera parameter is estimated.Of course, it is possible to adopt all suitable camera parameter estimation sides known in the art
Method realizes step S21.
In step S22, based on estimated camera parameter, one or more of calculating voxel are initial
Local T SDF value.
Step S22 can pass through the method shown in Fig. 3 and realize.
As shown in figure 3, for each voxel and multiple depth of initial local TSDF values to be calculated
Corresponding each camera of figure, in step S31, based on the camera parameter estimated by camera, counts
Calculate first depth value of the voxel relative to the camera.In step s 32, determine the voxel it is corresponding,
The second depth value in the depth map of the camera association.In step S33, by the first depth value and
The difference of two depth values block after result, as voxel initial local corresponding with the camera
TSDF values.
It should be noted that during when calculating the initial local TSDF values of voxel, the scope of voxel is three dimensions
Predetermined estimation can be fully contemplated by the region of object.As the calculating of initial local TSDF is related to cut
Disconnected operation, so Jing after blocking, the scope of the voxel with initial local TSDF values further will contract
It is little.By the threshold value for controlling to block, it is right to be limited to the voxel with initial local TSDF values
The near surface of elephant.The threshold value blocked can for example be positive and negative 5 centimetres, namely initial local
The span of TSDF values can be [- 5,5].
For convenience of calculation, also initial local TSDF values are normalized.Normalized threshold value is
For the threshold value blocked.
Block the near surface that voxel is ensure that in object, will also be seen that, voxel from follow-up explanation
Jing rigid transformations can correspond to the voxel itself or another voxel, in order to ensure the body before and after rigid transformation
Element is further limited to the initial local TSDF values of voxel all in the near surface of object.
Specifically, the absolute value for limiting the initial local TSDF values of voxel is less than or equal to specific threshold
α, specific threshold α belong to (0,1).That is, the initial local TSDF values of [- α, α]
Voxel Jing rigid transformations can correspond to voxel of its initial local TSDF values between [- 1,1].
If additionally, respectively in coordinate system (such as seat with the camera as origin of each camera association
Mark system) middle calculating initial local TSDF values, then the initial local TSDF values of calculated voxel are right
Should be in the coordinate system of multiple magazine camera associations.Can be by the initial office of calculated voxel
Portion's TSDF values are transformed in the same coordinate system.For example, by the initial local TSDF of calculated voxel
Value is transformed in the coordinate system of first camera association.By by initial local TSDF primary systems one to
Individual coordinate system, can facilitate from initial local TSDF values to the rigid transformation of final local T SDF value with
And and then try to achieve global TSDF values, can reduce amount of calculation, and globally optimal solution can be obtained and
Non local optimal solution.
It is of course also possible to direct calculate initial office of the voxel relative to each camera in global coordinate system
Portion's TSDF values.Using the benefit of this kind of mode can be avoid by the initial local TSDF values of voxel from
The coordinate system of each camera association is transformed into the loss of significance produced during the same coordinate system.
As previously described, because initial local TSDF values are calculated based on the camera parameter estimated, and not
With the corresponding different cameral of depth map estimation parameter not necessarily entirely accurate and inaccurate
Situation is not necessarily identical, so being not suitable for all initial locals in a uniform manner to voxel
TSDF values are processed to obtain the global TSDF values of voxel.In fact, multiple voxels are corresponding to same
The calculating of the initial local TSDF values of one camera depends on the estimation parameter of same camera, so
It is considered that multiple first voxels should pass through flat corresponding to the initial local TSDF values of same camera
After moving and rotating, integral transformation (inverse transformation of rigid transformation) becomes many at multiple second voxels
The final local T SDF value of individual second voxel.Multiple first voxels can be with identical with multiple second voxels
(null transformation), it is also possible to different (for the integral transformations of a camera).Specific threshold hereinbefore
α defines the span of the initial local TSDF of the second voxel, so as to ensure that the second voxel
Final local T SDF value belongs to [- 1,1].
It should be noted that rigid transformation is carried out for same camera.That is, the corresponding category of same camera
In multiple first voxels multiple initial local TSDF values in these first voxels through same rigidity
The inverse transformation of conversion become while corresponding to multiple second voxels these the second voxels corresponding to this
The multiple final local T SDF value of camera.So-called rigidity is referred to corresponding to the multiple initial of same camera
The rotationally and/or translationally conversion of local T SDF value Jing unification reaches the second voxel (TSDF from the first voxel
Value itself is constant), the relative position relation between voxel associated by these values before and after conversion not
Change.
Through the amendment of rigid transformation, counteract the error of camera parameter estimation so that TSDF values and
The corresponding relation of voxel is more accurate.Meanwhile, global TSDF values are worth to from final local T SDF,
Just three dimensional object can be rebuild based on overall situation TSDF values further.
The present invention is realized from initial local TSDF values by the way of design and solution optimization problem
The rigid transformation that is related to final local T SDF value and from final local T SDF value to global TSDF
The calculating of value.
The design key of optimization problem is the design of variable and cost function.
In optimization problem, the final local of the global TSDF values of a voxel based on the voxel
TSDF is worth to, and the final local T SDF value of a voxel is corresponding equal to voxel Jing rigid transformations
The initial local TSDF values of voxel, variable are the global TSDF values and the parameter of rigid transformation of voxel,
Cost function is related to following factors：The global TSDF values of voxel are corresponding with voxel Jing rigid transformations
Voxel initial local TSDF values difference quadratic sum.
Cost function for example can be designed as the global TSDF values and voxel Jing rigid transformations pair of voxel
The summation of the quadratic sum of the difference of the initial local TSDF values of the voxel answered.Step S1 is noted above
When obtaining initial local TSDF values, due to having carried out blocking, normalization, and in some embodiments
Middle utilization threshold alpha is further limited, and causes only part voxel to have initial local TSDF values, after
Continuous step will be carried out for these voxels.Therefore, cost function is can be designed as these voxels
Each calculation cost item, cost function are the summations of cost item, and each cost item is the overall situation of voxel
The difference of the initial local TSDF values of TSDF values voxel corresponding with voxel Jing rigid transformations square
With, namely the quadratic sum of the difference of the final local T SDF value of the global TSDF values of voxel and the voxel.
Here why it is designed as final local of the global TSDF values of a voxel based on the voxel
TSDF is worth to, and the final local T SDF value of a voxel is corresponding equal to voxel Jing rigid transformations
The initial local TSDF values of voxel, rather than the global TSDF values of a voxel are designed as based on the body
The initial local TSDF of element is worth to, if being because designing according to the latter, optimization problem
Solution can be a voxel global TSDF values be equal to the voxel initial local TSDF values arithmetic/add
Weight average value.By introduce for camera it is overall only relate to the rigid transformation for translating and rotating,
The conversion from initial local TSDF values to final local T SDF value is increased, so that optimization is asked
The solution of topic is more accurate, causes the result of three-dimensional reconstruction object more accurate.
If overall situation TSDF values are V, depth map number is n, then the corresponding initial local of each depth map
TSDF values are V1、V2、……、Vn, the corresponding final local T SDF value of each depth map is V1’、
V2’、……、Vn', serial number i of depth map, i=1,2 ... ..., n.For the voxel p in V,
Which is in Vi' in corresponding voxel be still p, and in ViIn corresponding voxel be Ti(p), wherein TiFor optimization
The rigid transformation for solving is needed in problem.
As it was noted above, voxel Jing block, normalization and there is initial local TSDF values, that is, require
ViP the value of () is located at [- 1,1] interval.Also need to herein seek Vi(Ti(p))∈[-1,1].Due to ViWith Vi' between
Violent conversion will not be carried out, that is to say, that voxel p and voxel TiP the distance between () will not be far,
So definition set Pi：Pi=p | Vi(p) ∈ [- α, α] }, enabling ensure Vi(Ti(p))∈[-1,1]。
That is, during calculation cost function, for belonging to set PiVoxel p calculation cost items
Summation, each cost item is global TSDF values of the voxel p in V and voxel p Jing rigid transformations pair
The V for answeringiIn voxel TiP the quadratic sum of the difference of the initial local TSDF values of (), a square summation are because
Voxel p may have multiple final local T SDF values, so as to there is multiple voxel Ti(p) it is initial
Local T SDF value.
So cost function can be expressed as：
Optimization problem is solved, that is, solves the final global TSDF for minimizing above-mentioned cost function
The optimal value of value and rigid transformation parameters.
Therefore, in step s 2, optimization problem is solved, to obtain an overall situation TSDF of voxel
Value.
An overall situation TSDF value of voxel is to emphasize that a voxel is finally only global with one herein
TSDF values, and the scope of such voxel is set Pi。
Optimization problem can be solved by Gauss-Newton (Gauss-Newton) method of iteration or
Calculated come iterative respectively by the parameter of global TSDF values and rigid transformation for voxel.
But no matter which kind of solves mode, is directed to iteration, it is necessary to set the initial value of iteration.
In optimization problem, the initial value of the global TSDF values of voxel is equal to all first of the voxel
The weighted mean of beginning local T SDF value.On the one hand weighting herein can be arithmetic average, another
Aspect can also calculate weight according to the distance of camera and voxel.
It is the body that the initial value of the parameter of rigid transformation causes the corresponding voxel of voxel Jing rigid transformations
Element itself.One voxel may have multiple initial local TSDF values, if voxel Jing rigidly becomes
Change and correspond to the voxel itself, then the initialized final local T SDF value of this voxel also has many
It is individual, each correspond to a camera.
For example, rigid transformation Ti(p)=Ri*p+ti, wherein RiIt is spin matrix, list can be initialized as
Position battle array, tiFor translation vector, null vector can be initialized as.
Obtain in step s 2 and belong to set PiAll voxel p global TSDF values after, so that it may
With based on this reconstructed object.
In step s3, based on resulting global TSDF values, object described in three-dimensional reconstruction.
For example, using marching cube (marching cube) algorithm, based on resulting complete
Office's TSDF values, the surface of object described in three-dimensional reconstruction.Marching cubes algorithm is known in the art
Algorithm, will not be described here.
The equipment of three-dimensional reconstruction object according to an embodiment of the invention is described next, with reference to Fig. 4.
Fig. 4 shows the structure square frame of the equipment of three-dimensional reconstruction object according to an embodiment of the invention
Figure.As shown in figure 4, three-dimensional reconstruction object-based device of the invention 400 includes：Obtain device
41, it is configured to：The initial local for obtaining voxel in three dimensions blocks signed distance function TSDF
Value；Solving device 42, is configured to：Optimization problem is solved, to obtain voxel is global
TSDF values；And reconstructing device 43, it is configured to：It is based on resulting global TSDF values, three-dimensional
Rebuild the object；Wherein, in the optimization problem, the global TSDF values base of a voxel
It is worth in final local T SDF of the voxel, the final local T SDF value of a voxel is equal to the body
The initial local TSDF values of the corresponding voxel of plain Jing rigid transformations, variable is the global TSDF of voxel
The parameter of value and rigid transformation, cost function are related to following factors：The global TSDF values of voxel with
The quadratic sum of the difference of the initial local TSDF values of the corresponding voxel of voxel Jing rigid transformations.
In one embodiment, the acquisition device 41 includes：Estimation unit, is configured to：Estimate
Count the camera parameter of multiple magazine each camera；Computing unit, is configured to：Based on being estimated
The camera parameter of meter, calculates one or more initial locals TSDF values of voxel.
In one embodiment, the computing unit is further configured to：Based on estimated by camera
Camera parameter, calculate voxel relative to the camera the first depth value；Determine the voxel it is corresponding,
The second depth value in the depth map of the camera association；By the first depth value and the difference of the second depth value
Result after blocking, as voxel initial local TSDF values corresponding with the camera.
In one embodiment, the acquisition device 41 also includes：Normalization unit, is configured to：
By the initial local TSDF value normalization of calculated voxel.
In one embodiment, the absolute value of the initial local TSDF values of voxel is less than or equal to specific
Threshold value, the specific threshold belong to (0,1).
In one embodiment, the initial local TSDF values of calculated voxel are corresponding to described more
The coordinate system of individual magazine camera association；The acquisition device 41 also includes：Coordinate system turns
Unit is changed, is configured to：The initial local TSDF values of calculated voxel are transformed into into same seat
In mark system.
In one embodiment, the coordinate system converting unit is further configured to：To be calculated
The initial local TSDF values of voxel be transformed in the coordinate system of first camera association.
In one embodiment, in the optimization problem, the global TSDF values of voxel it is initial
Weighted mean of the value equal to whole initial local TSDF values of the voxel；The parameter of rigid transformation
It is the voxel itself that initial value causes the corresponding voxel of voxel Jing rigid transformations.
In one embodiment, the parameter of the rigid transformation includes unit matrix and null vector.
In one embodiment, the solving device 42 is further configured to：By the height of iteration
This Newton method solves the optimization problem or by the global TSDF values for voxel and rigidity
The parameter of conversion is calculated respectively carrys out optimization problem described in iterative.
In one embodiment, the reconstructing device 43 is further configured to：Using mobile cube
Body algorithm, based on resulting global TSDF values, the surface of object described in three-dimensional reconstruction.
In one embodiment, rigid transformation is only related to rotate and is translated.
Due to included each device in three-dimensional reconstruction object-based device of the invention 400, list
In unit process respectively with each step included in three-dimensional reconstruction object method described above in
Process be similar to, therefore for simplicity, here omits the detailed description of these devices, unit.
Additionally, being still needed, it is noted that each component devices, unit can pass through in the said equipment here
The mode of software, firmware, hardware or its combination is configured.Specific means or side that configuration can be used
Formula is well known to those skilled in the art, and will not be described here.In the feelings realized by software or firmware
Under condition, from storage medium or network to the computer with specialized hardware structure (such as shown in Fig. 5
General purpose computer 500) install constitute the software program, the computer when various programs are provided with,
It is able to carry out various functions etc..
Fig. 5 shows the computer that can be used to implementing method and apparatus according to an embodiment of the invention
Schematic block diagram.
In Figure 5, CPU (CPU) 501 is stored according in read only memory (ROM) 502
Program or from storage part 508 be loaded into random access memory (RAM) 503 program performing it is each
Plant and process.In RAM 503, store when CPU 501 performs various process etc. always according to needs
Required data.CPU 501, ROM 502 and RAM 503 are connected to each other via bus 504.It is defeated
Enter/output interface 505 is also connected to bus 504.
Components described below is connected to input/output interface 505：Importation 506 (includes keyboard, mouse
Etc.), output par, c 507 (include display, such as cathode ray tube (CRT), liquid crystal display
(LCD) etc., and speaker etc.), storage part 508 (including hard disk etc.), communications portion 509 (wrap
Include NIC such as LAN card, modem etc.).Communications portion 509 via network such as
The Internet performs communication process.As needed, driver 510 can be connected to input/output interface
505.Detachable media 511 such as disk, CD, magneto-optic disk, semiconductor memory etc. can be with
Be installed in driver 510 as needed so that the computer program for reading out as needed by
It is installed in storage part 508.
In the case where above-mentioned series of processes is realized by software, it is situated between from network such as the Internet or storage
Matter such as detachable media 511 installs the program for constituting software.
It will be understood by those of skill in the art that this storage medium is not limited to shown in Fig. 5 wherein
Have program stored therein, and equipment separately distribute to provide a user with the detachable media 511 of program.
The example of detachable media 511 (includes CD comprising disk (including floppy disk (registered trade mark)), CD
Read only memory (CD-ROM) and digital universal disc (DVD)), magneto-optic disk (comprising mini-disk (MD) (note
Volume trade mark)) and semiconductor memory.Or, storage medium can be ROM 502, storage part 508
In the hard disk that includes etc., wherein computer program stored, and be distributed to together with comprising their equipment
User.
The present invention also proposes a kind of program product of the instruction code of the machine-readable that is stored with.The finger
When making code be read and performed by machine, above-mentioned method according to an embodiment of the invention is can perform.
Correspondingly, for carrying depositing for the program product of the instruction code of the above-mentioned machine-readable that is stored with
Storage media is also included within disclosure of the invention.The storage medium include but is not limited to floppy disk, CD,
Magneto-optic disk, storage card, memory stick etc..
In description above to the specific embodiment of the invention, for a kind of embodiment describe and/or
The feature for illustrating can be made in one or more other embodiments in same or similar mode
With, it is combined with the feature in other embodiment, or substitute the feature in other embodiment.
It should be emphasized that term "comprises/comprising" refer to when using herein feature, key element, step or
The presence of component, but it is not precluded from depositing for one or more further features, key element, step or component
Or it is additional.
Additionally, the method for the present invention be not limited to specifications described in time sequencing performing,
Can according to other time sequencings ground, concurrently or independently perform.Therefore, retouch in this specification
The execution sequence of the method stated is not construed as limiting to the technical scope of the present invention.
Although being draped over one's shoulders to the present invention by the description of the specific embodiment to the present invention above
Dew, however, it is to be understood that above-mentioned all embodiments and example are exemplary, and it is unrestricted
Property.Those skilled in the art can be designed in the spirit and scope of the appended claims to the present invention
Various modifications, improvement or equivalent.These modifications, improvement or equivalent should also be as being considered as
Including within the scope of the present invention.
Note
1. a kind of three-dimensional rebuilding method of object, including：
The initial local for obtaining voxel in three dimensions blocks signed distance function TSDF values；
Optimization problem is solved, to obtain an overall situation TSDF value of voxel；And
Based on resulting global TSDF values, object described in three-dimensional reconstruction；
Wherein, in the optimization problem, the global TSDF values of a voxel are based on the voxel
Final local T SDF is worth to, and the final local T SDF value of a voxel is equal to voxel Jing rigidity
The initial local TSDF values of corresponding voxel are converted, variable is the global TSDF values of voxel and rigidity
The parameter of conversion, cost function are related to following factors：The global TSDF values of voxel and voxel Jing
The quadratic sum of the difference of the initial local TSDF values of the corresponding voxel of rigid transformation.
2. note 1 as described in method, wherein, obtain three dimensions in voxel initial local cut
Disconnected signed distance function TSDF values include：
Estimate the camera parameter of multiple magazine each camera；
Based on estimated camera parameter, one or more initial locals TSDF values of voxel are calculated.
3. the method as described in note 2, wherein, based on estimated camera parameter, calculate voxel
One or more initial locals TSDF values include：
Based on the camera parameter estimated by camera, first depth of the voxel relative to the camera is calculated
Value；
Determine that the voxel is corresponding, the second depth value in camera association depth map；
Result after the difference of the first depth value and the second depth value is blocked, as the voxel and this
The corresponding initial local TSDF values of camera.
4. note 2 as described in method, wherein, obtain three dimensions in voxel initial local cut
Disconnected signed distance function TSDF values also include：
By the initial local TSDF value normalization of calculated voxel.
5. note 4 as described in method, wherein, the absolute value of the initial local TSDF values of voxel
Less than or equal to specific threshold, the specific threshold belong to (0,1).
6. note 2 as described in method, wherein, the initial local TSDF of calculated voxel
It is worth the coordinate system corresponding to the plurality of magazine camera association；
In acquisition three dimensions, the initial local of voxel blocks signed distance function TSDF values also includes：
The initial local TSDF values of calculated voxel are transformed in the same coordinate system.
7. note 6 as described in method, wherein, by the initial local TSDF of calculated voxel
Value is transformed into the same coordinate system to be included：The initial local TSDF values of calculated voxel are changed
To in the coordinate system of first camera association.
8. note 1 as described in method, wherein, in the optimization problem, the overall situation of voxel
The initial value of TSDF values is equal to the weighted mean of whole initial local TSDF values of the voxel；Just
Property conversion the initial value of parameter to cause the corresponding voxel of voxel Jing rigid transformations be the voxel itself.
9. the method as described in note 1, wherein, the optimization problem is by the Gauss cattle of iteration
Method of pausing is solved or is counted by the parameter of global TSDF values and rigid transformation for voxel respectively
Calculate and carry out iterative.
10. the method as described in note 1, wherein, based on resulting global TSDF values, three-dimensional
Rebuilding the object includes：
It is using marching cubes algorithm, based on resulting global TSDF values, right described in three-dimensional reconstruction
The surface of elephant.
A kind of three-dimensional reconstruction equipment of 11. objects, including：
Device is obtained, is configured to：The initial local for obtaining voxel in three dimensions blocks directed distance
Function TSDF values；
Solving device, is configured to：Optimization problem is solved, to obtain an overall situation TSDF of voxel
Value；And
Reconstructing device, is configured to：It is based on resulting global TSDF values, right described in three-dimensional reconstruction
As；
Wherein, in the optimization problem, the global TSDF values of a voxel are based on the voxel
Final local T SDF is worth to, and the final local T SDF value of a voxel is equal to voxel Jing rigidity
The initial local TSDF values of corresponding voxel are converted, variable is the global TSDF values of voxel and rigidity
The parameter of conversion, cost function are related to following factors：The global TSDF values of voxel and voxel Jing
The quadratic sum of the difference of the initial local TSDF values of the corresponding voxel of rigid transformation.
12. equipment as described in note 11, wherein, the acquisition device includes：
Estimation unit, is configured to：Estimate the camera parameter of multiple magazine each camera；
Computing unit, is configured to：Based on estimated camera parameter, one or many of voxel is calculated
Individual initial local TSDF values.
13. equipment as described in note 12, wherein, the computing unit is further configured to：
Based on the camera parameter estimated by camera, first depth of the voxel relative to the camera is calculated
Value；
Determine that the voxel is corresponding, the second depth value in camera association depth map；
Result after the difference of the first depth value and the second depth value is blocked, as the voxel and this
The corresponding initial local TSDF values of camera.
14. equipment as described in note 12, wherein, the acquisition device also includes：Normalization list
Unit, is configured to：By the initial local TSDF value normalization of calculated voxel.
15. note 14 as described in equipment, wherein, the initial local TSDF values of voxel it is absolute
Value less than or equal to specific threshold, the specific threshold belong to (0,1).
16. equipment as described in note 12, wherein, the initial local TSDF of calculated voxel
It is worth the coordinate system corresponding to the plurality of magazine camera association；
The acquisition device also includes：Coordinate system converting unit, is configured to：By calculated body
The initial local TSDF values of element are transformed in the same coordinate system.
17. equipment as described in note 16, wherein, the coordinate system converting unit is further matched somebody with somebody
It is set to：The initial local TSDF values of calculated voxel are transformed into into the seat of first camera association
In mark system.
18. note 11 as described in equipment, wherein, in the optimization problem, voxel it is complete
The initial value of office's TSDF values is equal to the weighted mean of whole initial local TSDF values of the voxel；
It is the voxel sheet that the initial value of the parameter of rigid transformation causes the corresponding voxel of voxel Jing rigid transformations
Body.
19. equipment as described in note 11, wherein, the solving device is further configured to：
The optimization problem is solved or by the overall situation for voxel by the Gauss-Newton method of iteration
The parameter of TSDF values and rigid transformation is calculated respectively carrys out optimization problem described in iterative.
20. equipment as described in note 11, wherein, the reconstructing device is further configured to：
It is using marching cubes algorithm, based on resulting global TSDF values, right described in three-dimensional reconstruction
The surface of elephant.
Claims (10)
1. a kind of three-dimensional rebuilding method of object, including：
The initial local for obtaining voxel in three dimensions blocks signed distance function TSDF values；
Optimization problem is solved, to obtain an overall situation TSDF value of voxel；And
Based on resulting global TSDF values, object described in three-dimensional reconstruction；
Wherein, in the optimization problem, the global TSDF values of a voxel are based on the voxel
Final local T SDF is worth to, and the final local T SDF value of a voxel is equal to voxel Jing rigidity
The initial local TSDF values of corresponding voxel are converted, variable is the global TSDF values of voxel and rigidity
The parameter of conversion, cost function are related to following factors：The global TSDF values of voxel and voxel Jing
The quadratic sum of the difference of the initial local TSDF values of the corresponding voxel of rigid transformation.
2. the method for claim 1, wherein the initial office of voxel in three dimensions is obtained
Signed distance function TSDF values are blocked in portion to be included：
Estimate the camera parameter of multiple magazine each camera；
Based on estimated camera parameter, one or more initial locals TSDF values of voxel are calculated.
3. method as claimed in claim 2, wherein, based on estimated camera parameter, calculate
One or more initial locals TSDF values of voxel include：
Based on the camera parameter estimated by camera, first depth of the voxel relative to the camera is calculated
Value；
Determine that the voxel is corresponding, the second depth value in camera association depth map；
Result after the difference of the first depth value and the second depth value is blocked, as the voxel and this
The corresponding initial local TSDF values of camera.
4. method as claimed in claim 2, wherein, obtain the initial office of voxel in three dimensions
Signed distance function TSDF values are blocked in portion also to be included：
By the initial local TSDF value normalization of calculated voxel.
5. method as claimed in claim 4, wherein, the initial local TSDF values of voxel it is exhausted
To value less than or equal to specific threshold, the specific threshold belong to (0,1).
6. method as claimed in claim 2, wherein, the initial local of calculated voxel
Coordinate system of the TSDF values corresponding to the plurality of magazine camera association；
In acquisition three dimensions, the initial local of voxel blocks signed distance function TSDF values also includes：
The initial local TSDF values of calculated voxel are transformed in the same coordinate system.
7. method as claimed in claim 6, wherein, by the initial local of calculated voxel
TSDF values are transformed into the same coordinate system to be included：By the initial local TSDF of calculated voxel
Value is transformed in the coordinate system of first camera association.
8. the method for claim 1, wherein in the optimization problem, voxel
The initial value of global TSDF values is equal to the weighted average of whole initial local TSDF values of the voxel
Value；It is the voxel that the initial value of the parameter of rigid transformation causes the corresponding voxel of voxel Jing rigid transformations
Itself.
9. the method for claim 1, wherein height of the optimization problem by iteration
This Newton method is solved or by the parameter point of global TSDF values and rigid transformation for voxel
Iterative Ji Suan not carried out.
10. the three-dimensional reconstruction equipment of a kind of object, including：
Device is obtained, is configured to：The initial local for obtaining voxel in three dimensions blocks directed distance
Function TSDF values；
Solving device, is configured to：Optimization problem is solved, to obtain an overall situation TSDF of voxel
Value；And
Reconstructing device, is configured to：It is based on resulting global TSDF values, right described in three-dimensional reconstruction
As；
Wherein, in the optimization problem, the global TSDF values of a voxel are based on the voxel
Final local T SDF is worth to, and the final local T SDF value of a voxel is equal to voxel Jing rigidity
The initial local TSDF values of corresponding voxel are converted, variable is the global TSDF values of voxel and rigidity
The parameter of conversion, cost function are related to following factors：The global TSDF values of voxel and voxel Jing
The quadratic sum of the difference of the initial local TSDF values of the corresponding voxel of rigid transformation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510591091.5A CN106548466B (en) | 2015-09-16 | 2015-09-16 | The method and apparatus of three-dimensional reconstruction object |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510591091.5A CN106548466B (en) | 2015-09-16 | 2015-09-16 | The method and apparatus of three-dimensional reconstruction object |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN106548466A true CN106548466A (en) | 2017-03-29 |
| CN106548466B CN106548466B (en) | 2019-03-29 |
Family
ID=58361948
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510591091.5A Active CN106548466B (en) | 2015-09-16 | 2015-09-16 | The method and apparatus of three-dimensional reconstruction object |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106548466B (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108986155A (en) * | 2017-06-05 | 2018-12-11 | 富士通株式会社 | The depth estimation method and estimation of Depth equipment of multi-view image |
| CN114851201A (en) * | 2022-05-18 | 2022-08-05 | 浙江工业大学 | Mechanical arm six-degree-of-freedom vision closed-loop grabbing method based on TSDF three-dimensional reconstruction |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103456038A (en) * | 2013-08-19 | 2013-12-18 | 华中科技大学 | Method for rebuilding three-dimensional scene of downhole environment |
| CN103500013A (en) * | 2013-10-18 | 2014-01-08 | 武汉大学 | Real-time three-dimensional mapping system and method based on Kinect and streaming media technology |
| US20140146057A1 (en) * | 2012-11-26 | 2014-05-29 | Electronics And Telecommunications Research Institute | Apparatus for 3d reconstruction based on multiple gpus and method thereof |
| WO2015006791A1 (en) * | 2013-07-18 | 2015-01-22 | A.Tron3D Gmbh | Combining depth-maps from different acquisition methods |
| CN104616345A (en) * | 2014-12-12 | 2015-05-13 | 浙江大学 | Octree forest compression based three-dimensional voxel access method |
| EP2886043A1 (en) * | 2013-12-23 | 2015-06-24 | a.tron3d GmbH | Method for continuing recordings to detect three-dimensional geometries of objects |
-
2015
- 2015-09-16 CN CN201510591091.5A patent/CN106548466B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140146057A1 (en) * | 2012-11-26 | 2014-05-29 | Electronics And Telecommunications Research Institute | Apparatus for 3d reconstruction based on multiple gpus and method thereof |
| WO2015006791A1 (en) * | 2013-07-18 | 2015-01-22 | A.Tron3D Gmbh | Combining depth-maps from different acquisition methods |
| CN103456038A (en) * | 2013-08-19 | 2013-12-18 | 华中科技大学 | Method for rebuilding three-dimensional scene of downhole environment |
| CN103500013A (en) * | 2013-10-18 | 2014-01-08 | 武汉大学 | Real-time three-dimensional mapping system and method based on Kinect and streaming media technology |
| EP2886043A1 (en) * | 2013-12-23 | 2015-06-24 | a.tron3d GmbH | Method for continuing recordings to detect three-dimensional geometries of objects |
| CN104616345A (en) * | 2014-12-12 | 2015-05-13 | 浙江大学 | Octree forest compression based three-dimensional voxel access method |
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108986155A (en) * | 2017-06-05 | 2018-12-11 | 富士通株式会社 | The depth estimation method and estimation of Depth equipment of multi-view image |
| CN108986155B (en) * | 2017-06-05 | 2021-12-07 | 富士通株式会社 | Depth estimation method and depth estimation apparatus for multi-viewpoint image |
| CN114851201A (en) * | 2022-05-18 | 2022-08-05 | 浙江工业大学 | Mechanical arm six-degree-of-freedom vision closed-loop grabbing method based on TSDF three-dimensional reconstruction |
| CN114851201B (en) * | 2022-05-18 | 2023-09-05 | 浙江工业大学 | Mechanical arm six-degree-of-freedom visual closed-loop grabbing method based on TSDF three-dimensional reconstruction |
Also Published As
| Publication number | Publication date |
|---|---|
| CN106548466B (en) | 2019-03-29 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108345890B (en) | Image processing method, device and related equipment | |
| US10547871B2 (en) | Edge-aware spatio-temporal filtering and optical flow estimation in real time | |
| US10410362B2 (en) | Method, device, and non-transitory computer readable storage medium for image processing | |
| Sorzano et al. | Elastic registration of biological images using vector-spline regularization | |
| US11656845B2 (en) | Dot product calculators and methods of operating the same | |
| US20050249398A1 (en) | Rapid and robust 3D/3D registration technique | |
| CN107679537A (en) | A kind of texture-free spatial target posture algorithm for estimating based on profile point ORB characteristic matchings | |
| CN113450396B (en) | Three-dimensional/two-dimensional image registration method and device based on bone characteristics | |
| Chandraker et al. | Globally optimal algorithms for stratified autocalibration | |
| US10025754B2 (en) | Linear FE system solver with dynamic multi-grip precision | |
| CN106548507A (en) | The method and apparatus of three-dimensional reconstruction object | |
| US20210264659A1 (en) | Learning hybrid (surface-based and volume-based) shape representation | |
| CN115546371A (en) | Point cloud optimization method and system, electronic device and storage medium | |
| CN106548466A (en) | The method and apparatus of three-dimensional reconstruction object | |
| Liu et al. | Image inpainting algorithm based on tensor decomposition and weighted nuclear norm | |
| Yan et al. | A high accuracy method for pose estimation based on rotation parameters | |
| CN114202632A (en) | Grid linear structure recovery method and device, electronic equipment and storage medium | |
| CN110706332B (en) | Scene reconstruction method based on noise point cloud | |
| CN117132737A (en) | Three-dimensional building model construction method, system and equipment | |
| CN113875228B (en) | Video frame inserting method and device and computer readable storage medium | |
| CN106558076A (en) | The method and apparatus of three-dimensional reconstruction object | |
| Xiao et al. | Spherical framelets from spherical designs | |
| Ying et al. | Accurate stereo image super-resolution using spatial-attention-enhance residual network | |
| CN109559271B (en) | Method and device for optimizing depth image | |
| CN116310145B (en) | Three-dimensional space model reconstruction method and device based on orthogonal basis functions |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |