The method and system of automatic Reconstruction bidimensional image and threedimensional model accurate relative location
Technical field
The present invention relates to texture mapping fields, are specifically related to automatic Reconstruction bidimensional image with threedimensional model accurately with respect to position
The method and system set.
Background technology
As the rapid development and computer graphics techniques of computer technology are in game, video display, analog simulation, virtual trip
The increasingly extensive application in the fields such as trip, cultural relic digitalization, the sense of reality of graphical display, which becomes in computer graphics, most to induce one
That gazes at studies a question.However to generate an amplitude ratio more actually figure needs to solve the problems, such as various, for example image paints
System, Lightness disposal, anti-aliasing, ray trace etc., so as to cause lower computational efficiency.In order to improve the generation effect of figure
Rate can generally describe complicated object with simple geometrical model, and in order to ensure the sense of reality of generation figure, texture mapping is just
As essential important method in computer graphics.
The process of texture mapping is according to the relative position relation between bidimensional image and threedimensional model, and structure mapping is calculated
Method, by the surface of bidimensional image Texture mapping to three-dimensional object.Therefore, how bidimensional image and three fast and accurately to be rebuild
The relative position relation of dimension module becomes the emphasis of texture mapping.
Presently, there are method for reconstructing have:
1) parameter of camera is resolved by several control points 2D-3D to texture image and three-dimensional surface.Largely grind
Study carefully and shows extraction and matching operational difficulties currently, automatic 2D-3D Control points, it is arbitrary there are no can reliably adapt to
The automation algorithm of practical application situation, therefore, generally use man-machine interactively formula chooses 2D-3D and corresponds to control in practical applications
It is low to rebuild efficiency for point.
2) by manually matching the characteristics of image that silhouette etc. is larger in bidimensional image texture and 3-D geometric model
Realize that 2D-3D is matched, line feature is stablized than point feature as registration primitive many, and such methods are for abundant pair of details
As there is preferable registration effect, but the cumbersome of silhouette is manually found, it is low to rebuild efficiency.
Invention content
Technical problem to be solved by the invention is to provide a kind of automatic Reconstruction bidimensional image and threedimensional model are accurately opposite
The method of position can choose control point or the artificial tedious work for finding silhouette to avoid artificial, automatically find
Accurate relative position improves the efficiency of reconstruction process.
The technical solution that the present invention solves above-mentioned technical problem is as follows:Automatic Reconstruction bidimensional image and the accurate phase of threedimensional model
To the method for position, include the following steps:
The initial relative position of bidimensional image and threedimensional model is arranged in S1, field angle when according to shooting bidimensional image;
S2, after threedimensional model is projected on bidimensional image according to the initial relative position of bidimensional image and threedimensional model,
Obtain the seed point of foreground and background in bidimensional image;
S3 will be before in bidimensional image using image segmentation algorithm by the seed point of foreground in bidimensional image and background
Scape and background separation, obtain the front and back scape binary map of bidimensional image;
S4 adjusts the position of threedimensional model, and the threedimensional model after adjustment is projected on bidimensional image, obtains three-dimensional mould
The projection binary map of type;
S5, as an optimization with the exclusive or accumulated value of the projection binary map of threedimensional model and the front and back scape binary map of bidimensional image
Condition optimizes using simplex method and seeks the relative position of bidimensional image and threedimensional model.
The beneficial effects of the invention are as follows:The side of the automatic Reconstruction bidimensional image and threedimensional model accurate relative location of the present invention
Method is the initial position relative to bidimensional image according to given threedimensional model, the seed point of bidimensional image segmentation is chosen, by two
Model part in dimension image comes with background separation, and the position by adjusting model in three dimensions relative to image utilizes
Simplex method seeks accurate relative position;The present invention takes full advantage of 3 dimensional coil geometry information and 2 d texture information, base
In image Segmentation Technology and simplex algorithm, the work of man-machine interactively, automatic Reconstruction bidimensional image and threedimensional model are reduced to the greatest extent
Accurate relative location, reduce the workload of manual intervention.
Based on the above technical solution, the present invention can also be improved as follows.
Further, step S1 is specially:
S11 calculates field angle when shooting bidimensional image according to the EXIF information of bidimensional image;
Then threedimensional model and bidimensional image are included in the same three dimensions, according to the shooting being calculated by S12
Three dimensions field angle size is arranged in field angle when bidimensional image, and threedimensional model phase is adjusted according to three dimensions field angle size
For an initial relative position on threedimensional model boundary in bidimensional image.
Further, step S2 is specially:Threedimensional model is thrown according to the initial relative position of bidimensional image and threedimensional model
It penetrates in bidimensional image plane, obtains binaryzation image, and corrosion and expansion process are carried out to binaryzation image, obtain two-dimentional shadow
The seed point of foreground and background as in;
Step S3 is specially:Using GraphCuts algorithms, using foreground in bidimensional image and the seed point of background as
The input of GraphCuts algorithms, the output of GraphCuts algorithms are the front and back scape binary map of bidimensional image.
Further, step S4 is specially:
S41 determines that six adjusting parameters of the relative position of threedimensional model and bidimensional image, initialization threedimensional model are opposite
Seven initial solutions of bidimensional image position, wherein each initial solution includes six adjusting parameters, initialization threedimensional model is with respect to two
Seven initial solutions of dimension image position correspond to the condition of convergence of error, initialize seven of threedimensional model relative two dimensional image position
The flare factor and constriction coefficient of initial solution;
S42 adjusts the position of threedimensional model according to six adjusting parameters in each initial solution, and will be after each adjustment
Threedimensional model projects on bidimensional image respectively, obtains seven projection binary maps of threedimensional model.
Further, step S5 is specially:
S51 obtains the different of seven of threedimensional model projection binary maps and the front and back scape binary map of the bidimensional image
Or accumulated value, and using the exclusive or accumulated value as the mistake corresponding to seven initial solutions of threedimensional model relative two dimensional image position
Difference, and error sequence is carried out to the error of seven initial solutions;
S52 calculates the iteration error for working as the error corresponding to the first seven initial solution, and judges whether iteration error meets just
The condition of convergence of beginningization, if conditions are not met, S53 is thened follow the steps, if it is satisfied, then executing step S54;
S53, the size of error or the flare factor of initialization or constriction coefficient corresponding to seven initial solutions calculate
The pip or compression point or inflexion point of threedimensional model relative two dimensional image position, are used in combination pip or compression point or inflexion point phase
The maximum initial solution of error in former seven initial solutions is substituted to the initial solution of bidimensional image position, obtains replaced 7 initially
Solution, and it is back to step S42;
S54 obtains the initial solution of error minimum in the error corresponding to seven initial solutions in S51, according to error minimum
Initial solution in six adjusting parameters adjustment threedimensional model spatial position, the opposite position of threedimensional model and bidimensional image at this time
Set the relative position relation that relationship is bidimensional image and threedimensional model when taking pictures.
Based on the method for above-mentioned automatic Reconstruction bidimensional image and threedimensional model accurate relative location, the present invention also provides one kind
The system of automatic Reconstruction bidimensional image and threedimensional model accurate relative location.
The system of automatic Reconstruction bidimensional image and threedimensional model accurate relative location, the system is according to automatic Reconstruction two dimension shadow
The method of picture and threedimensional model accurate relative location is built, including initial position setup module, seed point acquisition module, splitting die
Block, projection binary map generation module and relative position generation module;
Bidimensional image and three is arranged in the initial position setup module, field angle when being used for according to shooting bidimensional image
The initial position of dimension module;
The seed point acquisition module is used for three-dimensional mould according to the initial relative position of bidimensional image and threedimensional model
After type projects on bidimensional image, the seed point of foreground and background in bidimensional image is obtained;
The separation module is used for the seed point by foreground in bidimensional image and background, utilizes image segmentation algorithm
By in bidimensional image foreground and background separation, obtain the front and back scape binary map of bidimensional image;
The projection binary map generation module, is used to adjust the position of threedimensional model, and by the threedimensional model after adjustment
It projects on bidimensional image, obtains the projection binary map of threedimensional model;
The relative position generation module is used for the front and back scape two with the projection binary map of threedimensional model and bidimensional image
It is worth the exclusive or accumulated value condition as an optimization of figure, optimizes and seek the opposite position of bidimensional image and threedimensional model using simplex method
It sets.
The beneficial effects of the invention are as follows:Automatic Reconstruction bidimensional image and the threedimensional model accurate relative location of the present invention are
System is the initial position relative to bidimensional image according to given threedimensional model, the seed point of bidimensional image segmentation is chosen, by two
Model part in dimension image comes with background separation, and the position by adjusting model in three dimensions relative to image utilizes
Simplex method seeks accurate relative position;The present invention takes full advantage of 3 dimensional coil geometry information and 2 d texture information, base
In image Segmentation Technology and simplex algorithm, the work of man-machine interactively, automatic Reconstruction bidimensional image and threedimensional model are reduced to the greatest extent
Accurate relative location, reduce the workload of manual intervention.
Based on the above technical solution, the present invention can also be improved as follows.
Further, the initial position setup module is specially:According to the EXIF information of bidimensional image, shooting two dimension is calculated
Field angle when image;Include in the same three dimensions, according to the shooting being calculated by threedimensional model and bidimensional image
Three dimensions field angle size is arranged in field angle when bidimensional image, and adjustment threedimensional model is relative to threedimensional model in bidimensional image
One initial relative position on boundary.
Further, the seed point acquisition module is specially:According to the initial relative position of bidimensional image and threedimensional model
Threedimensional model is incident upon in bidimensional image plane, obtains binaryzation image, and carry out at corrosion and expansion to binaryzation image
Reason obtains the seed point of foreground and background in bidimensional image;
The separation module is specially:Using GraphCuts algorithms, foreground in bidimensional image and the seed point of background are made
For the input of GraphCuts algorithms, the output of GraphCuts algorithms is the front and back scape binary map of bidimensional image.
Further, the projection binary map generation module is specially:
It determines six adjusting parameters of the relative position of threedimensional model and bidimensional image, initializes threedimensional model relative two dimensional
Seven initial solutions of image position, wherein each initial solution include six adjusting parameters, initialization the condition of convergence, flare factor and
Constriction coefficient;
Adjust the position of threedimensional model according to six adjusting parameters in each initial solution, and by the three-dimensional after each adjustment
Model projects on bidimensional image respectively, obtains seven projection binary maps of threedimensional model.
Further, the relative position generation module is specially:
The exclusive or accumulated value of seven projection binary maps of threedimensional model and the front and back scape binary map of bidimensional image is obtained, and will
Error corresponding to seven initial solutions of the exclusive or accumulated value as threedimensional model relative two dimensional image position, and at the beginning of seven
The error for the solution that begins carries out error sequence;
The iteration error for working as the error corresponding to the first seven initial solution is calculated, and judges whether iteration error meets convergence item
Part;
If iteration error is unsatisfactory for the condition of convergence, the size or flare factor of the error corresponding to seven initial solutions
Or the pip or compression point or inflexion point of constriction coefficient calculating threedimensional model relative two dimensional image position, pip or pressure is used in combination
Point reduction or the initial solution of inflexion point relative two dimensional image position substitute the maximum initial solution of error in former seven initial solutions, are replaced
7 initial solutions after changing, and be back in projection binary map generation module;
If iteration error meets the condition of convergence, minimum initial of error in the error corresponding to seven initial solutions is obtained
Solution, according in the initial solution of error minimum six adjusting parameters adjust threedimensional model spatial position, at this time threedimensional model and
The relative position relation of bidimensional image is the relative position relation of bidimensional image and threedimensional model when taking pictures.
Description of the drawings
Fig. 1 is the overall flow figure of automatic Reconstruction bidimensional image of the present invention and the method for threedimensional model accurate relative location;
Fig. 2 is the particular flow sheet of automatic Reconstruction bidimensional image of the present invention and the method for threedimensional model accurate relative location;
Fig. 3 is that automatic Reconstruction bidimensional image of the present invention and threedimensional model in the method for threedimensional model accurate relative location are initial
Position view;
Fig. 4 is that automatic Reconstruction bidimensional image of the present invention is divided with bidimensional image in the method for threedimensional model accurate relative location
The schematic diagram of seed point;
Fig. 5 is that automatic Reconstruction bidimensional image of the present invention is divided with bidimensional image in the method for threedimensional model accurate relative location
Front and back scape binary map schematic diagram afterwards;
Fig. 6 is that automatic Reconstruction bidimensional image of the present invention is projected with threedimensional model in the method for threedimensional model accurate relative location
Image schematic diagram;
Fig. 7 is that automatic Reconstruction bidimensional image of the present invention is adjusted with simplex method in the method for threedimensional model accurate relative location
Error schematic diagram at front border;
Fig. 8 is that automatic Reconstruction bidimensional image of the present invention is adjusted with simplex method in the method for threedimensional model accurate relative location
Error schematic diagram at back boundary;
Fig. 9 is that automatic Reconstruction bidimensional image of the present invention is adjusted with simplex method in the method for threedimensional model accurate relative location
Effect diagram afterwards;
Figure 10 is the structure diagram of automatic Reconstruction bidimensional image of the present invention and the system of threedimensional model accurate relative location.
Specific implementation mode
The principle and features of the present invention will be described below with reference to the accompanying drawings, and the given examples are served only to explain the present invention, and
It is non-to be used to limit the scope of the present invention.
As shown in Figure 1, the method for automatic Reconstruction bidimensional image and threedimensional model accurate relative location, includes the following steps:
The initial relative position of bidimensional image and threedimensional model is arranged in S1, field angle when according to shooting bidimensional image;
S2, after threedimensional model is projected on bidimensional image according to the initial relative position of bidimensional image and threedimensional model,
Obtain the seed point of foreground and background in bidimensional image;
S3 will be before in bidimensional image using image segmentation algorithm by the seed point of foreground in bidimensional image and background
Scape and background separation, obtain the front and back scape binary map of bidimensional image;
S4 adjusts the position of threedimensional model, and the threedimensional model after adjustment is projected on bidimensional image, obtains three-dimensional mould
The projection binary map of type;
S5, as an optimization with the exclusive or accumulated value of the projection binary map of threedimensional model and the front and back scape binary map of bidimensional image
Condition optimizes using simplex method and seeks the relative position of bidimensional image and threedimensional model.
Wherein:Foreground in bidimensional image is the three-dimensional in bidimensional image after threedimensional model projects on bidimensional image
Model part;Background in bidimensional image is the background of bidimensional image, i.e. blank parts.
Fig. 2 is the particular flow sheet of automatic Reconstruction bidimensional image of the present invention and the method for threedimensional model accurate relative location.
In step sl:S11 calculates field angle when shooting bidimensional image according to the EXIF information of bidimensional image;S12,
Include visual field when according to the shooting bidimensional image being calculated in the same three dimensions by threedimensional model and bidimensional image
Three dimensions field angle size is arranged in angle, and threedimensional model is adjusted relative in bidimensional image three according to three dimensions field angle size
One initial relative position on dimension module boundary.Threedimensional model relative to model boundary in bidimensional image a rough position such as
Shown in Fig. 3.
Wherein:The EXIF information of bidimensional image contains the various metadata of clapped image:Such as focal length, camera brand, phase
The information such as type number;Calculate field angle method be:It builds between camera brand, camera model and camera photosensitive element size
Database calculates the size of field angle when use according to the result of inquiry database and visual field angle formula.
In step s 2, according to the initial relative position of bidimensional image and threedimensional model, threedimensional model is incident upon two dimension
On image plane, obtain binaryzation image, to binaryzation image carry out corrosion and expansion process, obtain bidimensional image in foreground and
The seed point of background, foreground and the seed point of background are as shown in Figure 4 in bidimensional image.
In step s3, the present invention detaches the model part in bidimensional image using GraphCuts algorithms, the algorithm
Several subgraphs are divided the image into according to information such as the Luminance Distribution of image, edges;Utilize the obtained bidimensional images of step S2
The seed point of foreground and background, as the input of GraphCuts algorithms, the foreground and background of the bidimensional image after exporting as separation
Image, indicated to get to the front and back scape binary map of bidimensional image, the front and back scape of the bidimensional image after separation with binary image
Binary map is as shown in Figure 5.
In step s 4:
S41 determines that six adjusting parameters of the relative position of threedimensional model and bidimensional image, initialization threedimensional model are opposite
Seven initial solutions of bidimensional image position, wherein each initial solution includes six adjusting parameters, initialization threedimensional model is with respect to two
Seven initial solutions of dimension image position correspond to the condition of convergence of error, initialize seven of threedimensional model relative two dimensional image position
The flare factor and constriction coefficient of initial solution;
S42 adjusts the position of threedimensional model according to six adjusting parameters in each initial solution, and will be after each adjustment
Threedimensional model projects on bidimensional image, obtains seven projection binary maps of threedimensional model.The projection of threedimensional model after adjustment
Image is as shown in Figure 6.
Wherein:Six adjusting parameters of the relative position of threedimensional model and bidimensional image include three translation parameters and three
Rotation parameter, when threedimensional model and bidimensional image are shown in the same three dimensions, this six parameters just control model
Position relative to image;In seven initial solutions of threedimensional model relative dimensional spatial position, wherein each initial solution represents
One spatial position of threedimensional model, is made of six adjusting parameters;The condition of convergence is the termination item that optimization algorithm stops iteration
Part, flare factor and the compressed coefficient seek inflexion point in subsequent step or compression point uses.
In step s 5:
Step S5 is specially:
S51, before and after the bidimensional image obtained in the seven of the threedimensional model obtained in S42 projection binary maps and S3
The exclusive or accumulated value of scape binary map, and it is initial using the exclusive or accumulated value as seven of threedimensional model relative two dimensional image position
The corresponding error of solution, and error sequence is carried out to the error of seven initial solutions;As shown in fig. 7, before Fig. 7 is simplex method adjustment
The error schematic diagram of boundary;
S52 calculates the iteration error for working as the error corresponding to the first seven initial solution, and judges whether iteration error meets
The condition of convergence initialized in S41, if conditions are not met, S53 is thened follow the steps, if it is satisfied, then executing step S54;
S53, the flare factor or constriction coefficient initialized in the size or S41 of the error corresponding to seven initial solutions
The pip or compression point or inflexion point of calculating threedimensional model relative two dimensional image position, are used in combination pip or compression point or expansion
Point substitutes the maximum initial solution of error in seven initial solutions, obtains replaced 7 initial solutions, and be back to step S42;Such as figure
Shown in 8, Fig. 8 is the error schematic diagram at simplex method adjustment back boundary;
S54 obtains the initial solution of error minimum in the error corresponding to seven initial solutions in S51, according to error minimum
Initial solution in six adjusting parameters adjustment threedimensional model spatial position, the opposite position of threedimensional model and bidimensional image at this time
Set the relative position relation that relationship is bidimensional image and threedimensional model when taking pictures.As shown in figure 9, Fig. 9 is after accurately adjusting
Effect diagram.
Wherein:Three-dimensional can be passed through by calculating the error corresponding to seven initial solutions of threedimensional model relative dimensional spatial position
XOR operation between the projection binary map of model and the front and back scape binary map of bidimensional image is counted;Iteration error is to working as
The total evaluation of error corresponding to the first seven initial solution.
Regarding when the automatic Reconstruction bidimensional image of the present invention and the method for threedimensional model accurate relative location are according to shooting
The initial position of rink corner and threedimensional model given accordingly relative to bidimensional image, chooses the seed point of bidimensional image segmentation, will
Model part in bidimensional image comes with background separation, the position by adjusting model in three dimensions relative to image, profit
Accurate relative position is sought with simplex method;The present invention takes full advantage of 3 dimensional coil geometry information and 2 d texture information,
Based on image Segmentation Technology and simplex algorithm, the work of man-machine interactively, automatic Reconstruction bidimensional image and three-dimensional mould are reduced to the greatest extent
The accurate relative location of type reduces the workload of manual intervention.
Based on the method for above-mentioned automatic Reconstruction bidimensional image and threedimensional model accurate relative location, the present invention also provides one kind
The system of automatic Reconstruction bidimensional image and threedimensional model accurate relative location.
As shown in Figure 10, the system of automatic Reconstruction bidimensional image and threedimensional model accurate relative location, the system is according to certainly
The dynamic method for rebuilding bidimensional image and threedimensional model accurate relative location is built, including initial position setup module, seed point obtain
Modulus block, separation module, projection binary map generation module and relative position generation module;
Bidimensional image and three is arranged in the initial position setup module, field angle when being used for according to shooting bidimensional image
The initial position of dimension module;
The seed point acquisition module is used for three-dimensional mould according to the initial relative position of bidimensional image and threedimensional model
After type projects on bidimensional image, the seed point of foreground and background in bidimensional image is obtained;
The separation module is used for the seed point by foreground in bidimensional image and background, utilizes image segmentation algorithm
By in bidimensional image foreground and background separation, obtain the front and back scape binary map of bidimensional image;
The projection binary map generation module, is used to adjust the position of threedimensional model, and by the threedimensional model after adjustment
It projects on bidimensional image, obtains the projection binary map of threedimensional model;
The relative position generation module is used for the front and back scape two with the projection binary map of threedimensional model and bidimensional image
It is worth the exclusive or accumulated value condition as an optimization of figure, optimizes and seek the opposite position of bidimensional image and threedimensional model using simplex method
It sets.
The initial position setup module is specially:First according to the EXIF information of bidimensional image, the two-dimentional shadow of shooting is calculated
As when field angle;Then include in the same three dimensions, according to the bat being calculated by threedimensional model and bidimensional image
Field angle setting three dimensions field angle size when bidimensional image is taken the photograph, adjustment threedimensional model is relative to three-dimensional mould in bidimensional image
One initial relative position on type boundary.
The seed point acquisition module is specially:According to the initial relative position of bidimensional image and threedimensional model by three-dimensional mould
Type is incident upon in bidimensional image plane, is obtained binaryzation image, and carry out corrosion and expansion process to binaryzation image, is obtained automatically
Take the seed point of foreground and background in bidimensional image.
The separation module is specially:Using GraphCuts algorithms, foreground in bidimensional image and the seed point of background are made
For the input of GraphCuts algorithms, the output of GraphCuts algorithms is the front and back scape binary map of bidimensional image.
The projection binary map generation module is specially:
Determine that six adjusting parameters of the relative position of threedimensional model and bidimensional image, initialization threedimensional model are opposite first
Seven initial solutions of bidimensional image position, wherein each initial solution includes six adjusting parameters, the initialization condition of convergence, expansion system
Number and constriction coefficient;
Then the position of threedimensional model is adjusted according to six adjusting parameters in each initial solution, and will be after each adjustment
Threedimensional model projects on bidimensional image respectively, obtains seven projection binary maps of threedimensional model;
The relative position generation module is specially:
First, the exclusive or for obtaining seven projection binary maps of threedimensional model and the front and back scape binary map of bidimensional image is cumulative
Value, and using the exclusive or accumulated value as the error corresponding to seven initial solutions of threedimensional model relative two dimensional image position, and
Error sequence is carried out to the error of seven initial solutions;
Then, the iteration error for working as the error corresponding to the first seven initial solution is calculated, and judges whether iteration error meets
The condition of convergence;
Then, if iteration error is unsatisfactory for the condition of convergence, the size of the error corresponding to seven initial solutions or expansion
Coefficient or constriction coefficient calculate the pip or compression point or inflexion point of threedimensional model relative two dimensional image position, and reflection is used in combination
Point or compression point or the initial solution of inflexion point relative two dimensional image position substitute the maximum initial solution of error in former seven initial solutions,
Replaced 7 initial solutions are obtained, and are back in projection binary map generation module;
Finally, if iteration error meets the condition of convergence, error minimum in the error corresponding to seven initial solutions is obtained
Initial solution adjusts the spatial position of threedimensional model according to six adjusting parameters in the initial solution of error minimum, at this time three-dimensional mould
The relative position relation of type and bidimensional image is the relative position relation of bidimensional image and threedimensional model when taking pictures.
The automatic Reconstruction bidimensional image of the present invention is according to given three-dimensional with the system of threedimensional model accurate relative location
Initial position of the model relative to bidimensional image chooses the seed point of bidimensional image segmentation, by the model part in bidimensional image
It comes with background separation, the position by adjusting model in three dimensions relative to image is sought accurately using simplex method
Relative position;The present invention takes full advantage of 3 dimensional coil geometry information and 2 d texture information, is based on image Segmentation Technology and list
Pure shape algorithm, reduces the work of man-machine interactively to the greatest extent, and the accurate relative location of automatic Reconstruction bidimensional image and threedimensional model is reduced
The workload of manual intervention.
The foregoing is merely presently preferred embodiments of the present invention, is not intended to limit the invention, it is all the present invention spirit and
Within principle, any modification, equivalent replacement, improvement and so on should all be included in the protection scope of the present invention.