CN103942795B - A kind of structuring synthetic method of image object - Google Patents

Abstract

The invention discloses a structured synthesis method of an image object, comprising the following steps: calibrating camera parameters according to the user's simple interaction, combining camera parameters and image object segmentation information to generate a structured three-dimensional proxy, and using the three-dimensional proxy and contact point information to generate a structured three-dimensional proxy Image components from different sources and different viewpoints are combined and connected into novel image objects, and the resulting image is obtained through intelligent color adjustment; based on consistent segmented image components, statistical learning is performed on image data sets of specific object types to obtain a probabilistic graphical model; through Probabilistic reasoning is performed on the acquired Bayesian graphical model, and component types and styles are sampled to obtain high-probability composition schemes and viewpoint attributes, and the resulting image is generated using a viewpoint-aware image object synthesis method. This method can synthesize a large number of new image objects with photographic quality and rich structural shape changes, and can provide a good foundation and guidance for 3D shape modeling.

Description

A Structured Synthesis Method for Image Objects

technical field

The present invention mainly relates to the field of digital media, in particular to image creation/editing, industrial/art design, virtual object/role creation, three-dimensional modeling and other application fields.

Background technique

The relevant technical background of the present invention is briefly described as follows:

1. Image synthesis and synthesis

The main purpose of image synthesis and synthesis is to create visually plausible new images from multiple image sources.

For image synthesis, the focus is often on novel hybrid methods for seamlessly stitching selected image content together. Early work included multiresolution spline techniques (BURT, P.J., AND ADELSON, E.H. 1983. A multiresolution spline with application to image mosaics. ACM Trans. Graph. 2, 4 (Oct.), 217–236.; OGDEN, J.M. , ADELSON, E.H., BERGEN, J.R., ANDBURT, P.J. 1985. Pyramid-based computer graphics. RCA Engineer 30, 5, 4–15.) and compositing operations (PORTER, T., AND DUFF, T. 1984. Compositing digital images. SIGGRAPH Comput. Graph. 18, 3 (Jan.), 253–259.). Since the emergence of Poisson image editing techniques (P′EREZ, P., GANGNET, M., AND BLAKE, A. 2003. Poisson image editing. ACM Transactions on Graphics (TOG) 22, 3, 313–318.), the synthesis of gradient fields Method (JIA, J., SUN, J., TANG, C.-K., AND SHUM, H.-Y. 2006. Drag-and-drop pasting. In ACM Transactions on Graphics (TOG), vol.25, ACM, 631–637.; FARBMAN, Z., HOFFER, G., LIPMAN, Y., COHEN-OR, D., AND LISCHINSKI, D. 2009. Coordinates for instant image cloning. In ACM Transactions on Graphics (TOG) , vol.28, ACM, 67.; TAO, M.W., JOHNSON, M.K., AND PARIS, S.2010. Error tolerant image compositing. In Computer Vision–ECCV2010. Springer, 31–44.; SUNKAVALLI, K., JOHNSON, M.K. , MATUSIK, W., AND PFISTER, H.2010. Multi-scale image harmonization. ACM Transactions on Graphics (TOG) 29, 4, 125.; SZELISKI, R., UYTTENDAELE, M., AND STEEDLY, D. 2011. Fast poisson blending using multi-splines.In Computational Photography (ICCP), 2011IEEEInternational Conference on, IEEE, 1–8.) became a standard technology for seamless splicing in the early years. Recently, Xue et al. (XUE, S., AGARWALA, A., DORSEY, J., AND RUSHMEIER, H. 2012. Understanding and improving the reality of image composites. ACM Transactions on Graphics (TOG) 31, 4, 84.) Improved visual plausibility of compositing by tweaking the appearance of compositing objects.

On the other hand, image synthesis (DIAKOPOULOS, N., ESSA, I., AND JAIN, R. 2004. Content based image synthesis. In Image and Video Retrieval. Springer, 299–307.; JOHNSON, M., BROSTOW, G.J. , SHOTTON, J., ARANDJELOVIC, O., KWATRA, V., AND CIPOLLA, R. 2006. Semantic photo synthesis. In Computer Graphics Forum, vol. 25, Wiley Online Library, 407–413.; LALONDE, J.-F .,HOIEM,D.,EFROS,A.A.,ROTHER,C.,WINN,J.,AND CRIMINISI,A.2007.Photo clip art.In ACM Transactions on Graphics(TOG),vol.26,ACM,3.) Focuses primarily on the selection and arrangement of visual content. One of the representative works is image collage, which combines multiple images into one image under certain constraints. The pioneer of this kind of work is the interactive digital montage technology (AGARWALA, A., DONTCHEVA, M., AGRAWALA, M., DRUCKER, S., COLBURN, A., CURLESS, B., SALESIN, D., AND COHEN ,M.2004.Interactive digital photomontage.InACM Transactions on Graphics(TOG),vol.23,ACM,294–302.), and many follow-up works emerged one after another, such as digital compilation (ROTHER,C.,KUMAR,S .,KOLMOGOROV,V.,AND BLAKE,A.2005.Digital tapestry[automatic image synthesis].In ComputerVision and Pattern Recognition,2005.IEEE Computer Society Conference on,vol.1,IEEE,589–596.), automatic spelling Paste (ROTHER, C., BORDEAUX, L., HAMADI, Y., ANDBLAKE, A.2006. Autocollage. In ACM Transactions on Graphics (TOG), vol.25, ACM, 847–852.), image collage ( WANG,J.,QUAN,L.,SUN,J.,TANG,X.,ANDSHUM,H.-Y.2006.Picture collage.In Computer Vision and Pattern Recognition,2006IEEE Computer Society Conference on,vol.1,IEEE , 347–354.), Puzzle Collage (GOFERMAN, S., TAL, A., 711 AND ZELNIK-MANOR, L. 2010. Puzzle-like collage. In Computer Graphics Forum, vol. 29, Wiley Online Library, 459– 468.), Sketch2Photo (CHEN, T., CHENG, M.-M., TAN, P., SHAMIR, A., AND HU, S.-M. 2009. Sketch2photo: Internet image montage. ACM Transactions on Graphics28, 5,124:1 –10.), PhotoSketcher (EITZ, M., RICHTER, R., HILDEBRAND, K., BOUBEKEUR, T., AND ALEXA, M. 2011. Photosketcher: interactive sketch-based image synthesis. Computer Graphics and Applications, IEEE31, 6 , 56–66.), Arcimboldo collage (HUANG, H., ZHANG, L., AND ZHANG, H.-C. 2011. Arcimboldo-like collage using internet images. ACM Transactions on Graphics (TOG) 30, 6, 155. ) and the latest circle packing collage (YU,Z.,LU,L.,GUO,Y.,FAN,R.,LIU,M.,AND WANG,W.2013.Content-aware photo collage using circle packing . IEEE Transactions on Visualization and Computer Graphics 99, PrePrints.).

Most of the image synthesis and synthesis algorithms mentioned above implicitly assume that the composite content has the same viewpoint as the source image, so they do not process camera parameter information. In photo clip art technology (LALONDE, J.-F., HOIEM, D., EFROS, A.A., ROTHER, C., WINN, J., ANDCRIMINISI, A.2007.Photo clip art.In ACM Transactions on Graphics (TOG ), vol.26, ACM, 3.), the authors try to infer the camera pose from the height of the object. However, this method cannot handle real 3D relationships, so it is difficult to perform complex rotation transformations. In a recent work, Zheng et al. (ZHENG, Y., CHEN, X., CHENG, M.-M., ZHOU, K., HU, S.-M., ANDMITRA, N.J. 2012. Interactive images: cuboid proxies for smart image manipulation. ACM Trans.Graph.31, 4 (July), 99:1–99:11.) represent image objects as 3D cuboid proxies and explicitly optimize camera and geometry parameters. The method in the present invention also uses 3D proxy representation, but needs to deal with the more challenging spatial relationship and structure between non-cuboid parts.

2. Data-driven 3D model synthesis

Data-driven 3D model synthesis has recently attracted a great deal of research interest in the field of graphics. Its purpose is to automatically synthesize a large number of novel 3D shapes that conform to the internal structural constraints of the input shape collection by combining parts from a batch of input 3D shapes. Data-driven 3D modeling was first proposed by Funkhouser et al. (FUNKHOUSER,T.,KAZHDAN,M.,SHILANE,P.,MIN,P.,KIEFER,W.,TAL,A.,RUSINKIEWICZ,S.,AND DOBKIN , D.2004.Modeling by example.ACM Trans.Graph.23,3(Aug.),652–663.), their example modeling system allows users to search a library of segmented 3D parts and then interactively assemble These parts are used to form new shapes. In follow-up work, some use user-input sketches to search for parts (SHIN, H., AND IGARASHI, T. 2007. Magic canvas: interactive design of a3-d scene prototype from freehand sketches. InGraphics Interface, 63–70. ; LEE, J., AND FUNKHOUSER, T.A. 2008. Sketch-basedsearch and composition of 3d models. In SBM, 97–104.), others enable users to interchange parts within a small set of matching shapes (KREAVOY, V. , JULIUS, D., AND SHEFFER, A. 2007. Model composition from interchangeable components. In Proceedings of the 15th Pacific Conference on Computer Graphics and Applications, IEEE Computer Society, Washington, DC, USA, PG'07, 129–138.). Chaudhuri et al. (CHAUDHURI, S., AND KOLTUN, V.2010. Data-driven suggestions for creativity support in 3dmodeling. ACM Trans. Graph. 29, 6 (Dec.), 183:1–183:10.) proposed a A data-driven approach to recommend suitable parts for incompletely designed shapes, and then design a probabilistic representation of shape structure, which can give more semantically and stylistic part recommendations (CHAUDHURI, S., KALOGERAKIS , E., GUIBAS, L., AND KOLTUN, V. 2011. Probabilistic reasoning for assembly-based 3d modeling. ACM Trans. Graph. 30, 4 (July), 35:1–35:10.). Kalogerakis et al. continued the above method of probabilistic reasoning and used it for the synthesis of complete shapes (KALOGERAKIS, E., CHAUDHURI, S., KOLLER, D., AND KOLTUN, V.2012.A probabilistic model for component-based shape synthesis. ACM Trans. Graph. 31, 4 (July), 55:1–55:11.).

Contents of the invention

The purpose of the present invention is to provide a structured synthesis method of image objects to address the deficiencies of the prior art. Based on a set of image objects of a given type and with different viewing angles, the method synthesizes visually reasonable and realistic image objects by combining image components.

The object of the present invention is achieved through the following technical solutions: a method for structured synthesis of image objects, comprising the following steps:

(1) Preprocessing of image object data: use digital or network equipment to collect image collections of specific types of objects, requiring clear and complete object structures, and using image segmentation and marking tools to obtain consistent segmentation regions of object components;

(2) Viewpoint-aware image object synthesis method: calibrate the camera parameters of a single image according to the user's simple interaction, and combine the camera parameters and image object segmentation information to generate a structured 3D proxy, and then use the 3D proxy and contact point information to convert the image The components are combined and connected to form a novel image object, and finally the resulting image is obtained through intelligent color adjustment;

(3) The training and synthesis method of the Bayesian probabilistic graph model: Based on consistent segmented image components, statistical learning is performed on image datasets of specific object types to obtain a model that can express shape style, object structure, component category and camera parameters A probabilistic graphical model of complex dependencies between them; and by performing probabilistic reasoning on the acquired Bayesian graphical model, sampling component types and styles to obtain the composition scheme and viewpoint attributes of high-probability image objects, and finally through step 2 The method in synthesizes the resulting image;

(4) Export of image object synthesis results: export and store the resulting images obtained in steps 2 and 3, including the camera parameters and 3D proxy data obtained in step 2, in a common format.

The beneficial effect of the invention is that: the viewpoint-aware component-level synthesis is performed on the image object, and the components from different viewpoint images can be connected and synthesized into a novel image object with consistent viewpoint and correct structure. At the same time, the present invention proposes a single-view camera calibration method based on a coordinate frame for the first time, which is suitable for general single-image camera calibration without obvious or complete geometric clues; a structure-aware 3D proxy construction method is proposed, which is applicable to Based on the cuboid proxy construction of the image object component layer; a 3D proxy-guided structural synthesis method for image components is proposed; the application of synthesizing a large number of image objects with rich shapes and styles based on a given sample image object is proposed; A Bayesian probabilistic graphical model integrating image viewpoint information is proposed, which is suitable for representing the viewpoint, structure and shape changes of image object collections. Compared with the existing 3D shape synthesis technology, this method can make full use of the advantages of large amount of existing image data, easy acquisition, and rich color appearance information, and synthesize a large number of new image objects with photographic quality and rich structural shape changes, satisfying many It meets the requirements of image editing related applications, and can provide a good foundation and guidance for 3D shape modeling.

Description of drawings

FIG. 1 is a schematic flow chart of a viewpoint-aware image object synthesis method in the present invention;

Fig. 2 is a schematic diagram of single-view camera calibration and structured 3D proxy construction in the present invention, in which (a) is a schematic diagram of user interaction between an input image and camera calibration, and (b) is an initial 3D view of each part of an object obtained based on camera parameters Schematic diagram of agent, (c) is a schematic diagram of optimization of 3D agent based on object structure constraints, (d) is the result of structural optimization;

Fig. 3 is the key element used in the synthesis process of the image object among the present invention, among the figure, (a) is three-dimensional agent and its connection groove, (b) is the segmentation part of image object, (c) is on the boundary of the image A two-dimensional reference point for image bending deformation, (d) is a two-dimensional contact point for image component connection;

Fig. 4 is a schematic diagram of the component color optimization process of an image object in the present invention;

Fig. 5 is a schematic diagram of the general flow of image object training and synthesis based on the probability graph model in the present invention;

Fig. 6 is the structured synthesis result figure obtained based on the chair image set in the present invention;

Fig. 7 is the result map of the structured synthesis obtained based on the cup image set in the present invention;

Fig. 8 is the structured synthesis result figure obtained based on the table lamp image set in the present invention;

Fig. 9 is a structured synthesis result diagram obtained based on a collection of toy airplane images in the present invention;

Fig. 10 is a structured synthesis result diagram obtained based on a robot image collection in the present invention;

FIG. 11 is a result diagram of user evaluation based on the novelty of the experimentally obtained synthetic image object in the present invention;

Fig. 12 is a comparison diagram of the results of direct synthesis of chairs and synthesis of viewpoint-aware image objects in the present invention;

Fig. 13 is a comparison diagram of the results of the direct synthesis of the toy airplane and the synthesis of viewpoint-aware image objects in the present invention.

detailed description

The core of the present invention is an image object synthesis method based on the image object collection for component-level structure perception and rich shape and style changes. The core method of the present invention is mainly divided into the following four parts: preprocessing of image object data, viewpoint-aware image object synthesis, Bayesian probability map model training and synthesis, and image object synthesis result derivation.

1. Preprocessing of image object data: use digital devices or networks to collect image collections of a certain type of object, requiring clear and complete structure of object parts, and use image segmentation tools to obtain the regions of each part in the image, and make semantic marks at the same time .

1.1 Acquisition of image object collection

This method is applied to ordinary digital images. As input data, this method requires collecting a certain number of images of similar objects, and scaling and cropping them to a uniform size. Since this method is a structural synthesis at the component level, this step requires that the component structures of the collected image objects be clear and complete.

1.2 User Assisted Interaction

Due to the complex morphological features of the content and boundaries of the object parts in the image, it is difficult to automatically identify and segment robustly. Therefore, this method relies on an appropriate amount of user interaction to preprocess the image object set for subsequent steps. By adopting Lazy Snapping technology (LI, Y., SUN, J., TANG, C.-K., AND SHUM, H.-Y. 2004. Lazy snapping. ACM Transactions on Graphics (ToG) 23, 3, 303–308. ) to segment the entire object area, and then use LabelMe technology (RUSSELL, B.C., TORRALBA, A., MURPHY, K.P., AND FREEMAN, W.T.2008.LabelMe: adatabase and web-based tool for image annotation.International journal of computervision77,1-3,157 –173.) Segment and label each part region of an object. For occluded image parts, use PatchMatch technology (BARNES, C., SHECHTMAN, E., FINKELSTEIN, A., AND GOLDMAN, D.2009. PatchMatch: a randomized correspondence algorithm for structural image editing. ACM Transactions on Graphics (TOG) 28,3,24:1–11.) to complete the occluded area.

2. Viewpoint-aware image object synthesis

This image object synthesis method uses a collection of image objects (such as chairs) of the same class as input, analyzes their structure and extracts camera parameters semi-automatically. A 3D cuboid proxy is then fitted according to the underlying structure to represent the image object. The components of image objects can be connected and synthesized into novel and perspective-correct complete image objects under the guidance of 3D agents.

2.1 Image Object Representation

Based on the processing results of steps 1.1 and 1.2, construct a structured representation that characterizes the relationship between semantic components of image objects. Each image object can be expressed as a graph G={V,E}, where V is the set of nodes in the graph, and E is the set of edges in the graph. Each component C _i is a node in V. When two nodes C _i and C _j are connected, there is an edge e _ij in E. Among them, C _i ＝{P _i , S _i ,cl,B _i }, P _i is the area pixel belonging to component C _i , S _i is its segmentation boundary, cl is the component pixel extracted by k-means method The main color (k=2), B _i is its corresponding three-dimensional cuboid proxy (obtained from the subsequent step 2.2). This structured representation will be used frequently in subsequent steps.

2.2 Generation of image 3D proxy: calibrate the camera parameters of each image according to the user's simple interaction, and combine the camera parameters and the part segmentation information of the image object to generate a structured 3D proxy.

2.2.1 Camera Calibration Based on Coordinate Axis System

This method uses a two-dimensional vertex and three two-dimensional vectors (the origin of the three-dimensional coordinate system and the image projection of the coordinate axis) as input. Compared with the existing single-view camera calibration method, this method is more suitable for general images (on the object) less geometric clues).

The camera projection matrix M _3×4 can be expressed as:

${\mathrm{<span\; class="notranslate">\; m</span>}}_{\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; \&times;</span>\mathrm{<span\; class="notranslate">\; 4</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; ]</span><span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; K</span>}<span\; class="notranslate">\; =</span>\left(\begin{array}{ccc}\mathrm{<span\; class="notranslate">\; f</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; f</span>}& \mathrm{<span\; class="notranslate">\; v</span>}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right)<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; t</span>}<span\; class="notranslate">\; =</span><span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}<span\; class="notranslate">\; \}</span>$

Among them, K is the internal reference matrix of the camera, {u, v} is set as the center of the image, the focal length f is used as a variable, R is an orthogonal matrix parameterized with Euler angles, t is a translation vector, and there are 7 variable parameters in total.

The image projection points P _o and P _up of the origin and point (0,0,1) of the three-dimensional coordinate system can be expressed as homogeneous coordinates:

${\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; o</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; up</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; z</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

And the projection point {l _x , l _y , l _z } of the three-dimensional coordinate axis {x, y, z} can be expressed as:

${\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\begin{array}{}\end{array}\begin{array}{}\end{array}\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; the\; y</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{c}{\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}\mathrm{<span\; class="notranslate">\; 1</span>}}\\ {\mathrm{<span\; class="notranslate">\; l</span>}}_{\mathrm{<span\; class="notranslate">\; z</span>}\mathrm{<span\; class="notranslate">\; 2</span>}}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]$

According to the theory of projective geometry, the following equations can be established:

After expansion, the following seven equations are obtained:

(fc ₁ c ₂ -us ₂ )l _x1 +(fc ₂ s ₁ -vs ₂ )l _x2 -s ₂ ＝0

(fc ₁ s ₂ s ₃ -fc ₃ s ₁ +uc ₂ s ₃ )l _y1 +(fc ₁ c ₃ +fs ₁ s ₂ s ₃ +vc ₂ s ₃ )l _y2 +c ₂ s ₃ ＝0

(fs ₁ s ₃ +fc ₁ c ₃ s ₂ +uc ₂ c ₃ )l _z1 +(fc ₃ s ₁ s ₂ -fc ₁ s ₃ +vc ₂ c ₃ )l _z2 +c ₂ c ₃ ＝0

ft ₁ +ut ₃ -o ₁ t ₃ =0

ft ₂ +vt ₃ -o ₂ t ₃ =0

fs ₁ s ₃ +fc ₁ c ₃ s ₂ +uc ₂ c ₃ +ft ₁ +ut ₃ -z ₁ c ₂ c ₃ -z ₁ t ₃ =0

fc ₃ s ₁ s ₂ -fc ₁ s ₃ +vc ₂ c ₃ +ft ₂ +vt ₃ -z ₂ c ₂ c ₃ -z ₂ t ₃ =0

In this method, the camera parameters are obtained by solving the above equations through nonlinear optimization.

2.2.2 Structure-aware 3D proxy fitting

Based on the camera projection matrix M obtained in step 2.2.1, this method uses interactive image technology (ZHENG, Y., CHEN, X., CHENG, M.-M., ZHOU, K., HU, S.-M ., AND MITRA, N.J. 2012. Interactive images: cuboid proxies for smart image manipulation. ACMTrans. Graph. 31, 4 (July), 99:1–99:11.) Initializing axis-aligned cuboids. Since there is no structural relationship, these independently initialized cuboids are scattered in space. Therefore, this method performs global optimization to reconstruct the structural relationship between these parts. The goal of optimization is to make the parts meet the three-dimensional space while conforming to the image boundary. Geometric structure relationship, its energy equation is as follows:

E(B ₁ ,B ₂ ,...,B _N )＝E _fitting +E _unary +E _pair

Among them, the degree of deviation between the first E _fitting penalty optimization result and the initial cuboid is expressed as the cumulative distance between the two-dimensional projection points of the optimized cuboid and the vertices of the initial cuboid:

${\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; fitting</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\overset{\mathrm{<span\; class="notranslate">\; N</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}}\underset{\mathrm{<span\; class="notranslate">\; k</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>\mathrm{<span\; class="notranslate">\; m</span>}{\mathrm{<span\; class="notranslate">\; v</span>}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; m</span>}{\stackrel{<span\; class="notranslate">\; \&OverBar;</span>}{\mathrm{<span\; class="notranslate">\; v</span>}}}^{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}$

where N is the number of parts, v ^k and are the optimized cuboid B _i and the initial cuboid The vertices of are calculated here using normalized homogeneous coordinates.

The unary constraint term E _unary penalizes how far the structural constraints on a single agent deviate. It mainly includes two structural constraints {Globreflection, OnGround} to ensure the correct relationship between parts and camera parameters. Globreflection means that a cuboid is reflectively symmetrical about a global coordinate plane, and OnGround means that the cuboid must be placed on the ground. This entry is defined as:

A collection of cuboids constrained by OnGround. dist is the distance function from point to plane, is the vertex of the cuboid surface with the smallest z value.

The binary constraint term E _pair penalizes the deviation of the structural constraints between the two agents. It mainly includes three structural constraints {Symmetry, On, Side} to ensure the structural relationship between two cuboids. Symmetry means that two cuboids are reflectively symmetrical about a global coordinate plane, while On and Side mean that a cuboid is placed on top of another cuboid or leaning against the side. This entry is defined as:

in is a set of cuboids that satisfy the reflective symmetry constraints in pairs, and S _p are cuboid sets satisfying pairwise On and Side constraints respectively. The rf function calculates the mirror image position of a point according to the plane p, and c _i is the center point of cuboid B _i . The first term in the above formula ensures the reflective symmetry constraint by requiring the center point of the geometric agent to be reflectively symmetric about the plane, while the latter two penalize the distance between the face center point of one cuboid and the top or side of another cuboid to ensure On and Side constraint. _bci is the center of the base of B _i and tp _j is the top of B _j ; similarly sc _i is the center of the side of B _i and sp _j is the side of B _j .

In this method, the cuboids are aligned with the coordinate axes, and only six parameters need to be optimized for each cuboid, that is, the scale and center position {l _x , l _y , l _z , c _x , c _y , c _z } of each cuboid. This method uses the nonlinear optimization method Levenberg-Marquardt (LOURAKIS, M., 2004.levmar:Levenberg-marquardtnonlinear least squares algorithms in C/C++.[webpage]http://www.ics.forth.gr/～lourakis/levmar /.) to minimize the total energy.

2.3 Proxy-guided part synthesis: After estimating camera parameters and constructing 3D proxies, all selected parts are composited by translating and scaling each 3D proxy in 3D space, followed by 2D image warping and gluing based on the transformed proxies A single image object.

2.3.1 Connection of components

This method uses "connection slots" (KALOGERAKIS, E., CHAUDHURI, S., KOLLER, D., AND KOLTUN, V.2012.A probabilistic model for component-based shape synthesis. ACM Trans.Graph.31,4(July ), 55:1–55:11.; SHEN, C.-H., FU, H., CHEN, K., AND HU, S.-M. 2012. Structure recovery by part assembly. ACM Transactions on Graphics (TOG )31,6,180.) to connect object parts. After building the 3D proxy, a connection slot pair will be generated between the connected parts, where each slot contains: a) the target part connected to the part to which the slot belongs; b) the contact point between the two parts connected to each other; c) The scale size of the connection target part. This method adopts the following strategy to generate three-dimensional contact points: if two agents intersect, take the midpoint of the intersecting part; otherwise, take the midpoint of the closest surface of the agent with a smaller volume to the agent with a larger volume. A two-dimensional contact point is a point where the connected parts are relatively close to each other on the image boundary (within a given threshold, this method uses 5 pixels).

(1) Connect to 3D Proxy

This method optimizes the position and size of each agent B _i under the constraints of the agent connection relationship. c _i , l _i and represent the center, size, and 3D contact point of the connecting slot _k of agent Bi, respectively. Define transformation B _i and The rigid body transformation of : Wherein Λ _i =diag(s _i ), s _i and t _i are the scaling and translation parts in the rigid body transformation respectively. The transformation applied to B _i needs to be constrained by the size and position of the agent connected to it, so the definition is the size of the target agent connected to Bi through slot _k in the original training image. This method defines the contact energy function term E _c as follows:

in is the set of agent pairs matched by connection slots, and m _i and m _j are the indexes of the matching connection slots in their respective agents. This energy function term connects matching contact points together and makes the connected agents have matching sizes. In addition, this method uses two shape-preserving energy function terms E _s and E _t to avoid excessive deformation during optimization:

${\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; the\; s</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{<span\; class="notranslate">\; [</span>\mathrm{<span\; class="notranslate">\; 1,1,1</span>}<span\; class="notranslate">\; ]</span>}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; t</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}$

This method obtains the optimal transformation for each agent B _i by minimizing the following energy function

${\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arg</span>}\underset{{\mathrm{<span\; class="notranslate">\; T</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\mathrm{<span\; class="notranslate">\; min</span>}}{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; c</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; the\; s</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; \&omega;</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}{\mathrm{<span\; class="notranslate">\; E.</span>}}_{\mathrm{<span\; class="notranslate">\; t</span>}}$

Wherein, this method uses parameters ω _c =1, ω _s =0.5 and ω _t =0.1.

(2) Bending two-dimensional image parts

After connecting the 3D proxy, the method computes and warps the corresponding 2D image component of the 3D proxy. When constructing the 3D proxy of the image in step 2.2, the method uniformly samples n _i 2D reference points on the image boundary of each proxy B _i and will The corresponding 3D reference points are obtained by projecting onto the visible surface of the agent (points outside the boundary of the 2D projection of the agent are removed). n _i =200 in all experiments of this method.

In this method, the above three-dimensional reference point is obtained through the optimal transformation calculated in step (1) to obtain the corresponding three-dimensional target point, and then the three-dimensional target point is projected onto the two-dimensional plane of the synthetic scene to obtain the two-dimensional target point

Due to the limited viewpoint variation of 3D proxies, this method employs 2D affine transformations for image warping. The optimal affine transformation matrix is obtained by minimizing the distance between the curved 2D reference point and the 2D target point

${\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; arg</span>}\underset{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}{\mathrm{<span\; class="notranslate">\; min</span>}}\underset{\mathrm{<span\; class="notranslate">\; i</span>}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}{<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \&CenterDot;</span>{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; a</span>}}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; -</span>{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; b</span>}}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; ,</span>\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; |</span><span\; class="notranslate">\; |</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}$

then use Image warp each part.

(3) Connect two-dimensional image components

This method will also be used after step (2) The two-dimensional contact points in each connection slot k of each component i Perform transformation to obtain the current coordinates of the two-dimensional contact point. The image parts are then moved using a breadth-first search strategy. The largest part is queued as the reference. Whenever a component is popped from the queue, all components connected to this component and not accessed will be moved according to the current two-dimensional contact point information in the connection slot, and then put into the queue in turn. After all components have been accessed, the search is terminated, resulting in the composite image object after the connection of the two-dimensional image components. Among them, when moving part i to part j (its matching connecting slots are m _i and m _j ), firstly make center and Align the center of the , and then align the A strategy in which points located outside the segmentation boundary of component j continuously move towards the closest point on the boundary.

2.3.2 Color optimization

On top of the synthetic results, this method is based on a color matching model (O'DONOVAN, P., AGARWALA, A., AND HERTZMANN, A. 2011. Color compatibility from large datasets. ACM Transactions on Graphics (TOG) 30, 4, 63 .) and data-driven palettes for color optimization. After step 1, this method uses k-means to extract a palette of 5 colors from each image in the image dataset, and then uses k-means to extract a palette of 40 colors from all these colors as the data palette . In the synthesized image, the hues in the dominant colors of the largest components are assigned to the data palette (fitted with a variance parameter σ) to generate a new palette. Next, the present method is adapted from the work of Yu et al. (YU, L.-F., YEUNG, S.K., TERZOPOULOS, D., AND CHAN, T.F. 2012. DressUp!: Outfit synthesis through automatic optimization. ACM Transactions on Graphics 31, 6, 134: 1–134:14.) to select the color set with the greatest color matching from the new palette, and through the color transfer method (REINHARD, E., ADHIKHMIN, M., GOOCH, B., AND SHIRLEY, P. 2001. Color transfer between images. Computer Graphics and Applications, IEEE21, 5, 34–41.) assign it to each component.

3. Training and Synthesis of Bayesian Probabilistic Graphical Models

In this step, statistical learning is first performed on an object-type-specific image dataset based on consistently segmented image parts to obtain a probabilistic graphical model that can express complex dependencies among shape style, object structure, part category, and camera parameters , and then perform probabilistic reasoning on the acquired probabilistic graphical model, and sample the component types and styles to obtain the composition scheme and viewpoint attributes of high-probability image objects, and finally use the viewpoint-aware image object synthesis method to generate all the resulting images.

3.1 Training of Bayesian Probabilistic Graphical Model

This method is similar to the work of Kalogerakis et al. (KALOGERAKIS, E., CHAUDHURI, S., KOLLER, D., AND KOLTUN, V.2012. A probabilistic model for component-based shape synthesis. ACM Trans.Graph.31,4 (July), 55:1–55:11.) to model collections of certain types of image objects, where hidden variables represent the overall structure and part style of objects, and observed variables represent part categories , geometric features and their adjacency relations. The feature of this method is to introduce additional observation variables to represent the viewpoint information of the image.

3.1.1 Representation of Bayesian Probabilistic Graphical Models

The above table lists the random variables in the probabilistic graphical model used in this method. Where V is the viewpoint parameter, which is obtained by Mean Shift clustering (with a radius of 0.2) on the camera parameters obtained in step 2.2.1, expressed as an integer value of the category index. The geometric feature vector C _l contains the size of the three-dimensional cuboid agent of the l category part and the point distribution model feature vector of the two-dimensional contour of the image (COOTES, TF, TAYLOR, CJ, COOPER, 599 D.H., GRAHAM, J., ET AL. 1995. Active shape models-their training and application. Computer vision and image understanding61,1,38–59.). The calculation of the point distribution model is performed separately according to different viewpoint categories. Hidden variables R and S are learned from the training data.

The entire joint probability distribution can be decomposed into the product of a set of conditional probabilities:

3.1.2 Bayesian probabilistic graphical model training

After steps 1 and 2, a set of feature vectors is extracted from the image of the training data (the amount of data is K) where O _k = {V _k , N _k , C _k , D _k }. This method learns the structure and parameters of the probability graph by maximizing the following likelihood function J:

Among them, the evaluation score of Bayesian information criterion (SCHWARZ, G.1978. Estimating the dimension of a model. The annals of statistics6,2,461–464.) is used to select the optimal probability graph structure G (defined domain range). is the maximum a posteriori probability estimate (MAP) of the parameters in G, m _θ is the number of independent parameters, and K is the number of data. This method uses the Expectation Maximum (EM) algorithm to calculate the parameters corresponding to the maximum likelihood

where P(θ|G) is the prior probability distribution for the parameter θ. In the M step of the EM algorithm, the calculation method of the conditional probability table parameters (discrete random variable) R, V, S _l , D _l and the conditional linear Gaussian distribution parameter (continuous random variable) C _l is the same as that of Kalogerakis et al. (KALOGERAKIS ,E.,CHAUDHURI,S.,KOLLER,D.,AND KOLTUN,V.2012.A probabilistic model for component-basedshape synthesis.ACM Trans.Graph.31,4(July),55:1–55:11. ) in the same form. In step E, the conditional probabilities of the hidden variables R and S under the observed variable O _k are calculated using the following formula:

$\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}}{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span>$

$\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; |</span>{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; =</span>\frac{\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; R</span>}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; ,</span>{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span>}{\mathrm{<span\; class="notranslate">\; P</span>}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; o</span>}}_{\mathrm{<span\; class="notranslate">\; k</span>}}<span\; class="notranslate">\; )</span>}$

where l is the part class flag. This method calculates the joint probability P(R,S _l ,O _k ) according to the following formula:

This method searches for the graph structure G with the largest J through the greedy strategy of gradually increasing the domain ranges of hidden variables R and S (optional values of discrete variables).

3.2 Synthesis of Bayesian Probabilistic Graphical Models

The synthesis of image objects can be divided into three steps. First, a set of image components for synthesis is determined. These parts are then concatenated into a single image object. Finally, optimize the color of the composited object. Among them, the last two steps can be realized by the viewpoint-aware image object synthesis technology in step 2.

3.2.1 Synthesis of component assemblies

In a mathematical sense, different sets of components can be regarded as different sampling points of the probabilistic model. Therefore, this method adopts a depth-first search strategy to search the shape space of image objects. Starting from the root node variable R, each random variable on the search path is assigned its possible value. With the work of Kalogerakis et al. (KALOGERAKIS, E., CHAUDHURI, S., KOLLER, D., AND KOLTUN, V.2012. A probabilistic model for component-based shape synthesis. ACM Trans.Graph.31, 4 (July) , 55:1–55:11.) is consistent with the deterministic algorithm, and this method prunes the values of variables whose probability is less than a certain threshold (10 ^-10 is used in the implementation). To ensure search feasibility, the continuous variable C _l will only be assigned values corresponding to components that have appeared in the training data. Among the effective sampling points finally found by the search program, the value of the variable C _l determines the set of components used for synthesis.

4. Export of image object synthesis results: export and store the image object synthesis results of the above steps in a common format, so that it can be used in other digital media products and applications.

4.1 Export of results

Steps 2 and 3 of the method can synthesize a large number of image data of new objects. In order to be compatible with the general data format in the industry, specifically, the synthesized image can be stored in a related file format as the final result export form of this method. On the other hand, step 2 constructs a 3D proxy of the parts for the image object, and the 3D proxy obtained by analyzing the training image object together with its part structure, segmented area, and texture image can be exported into a clear and easy-to-read file format for the needs technology or application use.

4.2 Application of results

As a general image representation form, the derived results of this method can be applied to all existing image editing/design systems.

Implementation example

The inventor has realized all implementation examples of the present invention on a machine equipped with Intel Core2Quad Q9400 central processing unit, 4GB internal memory and Win7 operating system. The inventors obtained all the experimental results shown in the accompanying drawings using all parameter values listed in the detailed description.

Figure 12 and Figure 13 show the comparison of the image objects synthesized by direct synthesis and our method. The direct synthesis results often produce obvious distortion and unnaturalness, but the viewpoint-aware synthesis method in the present invention can generate novel image objects that are visually more reasonable and realistic.

The inventors invited some users to test the synthetic results generated by the synthesis algorithm of Bayesian probabilistic graphical models in this method (Figs. 6-10). Evaluation results show that users perceive the synthesized image objects to be visually perfectly plausible and indeed novel objects compared to objects in the original image collection. Among them, a total of 48% of the choices considered the synthetic object to be visually plausible (no significant difference compared with 52% of the selected training data); while for each category of synthetic novelty (see Figure 11), the highest 90% of the users believed The result of the robot assembly is a new object, and the lowest 79% of users think the result of the table lamp assembly is a new object.

In terms of training time for the Bayesian probabilistic graphical model, the chair set (42 images, 6 part categories, 243 parts in total) takes about 20 minutes, and the cup set (22 images, 3 part categories, 44 parts in total) takes about 20 minutes. ) takes about 5 minutes, lamp collection (30 images, 3 parts categories, 90 parts in total) takes about 12 minutes, robot collection (23 images, 5 parts categories, 130 parts in total) takes about 15 minutes, Toy airplane assembly (15 images, 4 part categories, 63 parts in total) takes approximately 3 minutes. In terms of the synthesis time of image objects, it takes an average of 20 seconds to 1 minute to enumerate the set of components required for synthesis; while the connection process of the components in each image takes an average of 4 seconds, and the color optimization takes an average of 1 second.

Claims (2)

1. the structuring synthetic method of an image object, it is characterised in that comprise the steps:

(1) pretreatment of image object data: use the digital or image set of network equipment collection particular type object Close, it is desirable to object structures complete display, and use image segmentation and marking tool to obtain object building block Consistent cut zone；

(2) the image object synthetic method of viewpoint perception: according to the simple mutual camera demarcating single image of user Parameter, and combining camera parameter and the three-dimensional agency of image object segmentation information generating structure, then utilize Image component is connected into the image object of novelty by three-dimensional agency and contact point information, finally by intelligence Color adjusts and obtains result images；This step specifically includes following sub-step:

(2.1) user assists alternately, to the coordinate center of world coordinate system and coordinate axes in each image of input Direction is demarcated；

(2.2) it is optimized based on customer interaction information and is calculated the camera parameter of input picture；

(2.3) based on the image object segmentation information obtained in step (1), integrating step (2.2) obtains Each parts of image object are calculated initial three-dimensional agency by camera parameter；

(2.4) initial three-dimensional agencies based on each parts of object required by step (2.3), between each parts of binding object Overall structure constraint information is optimized and generates final three-dimensional agency；

(2.5) assembled scheme based on input and view information, by the three-dimensional agency of each building block by it even Three-Dimensional contact dot information defined in access slot links together；

(2.6) based on the three-dimensional agency connected, the primitive part image in agency is raw by two dimensional affine conversion Becoming the image of component under current view point, wherein affine transformation calculates under the constraint that three-dimensional is acted on behalf of and obtains；

(2.7) based on the two dimensional touch dot information defined in parts link slot by all parts image in image space Seamless link is together；

(2.8) color combined information based on source image and existing color harmony degree evaluation model optimization calculate To the final mass-tone of each parts of object, and by the color of the color transfer method conversion each parts of object, obtain The result images of new object；

(3) training of Bayesian probability graph model and integrated approach: image component based on consistent segmentation, specific Carry out statistical learning on the image data set of object type, obtain one can express shape style, object structures, The probability graph model of complicated dependence between component categories and camera parameter；And by the Bayes in acquistion Carrying out probability inference on graph model, sampling unit type and style obtains the group of high probability image object One-tenth scheme and viewpoint attribute, synthesize result images finally by the method in step (2)；

(4) derivation of image object synthesis result: the result images that step (2) and step (3) are obtained, bag Include camera parameter and three-dimensional proxy data that step (2) obtains, derive and storage with general format.

The structuring synthetic method of image object the most according to claim 1, it is characterised in that described step (3) Including following sub-step:

(3.1) structural information of image object is set up based on the consistent segmentation result obtained in step (1), including each The type of parts, number, shape facility, interconnected relationship attribute, wherein shape facility is by two-dimensional silhouette Points distribution models coordinate describe；

(3.2) carry out clustering based on the camera parameter obtained in step (2.2) and obtain the discrete representation of image viewpoint, The i.e. index value of viewpoint classification；

(3.3) based on the view information obtained in the object structures information obtained in step (3.1) and step (3.2), The image data set of whole particular type object utilizes EM algorithm and bayesian information criterion training One Bayesian probability graph model, with this generate model to characterize whole image data set space object structures, The complicated dependence of annexation between shape style, component categories and parts；

(3.4) sample based on probability graph model, obtain composition proposal and the view information of new object, the most each The source of building block；

(3.5) based on a large amount of assembled schemes obtained in step (3.4) and view information, utilize in step (2) Method generate all of result images.