CN113436335A - Incremental multi-view three-dimensional reconstruction method - Google Patents

Incremental multi-view three-dimensional reconstruction method Download PDF

Info

Publication number
CN113436335A
CN113436335A CN202110681635.2A CN202110681635A CN113436335A CN 113436335 A CN113436335 A CN 113436335A CN 202110681635 A CN202110681635 A CN 202110681635A CN 113436335 A CN113436335 A CN 113436335A
Authority
CN
China
Prior art keywords
human body
dimensional
shape
deformation
principal component
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202110681635.2A
Other languages
Chinese (zh)
Other versions
CN113436335B (en
Inventor
金晶
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhaoyuan State Owned Assets Management Co ltd
Original Assignee
Zhaoyuan State Owned Assets Management Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhaoyuan State Owned Assets Management Co ltd filed Critical Zhaoyuan State Owned Assets Management Co ltd
Priority to CN202110681635.2A priority Critical patent/CN113436335B/en
Publication of CN113436335A publication Critical patent/CN113436335A/en
Application granted granted Critical
Publication of CN113436335B publication Critical patent/CN113436335B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20—Finite element generation, e.g. wire-frame surface description, tesselation
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02T—CLIMATE CHANGE MITIGATION TECHNOLOGIES RELATED TO TRANSPORTATION
    • Y02T90/00—Enabling technologies or technologies with a potential or indirect contribution to GHG emissions mitigation

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Image Processing (AREA)
  • Image Analysis (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)

Abstract

The invention provides an incremental multi-view three-dimensional reconstruction method, which comprises the steps of firstly, analyzing human body appearance principal components oriented to three-dimensional grid parameterization, and converting human body shape parameters into principal component coefficients; then, a loss function is designed according to the human body three-dimensional shape prediction, three additional output units of thickness deformation, width deformation and height deformation are added, the average error of all vertexes of a human body shape grid is used as the loss function, and the predicted human body shape and the real shape are scaled to a fixed length; finally, constructing and training a network regression model, inputting the two-dimensional contour of the human body and the posture of the camera into an encoder, performing regression on principal component coefficients, width deformation, height deformation and thickness deformation factors, and obtaining a reconstructed three-dimensional shape of the human body by applying inverse principal component analysis; the invention simplifies the manual process and the registration process, simultaneously does not need to consider the sequence and the number of the multiple views in the reconstruction process, and has wide application range and simple and convenient use mode.

Description

Incremental multi-view three-dimensional reconstruction method
Technical Field
The invention belongs to the technical field of artificial intelligence, and particularly relates to an incremental multi-view three-dimensional reconstruction method.
Background
In the traditional three-dimensional modeling of the human body appearance, the size of different parts of the human body is measured, and a reconstruction result is obtained by adopting a geometric method to perform parameter fitting. With the development of artificial intelligence technology, the problem of human body three-dimensional modeling can be solved through image reconstruction based on a deep learning method. Convolutional neural networks for three-dimensional reconstruction typically have an encoder that maps images of the human body into feature vectors and then decodes the feature vectors into the desired three-dimensional model output according to the representation of the three-dimensional shape. Common representations include voxels, meshes, and point clouds.
Multi-view three-dimensional reconstruction can overcome the self-occlusion problem. A convolutional neural network based three-dimensional reconstruction for the case of a fixed number of views can be solved. The long-short term memory neural network framework needs to be used when the number of views changes, but if the order of inputting the views changes, the output is not consistent.
Disclosure of Invention
The invention provides an incremental multi-view three-dimensional reconstruction method for solving the problem of multi-view three-dimensional model reconstruction under the conditions of different image sequences and different image quantities, which is not influenced by the sequence of input images and can be popularized to any number of views.
The invention is realized by the following scheme:
an incremental multi-view three-dimensional reconstruction method comprises the following steps:
the method comprises the following steps:
step a: carrying out human body two-dimensional contour processing on the image, and designing a three-dimensional model into a standard grid structure; analyzing the human body appearance principal components oriented to the three-dimensional grid parameterization; converting the human body shape parameters into principal component coefficients;
step b: designing a loss function aiming at the human body three-dimensional shape prediction; taking the average error of all the vertexes of the human body shape grid as a loss function, and scaling the predicted human body shape and the real shape to a fixed length;
step c: and constructing and training a network regression model, inputting the two-dimensional outline of the human body and the camera posture into an encoder, and obtaining the reconstructed three-dimensional shape of the human body by applying inverse principal component analysis.
Further, in step a:
step a 1: making a two-dimensional outline shape data set of a human body image; more than 1000 pictures are taken of the human body at each angle, external parameters of a camera are marked on each image, and two-dimensional contour processing of the human body is carried out on the images based on the existing segmentation network;
step a 2: designing a standard grid of a human body three-dimensional model; the three-dimensional model of the human body is designed as a standard mesh with 6980 vertices whose coordinates are expressed as P ═ Pxi pyi pzi]TWherein p isxi、pyiAnd pziEach vertex position in the human body thickness, width and height directions is respectively represented, i is 1, …, 6980;
step a 3: analyzing the principal component coefficient of the human body appearance change; applying a principal component analysis method to a human body two-dimensional contour data set to obtain a principal component coefficient C ═ C for describing human body shape change0 c1 c2 c3 c4 c5 c6 c7 c8 c9]TWherein c is0Represents height; c. C1Representing the circumference of the chest; c. C2Representing the hip circumference; c. C3Representing the abdominal circumference; c. C4Represents lateral compressive stretching; c. C5Indicating that the belly is large but the whole is thin; c. C6Showing that the belly is enlarged and other parts are thin and small; c. C7Indicating the degree of longitudinal body compression; c. C8Represents the horizontal width; c. C9Indicating the shoulder width.
Further, in step b:
step b 1: aiming at the prediction of a human body three-dimensional model, the thickness deformation k is increasedtWidth of deformation kwHigh degree of deformation khThe output unit of (1); the vertex coordinates after deformation are denoted as P' ═ ktpxi kwpyi khpzi]T；
Step b 2: taking the average error of all the vertexes of the human body shape mesh as a loss function;
the Loss function Loss of the human body three-dimensional shape prediction is set as the average error described by the Euclidean distance d (·,) of the 6980 vertexes caused by the change of the human body shape parameters,
Figure BDA0003122860090000021
step b 3: in the process of calculating the human body three-dimensional shape loss function, the predicted human body shape and the real shape are scaled to the fixed length of 170 cm.
Further, in step c:
step c 1: inputting the two-dimensional outline of the human body and the posture of the camera into an encoder, and calculating a one-dimensional feature vector; combining a plurality of feature vectors generated by multiple views into a single feature vector through a pooling layer;
step c 2: decoder for principal component coefficient C, thickness deformation ktWidth of deformation kwHigh degree of deformation khThe factors are regressed, and a linear activation function is applied to all output units;
step c 3: by applying inverse principal component analysis, the k is deformed by the thicknesstWidth of deformation kwHigh degree of deformation khAnd carrying out horizontal and vertical scaling on the vertex, and scaling to the whole human body to obtain the reconstructed human body three-dimensional shape.
The invention has the beneficial effects
(1) The method of the invention aims at the human body appearance, draws the human body three-dimensional outline by analyzing the sampling shape in the principal component model;
(2) the method extracts the human body contour and the camera attitude data based on the depth network to infer the parameter characteristics and the appearance of the human body, can process any number of input views without being influenced by the input sequence, and realizes incremental dense grid reconstruction;
(3) the incremental reconstruction is carried out based on the multi-view three-dimensional reconstruction model, the manual process and the registration process are simplified, meanwhile, the sequence and the number of the multi-view are not required to be considered in the reconstruction process, the reconstruction is carried out by the method, the application range is wide, and the use mode is simple and convenient.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Detailed Description
The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments; all other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
As shown in fig. 1, an incremental multi-view three-dimensional reconstruction method:
the method comprises the following steps:
step a: carrying out human body two-dimensional contour processing on the image, and designing a three-dimensional model into a standard grid structure; analyzing the human body appearance principal components oriented to the three-dimensional grid parameterization; converting the human body shape parameters into principal component coefficients;
step b: designing a loss function aiming at the human body three-dimensional shape prediction; taking the average error of all the vertexes of the human body shape grid as a loss function, and scaling the predicted human body shape and the real shape to a fixed length;
step c: and constructing and training a network regression model, inputting the two-dimensional outline of the human body and the camera posture into an encoder, and obtaining the reconstructed three-dimensional shape of the human body by applying inverse principal component analysis.
In step a:
step a 1: making a two-dimensional outline shape data set of a human body image; more than 1000 pictures are taken of the human body at each angle, external parameters of a camera are marked on each image, and two-dimensional contour processing of the human body is carried out on the images based on the existing segmentation network;
step a 2: designing a standard grid of a human body three-dimensional model; the three-dimensional model of the human body is designed as a standard mesh with 6980 vertices whose coordinates are expressed as P ═ Pxi pyi pzi]TWherein p isxi、pyiAnd pziEach vertex position in the human body thickness, width and height directions is respectively represented, i is 1, …, 6980;
step a 3: analyzing the principal component coefficient of the human body appearance change; applying a principal component analysis method to a human body two-dimensional contour data set to obtain a principal component coefficient C ═ C for describing human body shape change0 c1 c2 c3 c4 c5 c6 c7 c8 c9]TWherein c is0Represents height; c. C1Representing the circumference of the chest; c. C2Representing the hip circumference; c. C3Representing the abdominal circumference; c. C4Represents lateral compressive stretching; c. C5Indicating that the belly is large but the whole is thin; c. C6Showing that the belly is enlarged and other parts are thin and small; c. C7Indicating the degree of longitudinal body compression; c. C8Represents the horizontal width; c. C9Indicating the shoulder width.
In step b:
step b 1: to provide greater flexibility in human three-dimensional shape prediction, the thickness deformation k is increased for human three-dimensional model predictiontWidth of deformation kwHigh degree of deformation khThe output unit of (1); the vertex coordinates after deformation are denoted as P' ═ ktpxikwpyi khpzi]T；
Step b 2: taking the average error of all the vertexes of the human body shape mesh as a loss function;
the Loss function Loss of the human body three-dimensional shape prediction is set as the average error described by the Euclidean distance d (·,) of the 6980 vertexes caused by the change of the human body shape parameters,
Figure BDA0003122860090000041
step b 3: in the process of calculating the human body three-dimensional shape loss function, the predicted human body shape and the real shape are scaled to the fixed length of 170 cm.
In step c:
step c 1: inputting the two-dimensional outline of the human body and the posture of the camera into an encoder, and calculating a one-dimensional feature vector; the number of copies of the encoder is the same as that of the views, and then a plurality of feature vectors generated by the multiple views are combined into a single feature vector through a pooling layer;
step c 2: decoder pairs the principal component coefficient C, the thickness deformation k through two fully connected layerstWidth of deformation kwHigh degree of deformation khThe factors are regressed, and a linear activation function is applied to all output units;
step c 3: by applying inverse principal component analysis, the k is deformed by the thicknesstWidth of deformation kwHigh degree of deformation khAnd carrying out horizontal and vertical scaling on the vertex, and scaling to the whole human body to obtain the reconstructed human body three-dimensional shape.
The incremental multi-view three-dimensional reconstruction method provided by the invention is described in detail, the principle and the implementation mode of the invention are explained, and the description of the embodiment is only used for helping to understand the method and the core idea of the invention; meanwhile, for a person skilled in the art, according to the idea of the present invention, there may be variations in the specific embodiments and the application scope, and in summary, the content of the present specification should not be construed as a limitation to the present invention.

Claims (4)

1. An incremental multi-view three-dimensional reconstruction method is characterized in that:
the method comprises the following steps:
step a: carrying out human body two-dimensional contour processing on the image, and designing a three-dimensional model into a standard grid structure; analyzing the human body appearance principal components oriented to the three-dimensional grid parameterization; converting the human body shape parameters into principal component coefficients;
step b: designing a loss function aiming at the human body three-dimensional shape prediction; taking the average error of all the vertexes of the human body shape grid as a loss function, and scaling the predicted human body shape and the real shape to a fixed length;
step c: and constructing and training a network regression model, inputting the two-dimensional outline of the human body and the camera posture into an encoder, and obtaining the reconstructed three-dimensional shape of the human body by applying inverse principal component analysis.
2. The method of claim 1, further comprising: in step a:
step a 1: making a two-dimensional outline shape data set of a human body image; more than 1000 pictures are taken of the human body at each angle, external parameters of a camera are marked on each image, and two-dimensional contour processing of the human body is carried out on the images based on the existing segmentation network;
step a 2: designing a standard grid of a human body three-dimensional model; the three-dimensional model of the human body is designed as a standard mesh with 6980 vertices whose coordinates are expressed as P ═ Pxi pyi pzi]TWherein p isxi、pyiAnd pziRespectively indicate the thickness and width of the human bodyEach vertex position in the degree and height directions, i ═ 1, …, 6980;
step a 3: analyzing the principal component coefficient of the human body appearance change; applying a principal component analysis method to a human body two-dimensional contour data set to obtain a principal component coefficient C ═ C for describing human body shape change0 c1 c2 c3 c4 c5 c6 c7 c8 c9]TWherein c is0Represents height; c. C1Representing the circumference of the chest; c. C2Representing the hip circumference; c. C3Representing the abdominal circumference; c. C4Represents lateral compressive stretching; c. C5Indicating that the belly is large but the whole is thin; c. C6Showing that the belly is enlarged and other parts are thin and small; c. C7Indicating the degree of longitudinal body compression; c. C8Represents the horizontal width; c. C9Indicating the shoulder width.
3. The method of claim 2, further comprising: in step b:
step b 1: aiming at the prediction of a human body three-dimensional model, the thickness deformation k is increasedtWidth of deformation kwHigh degree of deformation khThe output unit of (1); the vertex coordinates after deformation are denoted as P' ═ ktpxi kwpyi khpzi]T；
Step b 2: taking the average error of all the vertexes of the human body shape mesh as a loss function;
the Loss function Loss of the human body three-dimensional shape prediction is set as the average error described by the Euclidean distance d (·,) of the 6980 vertexes caused by the change of the human body shape parameters,
Figure FDA0003122860080000011
step b 3: in the process of calculating the human body three-dimensional shape loss function, the predicted human body shape and the real shape are scaled to the fixed length of 170 cm.
4. A method of lossing according to claim 3, characterised by: in step c:
step c 1: inputting the two-dimensional outline of the human body and the posture of the camera into an encoder, and calculating a one-dimensional feature vector; combining a plurality of feature vectors generated by multiple views into a single feature vector through a pooling layer;
step c 2: decoder for principal component coefficient C, thickness deformation ktWidth of deformation kwHigh degree of deformation khThe factors are regressed, and a linear activation function is applied to all output units;
step c 3: by applying inverse principal component analysis, the k is deformed by the thicknesstWidth of deformation kwHigh degree of deformation khAnd carrying out horizontal and vertical scaling on the vertex, and scaling to the whole human body to obtain the reconstructed human body three-dimensional shape.
CN202110681635.2A 2021-06-18 2021-06-18 Incremental multi-view three-dimensional reconstruction method Active CN113436335B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202110681635.2A CN113436335B (en) 2021-06-18 2021-06-18 Incremental multi-view three-dimensional reconstruction method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202110681635.2A CN113436335B (en) 2021-06-18 2021-06-18 Incremental multi-view three-dimensional reconstruction method

Publications (2)

Publication Number Publication Date
CN113436335A true CN113436335A (en) 2021-09-24
CN113436335B CN113436335B (en) 2023-06-30

Family

ID=77756661

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202110681635.2A Active CN113436335B (en) 2021-06-18 2021-06-18 Incremental multi-view three-dimensional reconstruction method

Country Status (1)

Country Link
CN (1) CN113436335B (en)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102136155A (en) * 2010-01-27 2011-07-27 首都师范大学 Object elevation vectorization method and system based on three dimensional laser scanning
CN105654548A (en) * 2015-12-24 2016-06-08 华中科技大学 Multi-starting-point incremental three-dimensional reconstruction method based on large-scale disordered images
CN106447763A (en) * 2016-07-27 2017-02-22 扬州大学 Face image three-dimensional reconstruction method for fusion of sparse deformation model and principal component regression algorithm
CN110211223A (en) * 2019-05-28 2019-09-06 哈工大新材料智能装备技术研究院（招远）有限公司 A kind of increment type multiview three-dimensional method for reconstructing
CN111093994A (en) * 2017-08-30 2020-05-01 洛桑联邦理工学院 Method and device for three-dimensional production by tomographic back-projection
CN111815757A (en) * 2019-06-29 2020-10-23 浙江大学山东工业技术研究院 Three-dimensional reconstruction method for large component based on image sequence

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102136155A (en) * 2010-01-27 2011-07-27 首都师范大学 Object elevation vectorization method and system based on three dimensional laser scanning
CN105654548A (en) * 2015-12-24 2016-06-08 华中科技大学 Multi-starting-point incremental three-dimensional reconstruction method based on large-scale disordered images
CN106447763A (en) * 2016-07-27 2017-02-22 扬州大学 Face image three-dimensional reconstruction method for fusion of sparse deformation model and principal component regression algorithm
CN111093994A (en) * 2017-08-30 2020-05-01 洛桑联邦理工学院 Method and device for three-dimensional production by tomographic back-projection
CN110211223A (en) * 2019-05-28 2019-09-06 哈工大新材料智能装备技术研究院（招远）有限公司 A kind of increment type multiview three-dimensional method for reconstructing
CN111815757A (en) * 2019-06-29 2020-10-23 浙江大学山东工业技术研究院 Three-dimensional reconstruction method for large component based on image sequence

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
陈佳宇 等: ""基于二值图像的三维人体模型重建"", 《毛纺科技》 *

Also Published As

Publication number Publication date
CN113436335B (en) 2023-06-30

Similar Documents

Publication Publication Date Title
US10679046B1 (en) Machine learning systems and methods of estimating body shape from images
CN111598998B (en) Three-dimensional virtual model reconstruction method, three-dimensional virtual model reconstruction device, computer equipment and storage medium
CN112396703B (en) Reconstruction method of single-image three-dimensional point cloud model
CN109584353B (en) Method for reconstructing three-dimensional facial expression model based on monocular video
Wang et al. Virtual human modeling from photographs for garment industry
Kähler et al. Geometry-based muscle modeling for facial animation
Wang Parameterization and parametric design of mannequins
CN106600683B (en) A kind of adaptive method for reconstructing of grid model towards Bone CT sequence image
CN109544666B (en) Full-automatic model deformation propagation method and system
CN109829972B (en) Three-dimensional human standard skeleton extraction method for continuous frame point cloud
Lei et al. Cadex: Learning canonical deformation coordinate space for dynamic surface representation via neural homeomorphism
CN107610221B (en) Three-dimensional model generation method based on isomorphic model representation
CN110889893B (en) Three-dimensional model representation method and system for expressing geometric details and complex topology
Tang et al. Skeletonnet: A topology-preserving solution for learning mesh reconstruction of object surfaces from rgb images
CN113111861A (en) Face texture feature extraction method, 3D face reconstruction method, device and storage medium
KR20230004837A (en) Generative nonlinear human shape model
CN111862278A (en) Animation obtaining method and device, electronic equipment and storage medium
CN111583408A (en) Human body three-dimensional modeling system based on hand-drawn sketch
Huang et al. Meshode: A robust and scalable framework for mesh deformation
CN116797733A (en) Real-time three-dimensional object dynamic reconstruction method
CN116385667A (en) Reconstruction method of three-dimensional model, training method and device of texture reconstruction model
Starck et al. Animated statues
CN113436335A (en) Incremental multi-view three-dimensional reconstruction method
CN109785435A (en) A kind of wall method for reconstructing and device
CN116310066A (en) Single-image three-dimensional human body morphology estimation method and application

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant