CN112070896A - Portrait automatic slimming method based on 3D modeling - Google Patents

Portrait automatic slimming method based on 3D modeling Download PDF

Info

Publication number
CN112070896A
CN112070896A CN202010927633.2A CN202010927633A CN112070896A CN 112070896 A CN112070896 A CN 112070896A CN 202010927633 A CN202010927633 A CN 202010927633A CN 112070896 A CN112070896 A CN 112070896A
Authority
CN
China
Prior art keywords
human body
model
slimming
modeling
algorithm
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010927633.2A
Other languages
Chinese (zh)
Other versions
CN112070896B (en
Inventor
张盛平
李宗霖
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Harbin Institute of Technology Weihai
Original Assignee
Harbin Institute of Technology Weihai
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Harbin Institute of Technology Weihai filed Critical Harbin Institute of Technology Weihai
Priority to CN202010927633.2A priority Critical patent/CN112070896B/en
Publication of CN112070896A publication Critical patent/CN112070896A/en
Application granted granted Critical
Publication of CN112070896B publication Critical patent/CN112070896B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20Finite element generation, e.g. wire-frame surface description, tesselation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00Pattern recognition
    • G06F18/20Analysing
    • G06F18/21Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
    • G06F18/213Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods
    • G06F18/2135Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods based on approximation criteria, e.g. principal component analysis
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00Geometric image transformations in the plane of the image
    • G06T3/06Topological mapping of higher dimensional structures onto lower dimensional surfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00Image enhancement or restoration
    • G06T5/77Retouching; Inpainting; Scratch removal
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00Indexing scheme for image analysis or image enhancement
    • G06T2207/30Subject of image; Context of image processing
    • G06T2207/30196Human being; Person

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Data Mining & Analysis (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Biology (AREA)
  • Evolutionary Computation (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • General Engineering & Computer Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computer Graphics (AREA)
  • Geometry (AREA)
  • Software Systems (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention discloses a 3D modeling-based portrait automatic slimming method, which comprises the following steps: acquiring position coordinates, sexes and fat-thin ratings of all human bodies in the picture by using a self-training human body detection algorithm; acquiring skeleton key points of a character main body aiming at each human body area to be processed, and removing a background; sending the human body area with the background removed and the key point coordinates into a 3D modeling algorithm to obtain a 3D model of the human body; obtaining human body shape parameters output by the 3D model by PCA principal component analysis, and realizing the slimming operation of the 3D model; projecting the 3D model which is used for finishing the slimming to a 2D image; and obtaining the triangular mesh by using a Delaunay triangle subdivision algorithm to obtain a final slimming result. The invention greatly simplifies the step of manually liquefying and slimming by using software and overcomes the difficulty that the main stream automatic slimming function can not adjust each part of the body with fine granularity.

Description

Portrait automatic slimming method based on 3D modeling
Technical Field
The invention relates to the technical field of computer vision and computer graphics, in particular to a 3D modeling-based portrait automatic slimming method.
Background
In the digital era, people gradually rely on software to modify portraits in pictures in a large quantity to try to create a satisfactory perfect image, the liquefaction and the slimming are time-consuming and difficult operations in the process, frequent operations of software users are required, the users are required to have perfect cognition on curves of all parts of human bodies, and the pursuit of beauty of people makes the task more and more difficult. Meanwhile, the existing mainstream automatic slimming technology cannot finish fine-grained slimming operation according to the characteristics of different parts of different human bodies, cannot ensure that the size of an original image is not influenced by portrait slimming operation, and is difficult to meet the requirement of a user on the figure body fineness.
Disclosure of Invention
The invention aims to provide a 3D modeling-based portrait automatic slimming method, which determines the position and the gender of a person in a picture through a self-trained human body detection algorithm, recovers human body information by using the 3D modeling algorithm, modifies a parameter combination obtained by PCA principal component analysis, determines what operation should be taken at each part by relying on the priori knowledge that the model has human body models with different statures so as to complete portrait slimming operation on the 3D model, and finally transfers 3D characteristics to an original image to obtain an effect image after the portrait is slim.
In order to achieve the purpose, the invention adopts the following technical scheme:
a portrait automatic slimming method based on 3D modeling comprises the following steps:
acquiring position coordinates, sexes and fat-thin ratings of all human bodies in a picture by using a human body detection algorithm, and obtaining a human body area to be processed after amplification processing and size judgment;
step two, obtaining skeleton key points of the character main body aiming at each human body area to be processed, and removing the background by using a background removal algorithm after the coordinates of the skeleton key points are arranged, so that the interference of the background of the human body area to be processed on the 3D modeling process is avoided;
step three, the human body area with the background removed is matched with the coordinates of the key points of the skeleton and sent to a 3D modeling algorithm to obtain a 3D model of the human body;
step four, obtaining human body appearance parameters output by the 3D model by PCA principal component analysis, and linearly combining a plurality of parameters according to a large number of test results, wherein the model has a large number of prior knowledge of human body models with different figures, so that the model can quite clearly determine which operation should be taken at which position to approach to the perfect figure, and the slimming operation of the 3D model is realized;
step five, projecting the 3D model which is subjected to slimming to the 2D image by using the mapping relation between the 3D model and the 2D image determined by a large number of tests, and acquiring the front and back position change of the vertex of the 3D model;
and sixthly, obtaining the triangular mesh by using a Delaunay triangle subdivision algorithm, and guiding the triangular mesh to deform by using the front and back position change of the vertex of the 3D model so as to obtain a final slimming result.
Preferably, in the first step, more than 10000 photos of people with different scenes and different clothes are used, and the human body position frame, the gender and the weight rating are manually calibrated to train the human body detection network, so that the correct judgment on the position, the gender and the weight of people in daily photos, art portraits and wedding photos can be quickly realized. The reason for further enlarging the position of the human body is to obtain a better background removal effect.
Preferably, in the second step, an openpos algorithm capable of identifying 25 key parts is used for obtaining coordinates and confidence numerical values of bone key points of the object in each human body position frame, and then the openpos result is sorted according to a format of a COCO data set; and sending the skeleton key point data which is arranged according to the format of the COCO data set into a Pose2Seg background removal algorithm to finish the background removal of the human body area.
Preferably, in the third step, the data of the human body region with the background removed and the bone key point are sent into a smplify-x modeling algorithm to obtain a 3D model of the human body.
Preferably, in the fourth step, the model parameters output by the 3D modeling are obtained by using PCA principal component analysis to obtain 10 parameters related to the shape, and after testing, 3 parameters are selected for linear combination, wherein the size of the coefficients can be directly determined by the obesity rating assessed in the first step.
Preferably, in the fifth step, the model vertex mapping value before the 3D model is deformed is compared with the model vertex mapping value after the 3D model is deformed, so as to obtain the starting position of the point on the 2D picture.
In consideration of the fact that camera parameters during photographing are difficult to acquire in a picture acquisition stage, great difficulty is added for mapping a 3D model to a 2D image, and the uncertain parameters relate to the focal length of a lens, the rolling, pitching, focal position and the like of a camera, so that the value taking problem of the parameters needs to be solved when the step is completed, after a large number of tests, a camera matrix is constructed by manually setting a plurality of parameters, and the process of mapping the vertex of the 3D model to the 2D plane is realized.
Preferably, in the sixth step, in the process of guiding the triangular mesh to deform by using the front and rear position changes of the vertices of the 3D model, the Delaunay triangle is first split according to all the control points to obtain the triangular mesh, an affine matrix is obtained according to the comparison between the front and rear points of deformation, and all the vertices in the triangular mesh are guided to deform, so that the final slimming effect map is obtained. Thus, the great distortion of a certain part of the human body caused by the deformation operation is avoided.
The scheme has the following advantages or excellent effects:
the invention provides a 3D modeling-based portrait slimming automatic method, which greatly simplifies the step of manually liquefying and slimming by using software and overcomes the difficulty that the main stream automatic slimming function cannot adjust the fine granularity of each part of a body. Through the portrait slimming method based on 3D modeling, the automatic slimming scheme is provided, meanwhile, the whole picture size can be guaranteed not to be affected, and the problem that other automatic slimming functions change the picture size is solved.
Drawings
FIG. 1 is a flow chart of a 3D modeling-based portrait automatic slimming method of the present invention.
Detailed Description
As shown in fig. 1, an automatic portrait slimming method based on 3D modeling includes the following steps:
s1, acquiring all human body positions, corresponding sexes and fat-thin ratings in the picture by using a self-training human body detection algorithm;
s2, obtaining key point coordinates and confidence coefficient values of the objects in each human body position frame by using an Openpos algorithm capable of identifying 25 key parts, and sorting the Openpos results according to the format of a COCO data set;
s3, sending the key point data sorted according to the format of the COCO data set into a Pose2Seg background removal algorithm to complete background removal of the human body area;
s4, sending the human body region with the background removed and the bone key point data into a smooth-x modeling algorithm to obtain a 3D model of the human body;
s5, analyzing and obtaining appearance parameters of a human body by using principal components of PCA (principal component analysis) according to an output result of modeling, selecting several groups of parameters to perform linear combination after a large number of tests, wherein the coefficient is determined by the fat-thin grade evaluated in the first step, and determining which strategy is adopted at which part based on the prior knowledge that the model has a large number of human body models with different statures so as to obtain a 3D (three-dimensional) slimming effect which accords with the mainstream aesthetic sense of the public;
and S6, mapping the 3D model after the model is thinned to a 2D picture through an algorithm to obtain the change of the model vertex before and after the model is thinned.
And S7, inputting the coordinates after the 3D model vertex mapping, establishing a triangular mesh by using Delaunay triangulation, acquiring displacement according to the coordinates before and after the change, and driving the triangular mesh to deform. And sequentially traversing all human bodies in the picture to obtain the portrait slimming operation.
In step S1, a self-trained human body detection algorithm is used, the functions of which include determining the position of a human body, determining the sex of the human body, and determining the fat or thin degree of the human body, wherein the output of the function of detecting the position of the human body is the x-axis and y-axis starting point coordinates of the position of the human body and the width and height of the position frame of the human body, the position frame of the human body is expanded by 20% as a whole, and the expanded position frame of the human body is determined whether the image boundary is exceeded, and if the image boundary is exceeded, the image size on the coordinate axis or 0 is selected according to the position. The purpose of enlarging the size of the human body position frame is to provide a judgment space for background removal in the subsequent step; in addition, the output of the fat-thin determination function can be divided into three stages, which determine the coefficient size of each parameter in the following step S5.
In step S2, for the human body in each human body position frame, 25 skeletal key points of the human body and confidence values of the key points are obtained by using an openposition algorithm that can identify 25 important parts, where the key points cover the head, the trunk, the arms, the legs, the feet, and the like. According to the method, a key point set is recombined in a queuing mode of key points in a COCO data set, but the third element of each coordinate in the outcome of openposition is a confidence coefficient value, and the specified value in the COCO data set is a flag bit, and in the actual use process, as long as the confidence coefficient value is greater than 0, the confidence coefficient value is set to be 2 in the flag bit of the COCO data set, namely, the key points are marked and visible.
In step S3, the sorted key point information is sent to a Pose2Seg background removal network, and in this step, a part that is obviously a background in the human body position frame is roughly removed, so as to obtain a rough outline of the human body, which can ensure that the modeling process is not interfered by the background, and can obtain points to be guided to be deformed in the following steps on the human body outline.
In step S4, the skeletal key points and the human body region map with the background removed are sent to a smplify-x modeling network, and a basic model of the corresponding gender is selected by using the gender of the human body determined by the network in step S1 to accelerate modeling. After a large number of tests, the whole modeling process is limited to 4 stages, the weight value of each stage is redesigned, the modeling time which is close to 120 seconds originally is compressed to 30 seconds, and the modeling precision is guaranteed.
In step S5, 10 parameters related to the shape are obtained from the model parameters output by 3D modeling by PCA principal component analysis, and 3 parameters are selected for linear combination after a large number of tests, wherein the size of the coefficient can be directly determined by the obesity level in step S1.
In step S6, considering that it is difficult to obtain camera parameters during photographing in the picture obtaining stage, which increases the difficulty in mapping a 3D model to a 2D image, and these uncertain parameters relate to the focal length of the lens, the roll value, the pitch value, and the focal position of the camera, so that the problem of taking values of these parameters needs to be solved when this step is completed, after a large number of tests, the focus is selected to be the picture center, the roll value of the camera is set to [0,0,0], the focal length value is set to [5000,5000], the pitch value of the camera is set to be the camera pitch value in the model output parameters, and the camera matrix thus constructed is used as the camera matrix
Figure BDA0002668986880000071
The following specific mapping method from the 3D model to the 2D image is: firstly, a rolling matrix is established according to the rolling value of the camera, and the matrix is
Figure BDA0002668986880000072
Wherein r is the roll value of the camera; then, an output matrix is established according to the rolling matrix and the vertex of the 3D model, wherein the matrix is
Figure BDA0002668986880000073
The three-dimensional model is a rolling matrix, namely the position where the vertex of the 3D model can be deformed and the pitching value of the camera; and normalizing the obtained output matrix, and forming the normalized output matrix and the camera matrix point to obtain a mapping value of the model vertex on the 2D plane.
In step S7, the model vertex map value before the 3D model is deformed is compared with the model vertex map value after the 3D model is deformed, and the start position of the point on the 2D picture is obtained. In the deformation process, firstly, the Delaunay triangle subdivision is used to obtain the triangular mesh according to all the control points, then, the affine matrix is obtained according to the comparison of the points before and after deformation, all the vertexes in the triangular mesh are guided to be deformed, and therefore the final slimming effect graph is obtained.
Although the embodiments of the present invention have been described with reference to the accompanying drawings, it is not intended to limit the scope of the present invention, and it should be understood by those skilled in the art that various modifications and variations can be made without inventive changes in the technical solutions of the present invention.

Claims (7)

1. A portrait automatic slimming method based on 3D modeling is characterized by comprising the following steps:
acquiring position coordinates, sexes and fat-thin ratings of all human bodies in a picture by using a human body detection algorithm, and obtaining a human body area to be processed after amplification processing and size judgment;
secondly, acquiring skeleton key points of the character main body aiming at each human body area to be processed, and removing the background by using a background removal algorithm after the coordinates of the skeleton key points are arranged;
step three, the human body area with the background removed is matched with the coordinates of the key points of the skeleton and sent to a 3D modeling algorithm to obtain a 3D model of the human body;
step four, obtaining human body shape parameters output by the 3D model by PCA principal component analysis, and selecting the parameters to perform linear combination to realize the slimming operation of the 3D model;
fifthly, projecting the 3D model which is finished with the slimming to the 2D image according to the mapping relation from the 3D model to the 2D image, and obtaining the front and back position change of the vertex of the 3D model;
and sixthly, obtaining the triangular mesh by using a Delaunay triangle subdivision algorithm, and guiding the triangular mesh to deform by using the front and back position change of the vertex of the 3D model so as to obtain a final slimming result.
2. The 3D modeling-based portrait automatic slimming method according to claim 1, wherein in said step one, more than 10000 photos of people with different scenes and different clothes are used, and the human body detection network is trained by manually calibrating the position frame, gender and weight rating of the human body, so that the correct judgment about the position, gender and weight of people in daily photos, art portraits and wedding photos can be quickly realized.
3. The portrait automatic slimming method based on 3D modeling according to claim 1, wherein in the second step, an openpos algorithm capable of identifying 25 key parts is used to obtain coordinates and confidence values of skeletal key points of the object in each human body position frame, and then the openpos result is sorted according to a format of a COCO dataset; and sending the skeleton key point data which is arranged according to the format of the COCO data set into a Pose2Seg background removal algorithm to finish the background removal of the human body area.
4. The portrait automatic slimming method based on 3D modeling of claim 1, wherein in the third step, the human body region with the background removed and the bone key point data are sent to smplify-x modeling algorithm to obtain the 3D model of the human body.
5. The method of claim 1, wherein in the fourth step, 10 parameters related to the shape are obtained from the model parameters outputted from the 3D modeling by using PCA principal component analysis, and after testing, 3 parameters are selected for linear combination, wherein the size of the coefficients can be directly determined by the fat-thin rating assessed in the first step.
6. The 3D modeling-based portrait automatic slimming method of claim 1, wherein in the fifth step, a model vertex mapping value before the 3D model is deformed is compared with a model vertex mapping value after the 3D model is deformed, so as to obtain a starting position of a point on the 2D picture.
7. The portrait automatic slimming method based on 3D modeling according to claim 1, wherein in the sixth step, the front and back position changes of the vertices of the 3D model are used to guide the triangular mesh to be deformed, the triangular mesh is obtained by using Delaunay triangulation according to all the control points, the affine matrix is obtained by comparing the front and back points of the deformation, and all the vertices in the triangular mesh are guided to be deformed, so that the final slimming effect map is obtained.
CN202010927633.2A 2020-09-07 2020-09-07 Portrait automatic slimming method based on 3D modeling Active CN112070896B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010927633.2A CN112070896B (en) 2020-09-07 2020-09-07 Portrait automatic slimming method based on 3D modeling

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010927633.2A CN112070896B (en) 2020-09-07 2020-09-07 Portrait automatic slimming method based on 3D modeling

Publications (2)

Publication Number Publication Date
CN112070896A true CN112070896A (en) 2020-12-11
CN112070896B CN112070896B (en) 2022-05-03

Family

ID=73663684

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010927633.2A Active CN112070896B (en) 2020-09-07 2020-09-07 Portrait automatic slimming method based on 3D modeling

Country Status (1)

Country Link
CN (1) CN112070896B (en)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117974867A (en) * 2024-04-01 2024-05-03 哈尔滨工业大学(威海) Monocular face avatar generation method based on Gaussian point rendering

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107292948A (en) * 2016-04-12 2017-10-24 香港理工大学 A kind of human body modeling method, device and electronic equipment
CN109859306A (en) * 2018-12-24 2019-06-07 青岛红创众投科技发展有限公司 A method of extracting manikin in the slave photo based on machine learning
AU2019101141A4 (en) * 2019-09-30 2019-10-31 Bao, Jincheng MR Human face recognition based on Principal Component Analysis
CN111508033A (en) * 2020-04-20 2020-08-07 腾讯科技(深圳)有限公司 Camera parameter determination method, image processing method, storage medium, and electronic apparatus

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107292948A (en) * 2016-04-12 2017-10-24 香港理工大学 A kind of human body modeling method, device and electronic equipment
CN109859306A (en) * 2018-12-24 2019-06-07 青岛红创众投科技发展有限公司 A method of extracting manikin in the slave photo based on machine learning
AU2019101141A4 (en) * 2019-09-30 2019-10-31 Bao, Jincheng MR Human face recognition based on Principal Component Analysis
CN111508033A (en) * 2020-04-20 2020-08-07 腾讯科技(深圳)有限公司 Camera parameter determination method, image processing method, storage medium, and electronic apparatus

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
JIAYUAN XING: "Multi person pose estimation based on improved openpose", 《PROCEEDINGS OF 2ND GLOBAL CONFERENCE ON ROBOTICS》 *
SHENGPING ZHANG: "Skin beautification detection using sparse coding", 《2017 FIFTEENTH IAPR INTERNATIONAL CONFERENCE ON MACHINE VISION APPLICATION》 *
曾志超: "基于视频 的三维人体模型姿态与形状重构", 《中国优秀硕士学位论文全文数据库 (信息科学辑)》 *

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN117974867A (en) * 2024-04-01 2024-05-03 哈尔滨工业大学(威海) Monocular face avatar generation method based on Gaussian point rendering

Also Published As

Publication number Publication date
CN112070896B (en) 2022-05-03

Similar Documents

Publication Publication Date Title
CN106778928B (en) Image processing method and device
CN113096234B (en) Method and device for generating three-dimensional grid model by using multiple color pictures
CN109583483B (en) Target detection method and system based on convolutional neural network
JP7526412B2 (en) Method for training a parameter estimation model, apparatus for training a parameter estimation model, device and storage medium
KR101007276B1 (en) Three dimensional face recognition
JP4785880B2 (en) System and method for 3D object recognition
EP1677250B9 (en) Image collation system and image collation method
KR101635730B1 (en) Apparatus and method for generating montage, recording medium for performing the method
US10762704B2 (en) Method for establishing a deformable 3D model of an element, and associated system
CN106652015B (en) Virtual character head portrait generation method and device
US10169891B2 (en) Producing three-dimensional representation based on images of a person
CN111127668B (en) Character model generation method and device, electronic equipment and storage medium
CN106652037B (en) Face mapping processing method and device
CN113763528A (en) Method for synthesizing realistic dynamic illumination change of single image
WO2023066120A1 (en) Image processing method and apparatus, electronic device, and storage medium
WO2021099003A1 (en) Methods and system for generating 3d virtual objects
CN114693570A (en) Human body model image fusion processing method, device and storage medium
CN112070896B (en) Portrait automatic slimming method based on 3D modeling
CN108573192B (en) Glasses try-on method and device matched with human face
CN115601430A (en) Texture-free high-reflection object pose estimation method and system based on key point mapping
CN113378864B (en) Method, device and equipment for determining anchor frame parameters and readable storage medium
CN111080754A (en) Character animation production method and device for connecting characteristic points of head and limbs
JP2012113438A (en) Posture estimation apparatus and posture estimation program
CN116740281A (en) Three-dimensional head model generation method, three-dimensional head model generation device, electronic equipment and storage medium
CN111275610A (en) Method and system for processing face aging image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant