Disclosure of Invention
The technical problem to be solved by the invention is as follows: the prior art is difficult to acquire an accurate human body 3D net model in a human body dressing mode so as to facilitate 3D fitting, and provides a method for establishing a human body 3D net model and application thereof in 3D fitting.
The invention provides a method for establishing a human body 3D net model, which comprises the following steps:
s1, establishing a human body 3D net model database containing height, weight and gender information;
s2, acquiring a depth image of the current human body under the dressing condition;
s3, obtaining the height, weight and sex information of the current human body;
s4, extracting at least one human body 3D net model with the height and the weight similar to the current human body and the same sex from the human body 3D net model database as a sample net model;
and S5, acquiring the current human body net model according to the sample net model.
Preferably, the human body 3D net model in step S1 is one of a 3D point cloud net model or a 3D mesh net model containing height, weight and gender information.
Preferably, the depth image in step S2 refers to a current depth image of a local body or a current depth image of a global body.
Preferably, the step S5 of obtaining the current net model of the human body according to the sample net model includes: and when only one human body 3D net model exists in the sample net models, taking the human body 3D net model as the current human body 3D net model.
Preferably, the step S5 of obtaining the current net model of the human body according to the sample net model includes: and when two or more than two human body 3D net models are arranged in the sample net model, taking the average model of the sample net model as the current human body net model, or obtaining the accurate human body 3D net model by means of a machine learning algorithm.
Preferably, when the number of human body 3D net models in the sample net model is 2-10, the average model of the sample net model is taken as the net model of the current human body.
Preferably, when the number of human body 3D net models in the sample net model is greater than 10, an accurate human body 3D net model can be obtained by means of a machine learning algorithm, and the specific steps are as follows:
s51, selecting one of the sample net models as a standard net model;
s52, obtaining a deformation relation through machine learning according to other models in the sample net model;
and S53, based on the current human body depth image, deforming the standard net model according to the deformation relation to obtain a 3D net model of the current human body.
Preferably, the deformation relationship in step S52 includes a posture-dependent deformation relationship and/or a body-type dependent deformation relationship.
Preferably, in the step S53, the standard net model is transformed according to the transformation relationship to obtain the current net model of the human body, and the method specifically includes the following steps:
s531, establishing an energy function for measuring the consistency of the standard net model and the depth image of the current human body model;
s532, performing non-rigid deformation on the standard net model under the constraint of the energy function;
and S533, taking the standard net model after the non-rigid deformation as a 3D net model of the current human body.
Based on the method for establishing the human body 3D net model, the invention also provides a system for establishing the human body 3D net model, which comprises a memory, a storage and a processing module, wherein the memory is used for storing programs; and the processor runs the program and is used for controlling the system for establishing the human body 3D net model to execute the method for establishing the human body 3D net model.
The invention further proposes a computer-readable storage medium containing a computer program operable to cause a computer to perform the above method of creating a 3D net model of a human body.
Based on the method for establishing the human body 3D net model, the invention also provides a 3D fitting method, which comprises the following steps:
t1, creating a human body 3D net model according to the method for creating the human body 3D net model;
t2, creating a clothes model;
and T3, synthesizing the clothes model and the human body 3D net model to show the fitting effect.
Based on the method for establishing the human body 3D net model, the invention also provides a 3D fitting system, which comprises a memory for storing programs; and the processor runs the program and is used for controlling the 3D fitting system to execute the 3D fitting method.
The invention further proposes a computer-readable storage medium containing a computer program operable to cause a computer to perform the above-mentioned 3D fitting method.
Compared with the prior art, the invention has the beneficial effects that: the invention provides a method for establishing a human body 3D net model, which can purposefully extract a sample net model by establishing a human body 3D net model database and considering the influence of height, weight and gender, can simultaneously remove the influence of clothes and the influence of local characteristics caused by the height, the weight, the gender and the like, and can accurately obtain the human body net model by only utilizing a depth image of a human body under the clothes wearing condition.
In addition, based on the method for creating the human body 3D net model, the invention also provides a 3D fitting method and a 3D fitting system, and no matter whether the human body wears loose clothes or compact clothes, the dressing effect can be displayed in real time and 360 degrees by using the 3D fitting method or the system.
Detailed Description
The premise of the parametric modeling method is to measure an accurate model of a human body, however, with the emphasis of people on personal privacy, it is not the optimal choice to require a user to measure in a near-nude state. Therefore, under the condition of dressing, the accurate human body 3D net model is obtained by performing dressing calculation on the dressing model, and further 3D fitting is a better choice. The present invention will be described in further detail with reference to the following detailed description and accompanying drawings.
1. Method for establishing human body 3D net model
The method for establishing the human body 3D net model, as shown in figure 1, comprises the following steps: s1, establishing a human body 3D net model database; s2, acquiring a depth image of the current human body under the dressing condition; s3, obtaining the height, weight and sex information of the current human body; s4, selecting a sample net model from the human body 3D net model database; and S5, acquiring the current human body net model according to the sample net model. Wherein, the step S1 is to establish a human body 3D net model database, and comprises the steps of obtaining a human body 3D point cloud or a net model, measuring height, measuring weight, obtaining gender and the like; step S3, obtaining height, weight and sex information of the current human body, including the steps of height measurement, weight measurement, sex acquisition and the like; in step S4, selecting the sample net model from the human body 3D net model database means that a model with similar height, weight and sex is selected from the established human body 3D net model database as the sample net model. The above steps will be described in detail below.
S1, establishing a human body 3D net model database
The establishment of the human body 3D model database is a precondition for obtaining a human body parameterized model, and the type of the model in the database can also influence the quality of the final parameterized model.
The number of the net models in the model database is as large as possible, the data is as comprehensive as possible, the net models not only contain the height, weight and sex information of the human body, but also contain the human body model data of the same human body under different postures, the human body model data of a plurality of different human bodies under similar postures, and the different human body model data with similar height, weight and same sex are respectively used for reflecting the difference of the human body under different postures, the difference of different human bodies on the body types and the difference of different human bodies on the local detail characteristics.
Establishing a human body 3D net model database, wherein each net model is obtained under the condition of not wearing clothes or tights, and the condition of only wearing underwear is referred to as the non-wearing clothes. The database is established for the subsequent dressing model to perform dressing calculation.
In this embodiment, the 3D net model in the database includes height, weight and gender information in addition to the 3D point cloud or mesh net model itself. Thus, establishing the net model database mainly includes the aspects of human body 3D net model measurement, height measurement, weight measurement and gender confirmation.
Human body 3D net model measuring method
Current methods for 3D measurements are broadly divided into laser scanner based and depth camera based methods. The laser scanner has high precision, but is expensive in manufacturing cost and relatively slow in scanning speed, and is mainly used for 3D measurement of small rigid objects; the depth camera-based method is a commonly used human body measurement method at present, and in the present embodiment, a depth camera-based method is preferentially selected, and a depth camera based on a structured light triangulation method is further selected for measurement of a human body 3D net model.
In other embodiments, laser scanners may be used for the measurement of the 3D net model of the human body, or depth cameras based on time-of-flight methods or binocular vision principles may be used.
The depth camera based on the structured light trigonometry utilizes a laser projector to project coded standard structured light patterns into a space, the standard structured light patterns are modulated according to different target depths in the space, the difference between the modulated structured light image and the standard structured light patterns is obtained through algorithms such as image correlation, and the like, and the depth image of the whole target space can be solved by establishing the relation between the difference and the target depths according to the structured light trigonometry.
Generally, it is difficult to obtain all human body information from one image, and it is necessary to obtain depth images of each part of a human body, and then obtain the whole human body 3D point cloud data after fusion by a registration algorithm. The 3D point cloud data acquired by the depth camera is generally not directly available as phantom data, and requires some pre-processing steps. Generally, the method comprises the steps of image segmentation, denoising, gridding, corresponding relation establishment and the like.
And (5) image segmentation. Therefore, the depth image obtained by the depth camera generally includes other background components besides the human body part, and the background removal by the image segmentation algorithm becomes a necessary step. Due to the uniqueness of the depth image data, namely the depth distance of an object represented by each pixel value, a simple image segmentation algorithm, namely a threshold method, can effectively remove the background. Specifically, a reasonable threshold value for distinguishing the human body from the background is set, the pixel value belonging to the background part is zeroed (or maximized), and the pixel value belonging to the human body part is retained.
And (5) denoising the image. The acquired 3D point cloud data inevitably has noise (namely outliers), and holes appear due to shielding among human body parts, and the smoothness of the point cloud data is poor. Therefore, the purpose of image denoising is to remove outliers on one hand and smooth and fill holes in point cloud data on the other hand.
And (6) gridding. In specific applications, such as model transformation and animation, the processing for only point clouds is complicated, and thus the correlation between points is not reflected in the 3D point cloud. The 3D network model retains the point clouds and increases the topological relation among the point clouds, and particularly has great advantages in deformation processing. Therefore, it is necessary to mesh the 3D point cloud model into a 3D mesh model. The mesh may be in the form of triangles, polygons, etc., and a triangular mesh model is commonly used.
And establishing a corresponding relation. Since different people have different body types and postures, the amount of 3D point cloud data obtained by the depth camera is different, and it is difficult to perform subsequent processing. It is necessary to establish correspondence relationships for all human body models when establishing the database. Specifically, a 3D point cloud data with high quality is selected as a reference. And for the current human body 3D net model, establishing a corresponding relation between points of the current human body 3D net model and the points by using a rigid registration algorithm or a non-rigid registration algorithm, and taking the corresponding relation as a part of the current human body 3D net model.
Body weight measurement
And measuring the body weight of each human body by using the body weight scale to obtain the body weight of the human body.
Height measurement
In this embodiment, the height measurement is performed accurately. Firstly, skeleton extraction is carried out on 3D human body point cloud data, or a 3D human body is divided into a plurality of semantic parts (a head, an upper body and legs), and then the lengths of all the parts are added to obtain the height of the human body.
In other embodiments, height measurement may be performed using conventional dimensional measurement methods, or height measurement may be performed directly using 3D body point cloud or mesh data. It should be noted that when the human body is in different postures, the height estimation cannot be performed by using the difference between the highest point and the lowest point in the point cloud or the grid data in a generalized manner.
Gender acquisition
In this embodiment, gender acquisition employs an automatic identification method. Namely, a color image of the human face is extracted by using the color image of the human body model, and the extracted color image is input into a trained gender classifier for judgment. According to the type of the classifier, the processing modes are different, and generally, Principal Component Analysis (PCA) is firstly carried out on the color image of the human face, so that the recognition efficiency can be improved.
In other embodiments, the setting may be made manually with human assistance.
The human body 3D net model in the model database is as comprehensive as possible, and on one hand, the human body net model with the height, the weight and the gender as much as possible is covered; on the other hand, the height and the weight are similar, and enough human body 3D net model data are needed for the human bodies with the same gender. The former aims to reflect the difference of human body in different postures and body types more comprehensively, and the latter aims to reflect the difference of human body in local detail characteristics more accurately.
S2, acquiring the depth image of the current human body under the dressing condition
Generally, when a current human body is subjected to 3D fitting, a net model thereof needs to be obtained first, and there are two assumptions: firstly, the current human body is not in a 3D net model database, otherwise, the net model can be directly extracted; secondly, the current human body is fitting under the condition of ordinary dressing, that is, the human body model corresponding to the depth image acquired by the depth camera includes the influence of clothes, and the model is larger than the human body 3D net model, and the net model is obtained according to the dressing model in the invention.
The current human body is collected to a depth image by a depth camera under the dressing condition, and the depth image only containing the human body is obtained after image segmentation and denoising. The depth image can be a local or global depth image of a human body, and for the global depth image, a plurality of depth images are often registered and fused.
S3 obtaining the height, weight and sex information of the current human body
The weight, sex and height information of the human body can be obtained by the method.
S4 selecting a sample net model from the human body 3D net model database
After the depth image and the relevant information of the current human body are obtained, models with similar height, weight and sex can be selected from the established human body 3D net model database to serve as sample net models. The number of sample net models is at least 1. When the number of the samples is multiple, the local characteristic difference of the height, the weight and the gender of the crowd can be well reflected, and a more accurate human body 3D net model can be obtained in subsequent processing. S5 obtaining the current human body net model according to the sample net model
According to the difference of the number of the sample net models, the current human body 3D net model obtaining modes are different.
(1) When there is only one net model of the sample, the net model can be directly used as the 3D net model of the current human body. Since height, weight, and gender are consistent, this approximation is also acceptable in the case of only one sample net model.
(2) When the number of the sample net models is small and is 2-10, the average value of all net models can be taken as the 3D net model of the current human body. Although this averaging method does not accurately reflect the actual body shape characteristics of the current human body, the degree of approximation is higher than that in the first case.
(3) When the number of the sample net models is more than 10, the accurate human body 3D net model can be obtained by means of a machine learning algorithm. The specific algorithm flow is shown in fig. 2, and includes the following steps:
s51, selecting one of the sample net models as a standard net model, and generally selecting a human body 3D net model with good point cloud or grid quality as the standard net model;
and S52, obtaining the deformation relation through machine learning according to other net models in the sample net models. The deformation relationship here includes a deformation relationship determined by the posture and a deformation relationship determined by the body type. In general, for the deformation relationship determined by the posture, that is, for all other net models in the sample net model, the standard net model can be calculated according to the posture parameters of the other net models through the deformation relationship to obtain the net model consistent with the other net models. For the deformation relation determined by the body type, namely for all other net models in the sample net model, the standard net model can be calculated according to the body type parameters of the other net models through the deformation relation to obtain the net model consistent with the other net models. The goal of machine learning is to obtain the posture deformation relationship and the body shape deformation relationship by learning all the net models in the sample net models.
And S53, based on the current human body depth image, deforming the standard net model according to the deformation relation to obtain the net model of the current human body. Specifically, as shown in fig. 3, the method includes the following steps:
and S531, establishing an energy function for measuring the consistency of the standard net model and the depth image of the current human body model. The energy function generally includes a similarity function in terms of distance, a smoothness function of the model, and a local feature function.
And S532, performing non-rigid deformation on the standard net model under the constraint of the energy function. In this step, substantially, initial values of the posture parameters and the body shape parameters are set under the constraint of the energy function, iterative deformation is performed based on the deformation relationship obtained in step S52, and the deformation is terminated when the energy function reaches a preset optimum value (generally, a minimum value). Deformation is referred to herein as non-rigid deformation.
And S533, taking the standard net model after the non-rigid deformation as the net model of the current human body. The standard net model after deformation can be exactly consistent with the current human body partial or whole depth image, and therefore the standard net model can be used as the net model of the current human body.
2. System for establishing human body 3D net model
In this embodiment, a system for creating a 3D net model of a human body includes a memory for storing a program; and the processor runs the program and is used for controlling the system for establishing the human body 3D net model to execute the method for establishing the human body 3D net model.
In other embodiments, the system for creating a net 3D model of a human body may also be a computer readable storage medium containing a computer program operable to cause a computer to perform the method for creating a net 3D model of a human body as described above.
3. Application of human body 3D net model in 3D fitting
The 3D fitting method, as shown in fig. 4, includes the following steps: t1: creating a human body 3D net model; t2: creating a clothes model; t3: and (4) synthesizing the clothes model and the human body 3D net model to show the fitting effect. Generally, a standard standing mannequin is first fitted, which can be considered as a static fitting; secondly, the real-time fitting when the actual human posture changes can be regarded as dynamic fitting. Wherein a dynamic fitting is actually an extension of a static fitting over time. The following description will therefore be mainly made of static fitting.
At present, a relatively mature clothes simulation model is a mass point-spring model, and after the clothes simulation model is established, the clothes and a human body 3D net model need to be registered. Generally, the highest point on the back of the clothes and the center of the back neck of the human body are considered, so that the preliminary registration of the clothes and the human body model can be realized; and then local registration of each part is realized according to the current skeleton information of the human body. After registration, calculation of force of mass points, clothes collision detection and the like can be carried out, so that a more real clothes display effect can be simulated.
In the subsequent real-time display process, the real-time 3D fitting can be realized only by identifying the skeleton information of the human body and then carrying out local registration according to the skeleton information.
3. 3D fitting system
In this embodiment, the 3D fitting system includes a memory for storing a program; and the processor runs the program and is used for controlling the 3D fitting system to execute the 3D fitting method.
In other embodiments, the 3D fitting system may also be a computer readable storage medium containing a computer program operable to cause a computer to perform the above-described 3D fitting method.
The specific embodiment provides a method for creating a human body 3D net model based on a depth camera, which comprises the steps of firstly creating a human body 3D net model database containing height, weight and gender information, then acquiring a depth image of a current human body under a dressing condition, acquiring the height, weight and gender information of the current human body, then extracting at least one human body 3D net model which is close to the height and weight of the current human body and has the same gender as the current human body from the human body 3D net model database to serve as a sample net model, and finally acquiring the 3D net model of the current human body according to the sample net model. In the specific embodiment, the sample net model is extracted in a targeted manner by establishing the human body 3D net model database and considering the influences of height, weight and gender, so that the influences of clothes and the influences of local characteristics caused by the height, the weight, the gender and the like can be removed simultaneously, and the human body 3D net model can be accurately acquired only by using the depth image of the human body under the clothes wearing condition.
In addition, based on the method for creating the human body 3D net model, the present embodiment further provides a 3D fitting method and a 3D fitting system, and no matter whether the human body wears loose clothes or compact clothes, the dressing effect can be displayed in real time and 360 degrees by using the 3D fitting method or the system of the present embodiment.
The foregoing is a more detailed description of the invention in connection with specific preferred embodiments and it is not intended that the invention be limited to these specific details. For those skilled in the art to which the invention pertains, several equivalent substitutions or obvious modifications can be made without departing from the spirit of the invention, and all the properties or uses are considered to be within the scope of the invention.