CN111612887A - Human body measuring method and device - Google Patents

Human body measuring method and device Download PDF

Info

Publication number
CN111612887A
CN111612887A CN202010367206.3A CN202010367206A CN111612887A CN 111612887 A CN111612887 A CN 111612887A CN 202010367206 A CN202010367206 A CN 202010367206A CN 111612887 A CN111612887 A CN 111612887A
Authority
CN
China
Prior art keywords
human body
point cloud
dimensional model
cloud data
preliminary
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
CN202010367206.3A
Other languages
Chinese (zh)
Other versions
CN111612887B (en
Inventor
户磊
蔡麟
李廷照
张举勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hefei Dilusense Technology Co Ltd
Original Assignee
Beijing Dilusense Technology Co Ltd
Hefei Dilusense Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beijing Dilusense Technology Co Ltd, Hefei Dilusense Technology Co Ltd filed Critical Beijing Dilusense Technology Co Ltd
Priority to CN202010367206.3A priority Critical patent/CN111612887B/en
Publication of CN111612887A publication Critical patent/CN111612887A/en
Application granted granted Critical
Publication of CN111612887B publication Critical patent/CN111612887B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00—Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01B—MEASURING LENGTH, THICKNESS OR SIMILAR LINEAR DIMENSIONS; MEASURING ANGLES; MEASURING AREAS; MEASURING IRREGULARITIES OF SURFACES OR CONTOURS
    • G01B11/00—Measuring arrangements characterised by the use of optical techniques
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10028—Range image; Depth image; 3D point clouds
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning

Abstract

The embodiment of the invention provides a human body measuring method and device. The human body measurement method comprises the steps of obtaining multi-view image data of a human body to be measured, and obtaining human body posture information based on a human body posture estimation library; converting the multi-view image data into point cloud data through internal reference of a depth camera; acquiring a rigid transformation matrix and a preliminary human body three-dimensional model according to the coefficient of the preliminary human body three-dimensional model and the point cloud data; and acquiring an accurate human body three-dimensional model and measuring the accurate human body three-dimensional model based on the point cloud data, the human body posture information and the preliminary human body three-dimensional model. The invention solves the problems of high requirement on environment, immobility and low measurement precision in the process of measuring the three-dimensional human body.

Description

Human body measuring method and device
Technical Field
The invention relates to the field of computer vision, in particular to a human body measuring method and device.
Background
At present, the human body size is required to be used in many fields, such as the fields of security protection, virtual fitting and the like. The human body size measurement is mainly through three-dimensional human body measurement based on measuring instrument, and the precision is higher for manual measurement precision, and stability is stronger, and the precision can reach the millimeter level at most, is the real three-dimensional data of object. However, the current market of anthropometric devices is generally expensive, requires professional training for use, and is not suitable for consumer-grade markets. Most instruments for human body three-dimensional measurement appearing in the current market are based on large-scale multi-camera acquisition equipment, have extremely high requirements on the installation precision of cameras, and cannot move or shake after the system is installed.
At present, for a three-dimensional human body modeling method, the most common modeling method is to collect human body data in different directions by means of multiple cameras, splice the data, and recover a complete three-dimensional human body model. However, the method has high requirements on the position of a place, illumination and a camera, and has no accurate position basis when measuring the size of a key part of a human body, and the measurement position is determined mainly by timely detection, so that certain influence is caused on the measurement precision.
Therefore, the problems of high environmental requirement, immobility, low measurement precision and the like in the three-dimensional human body measurement become problems to be solved urgently.
Disclosure of Invention
The embodiment of the invention provides a human body measuring method and device, which are used for solving the problems of high requirement on environment, immobility and low measuring precision in the three-dimensional human body measuring process in the prior art.
In a first aspect, an embodiment of the present invention provides a human body measurement method, including:
acquiring multi-view image data of a human body to be measured, and acquiring human body posture information based on a human body posture estimation library;
converting the multi-view image data into point cloud data through internal reference of a depth camera;
acquiring a rigid transformation matrix and a primary three-dimensional human body model according to the coefficient of the primary three-dimensional human body model and the point cloud data;
and acquiring an accurate human body three-dimensional model based on the point cloud data, the human body posture information and the preliminary human body three-dimensional model, and realizing the measurement of the size of the key part based on the accurate human body three-dimensional model.
Optionally, the converting the multi-view image data into point cloud data through internal parameters of a depth camera further includes:
projecting each pixel in the multi-view image to a world coordinate system according to an internal reference of a depth camera;
and reconstructing the coordinates of each pixel in the world coordinate system into point cloud data by using a Poisson reconstruction algorithm.
Optionally, the obtaining a rigid transformation matrix and a preliminary three-dimensional human body model according to the coefficients of the preliminary three-dimensional human body model and the point cloud data further includes:
acquiring coarse grain optimization residual errors according to different human body posture information in multi-view image data;
and minimizing the coarse-grained optimization residual error, acquiring a rigid transformation matrix of the point cloud data, and generating a preliminary human body three-dimensional model according to the coefficient of the preliminary human body three-dimensional model and the point cloud data.
Optionally, the obtaining a coarse-grained optimization residual according to different human posture information in the multi-view image data specifically includes:
calculating the sum of regular terms of different human posture information in all the multi-view image data E1;
and summing the product of the coefficient and the weight of the preliminary human body three-dimensional model and the E1 to obtain a coarse-granularity optimization residual error.
Optionally, the obtaining an accurate three-dimensional human body model based on the point cloud data, the human body posture information, and the preliminary three-dimensional human body model, and implementing measurement of the critical dimension based on the accurate three-dimensional human body model specifically include:
acquiring matching points of the point cloud data according to the preliminary human body three-dimensional model, and constructing all the matching points into matching point pairs;
optimizing the coefficient of the preliminary human body three-dimensional model and a rigid transformation matrix of the point cloud data according to the matching point pairs;
and obtaining an accurate human body three-dimensional model according to the optimized coefficient of the initial human body three-dimensional model and the rigid transformation matrix of the point cloud data, and realizing critical position size measurement based on the accurate human body three-dimensional model.
Optionally, the obtaining of the matching points of the point cloud data according to the preliminary human body three-dimensional model and constructing all the matching points into matching point pairs specifically includes:
screening out matching points by calculating the distance between the point cloud data and the preliminary human body three-dimensional model and the direction vector included angle;
and forming a matching point pair by the point cloud data and the matching points of the point cloud data on the preliminary human body three-dimensional model.
Optionally, the optimizing, according to the matching point pairs, coefficients of the preliminary human body three-dimensional model and a rigid transformation matrix of the point cloud data specifically includes:
fixing a rigid transformation matrix of the point cloud data according to the matching point pairs to obtain a coefficient of a preliminary human body three-dimensional model;
and updating the matching point pairs according to the coefficient of the preliminary human body three-dimensional model, fixing the coefficient of the preliminary human body three-dimensional model, and acquiring a rigid transformation matrix of the updated point cloud data.
Optionally, the fixing the rigid transformation matrix of the point cloud data according to the matching point pairs to obtain a coefficient of a preliminary human body three-dimensional model specifically includes:
fixing a rigid transformation matrix of the point cloud data according to the matching point pairs;
and obtaining the coefficient of the preliminary human body three-dimensional model by minimizing the error between the matching point and the point cloud data.
In a second aspect, an embodiment of the present invention provides a human body measuring apparatus, including:
an acquisition module: the system comprises a human body posture estimation library, a multi-view image data acquisition module, a human body posture estimation module and a human body posture estimation module, wherein the multi-view image data acquisition module is used for acquiring multi-view image data of a human body to be measured and acquiring human body posture information based on the human body posture estimation;
a conversion module: the system comprises a depth camera, a multi-view image data acquisition unit, a data acquisition unit and a data processing unit, wherein the depth camera is used for acquiring multi-view image data;
a processing module: the system comprises a point cloud data acquisition unit, a rigidity transformation matrix acquisition unit, a point cloud data acquisition unit and a data processing unit, wherein the point cloud data acquisition unit is used for acquiring rigidity transformation matrixes and a primary human body three-dimensional model according to coefficients of the primary human body three-dimensional model and the point cloud data;
a measurement module: the device is used for obtaining an accurate human body three-dimensional model based on the point cloud data, the human body posture information and the preliminary human body three-dimensional model, and realizing the measurement of the size of the key part based on the accurate human body three-dimensional model.
Optionally, the processing module is further configured to:
acquiring coarse grain optimization residual errors according to different human body posture information in multi-view image data;
and minimizing the coarse-grained optimization residual error, acquiring a rigid transformation matrix of the point cloud data, and generating a preliminary human body three-dimensional model according to the coefficient of the preliminary human body three-dimensional model and the point cloud data.
According to the human body measurement method provided by the embodiment of the invention, the coefficient of the preliminary human body three-dimensional model and the rigid transformation matrix of point cloud data are optimized, so that the preliminary human body three-dimensional model is optimized, the three-dimensional human body model with high precision is generated, the measurement of the size of the key part is realized based on the accurate human body three-dimensional model, and the human body measurement method has the advantages of low requirement on measurement environment, good mobility and high measurement precision.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and those skilled in the art can also obtain other drawings according to the drawings without creative efforts.
Fig. 1 is a schematic flow chart of a human body measurement method according to an embodiment of the present invention;
fig. 2 is a schematic structural diagram of a body measurement device according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the drawings in the embodiments of the present invention, and it is obvious that the described embodiments are some, but not all, embodiments of the present invention. All other embodiments, which can be obtained by a person skilled in the art without any inventive step based on the embodiments of the present invention, are within the scope of the present invention.
Fig. 1 is a schematic flow chart of a human body measurement method provided in an embodiment of the present invention, including:
acquiring multi-view image data of a human body to be measured, and acquiring human body posture information based on a human body posture estimation library;
specifically, the multi-view image data is image data of a human body to be measured photographed from different views, and the human body posture information is posture estimation of the human body image data of the human body to be measured from different views according to a human body posture library.
Converting the multi-view image data into point cloud data through internal reference of a depth camera;
specifically, multi-view image data (m, n) is converted into point cloud data (W)x，Wy，Wz) The formula of (1) is:
Figure BDA0002476918370000051
wherein f isx，fyFocal lengths of the depth camera in the x and y directions, c, respectivelyx，cyThe optical centers of the depth camera in the x and y directions, respectively.
Acquiring a rigid transformation matrix and a primary three-dimensional human body model according to the coefficient of the primary three-dimensional human body model and the point cloud data;
and acquiring an accurate human body three-dimensional model based on the point cloud data, the human body posture information and the preliminary human body three-dimensional model, and realizing the measurement of the size of the key part based on the accurate human body three-dimensional model.
The human body three-dimensional model is generated by training through a deep learning VAE method, wherein beta, theta and D respectively represent parameters related to human body identity, parameters related to human body action and offset of each vertex. The human model can be generated by sending the parameters β, θ, D into the network in the deep learning VAE, as follows:
Model＝HumanNet(β、θ、D、T)
wherein: HumanNet represents a trained human body VAE parameterized template;
t represents a mean model;
the Model represents the generated human body Model.
The method comprises the steps of calibrating a position to be measured on the accurate human body three-dimensional model, and calculating the shortest geodesic distance between two points on the triangular mesh model by using a geodesic distance calculation method to serve as the final distance output to realize the measurement of the size of the key part of the accurate human body three-dimensional model.
As an embodiment of the present invention, the converting the multi-view image data into point cloud data through internal reference of a depth camera further includes:
projecting each pixel in the multi-view image to a world coordinate system according to an internal reference of a depth camera;
and reconstructing the coordinates of each pixel in the world coordinate system into point cloud data by using a Poisson reconstruction algorithm.
Specifically, point cloud data converted from multi-view image data by a conversion formula is connected into pieces in a world coordinate system to form a point cloud.
As an embodiment of the present invention, the obtaining a rigid transformation matrix and a preliminary three-dimensional human body model according to the coefficients of the preliminary three-dimensional human body model and the point cloud data further includes:
acquiring coarse grain optimization residual errors according to different human body posture information in multi-view image data;
and minimizing the coarse-grained optimization residual error, acquiring a rigid transformation matrix of the point cloud data, and generating a preliminary human body three-dimensional model according to the coefficient of the preliminary human body three-dimensional model and the point cloud data.
Specifically, the formula for calculating the coarse-grained optimization residual is as follows:
Eloss＝Egeo+wβ*Eβ+wθ*Eθ；
wherein:
the calculation error term is:
Figure BDA0002476918370000061
wherein n represents the number of input fields of view, JkThe orientation information in the kth field of view is shown, and i represents the orientation information i. Respectively recording as: p _ Ji、Q_Ji；
The regularization term of coefficient β, denoted as Eβ＝||β||；
Regular term of coefficient theta, denoted as Eθ＝||θ||；
WβAnd W theta is used for adjusting each weight coefficient.
Calculating by optimizing
Figure BDA0002476918370000062
Output solving coefficients of β, theta, D,
Figure BDA0002476918370000063
As an embodiment of the present invention, the obtaining a coarse-grained optimization residual according to different human body pose information in multi-view image data specifically includes:
calculating the sum of regular terms of different human posture information in all the multi-view image data E1;
and summing the product of the coefficient and the weight of the preliminary human body three-dimensional model and the E1 to obtain a coarse-granularity optimization residual error.
In particular, the amount of the solvent to be used,
Figure BDA0002476918370000071
the formula for calculating the coarse grain optimization residual error is as follows:
Eloss＝Egeo+wβ*Eβ+wθ*Eθ；
and summing the product of the coefficient and the weight of the preliminary human body three-dimensional model and the E1 to obtain coarse-grained optimization residual error.
As an embodiment of the present invention, the obtaining an accurate three-dimensional human body model based on the point cloud data, the human body posture information, and the preliminary three-dimensional human body model, and implementing the measurement of the critical dimension of the part based on the accurate three-dimensional human body model specifically includes:
acquiring matching points of the point cloud data according to the preliminary human body three-dimensional model, and constructing all the matching points into matching point pairs;
optimizing the coefficient of the preliminary human body three-dimensional model and a rigid transformation matrix of the point cloud data according to the matching point pairs;
and obtaining an accurate human body three-dimensional model according to the optimized coefficient of the initial human body three-dimensional model and the rigid transformation matrix of the point cloud data, and realizing critical position size measurement based on the accurate human body three-dimensional model.
Specifically, matching points of the point cloud data are found in the preliminary human body three-dimensional model, and matching point pairs are generated by the point cloud data and the matching points. And optimizing the coefficients of the preliminary human body three-dimensional model and the rigid transformation matrix of the point cloud data by firstly fixing point cloud transformation parameters and solving beta, theta and D through an objective function.
The method specifically comprises the following steps: calculating fine-grained optimization residual errors, wherein the calculation formula is as follows:
Eloss＝Egeo+Wsmooth*Esmooth+Wcontour*Econtour+Wflor*Eflor+Wother*Eother+Wβ*Eβ+Wθ*Eθ+WD*ED
wherein, the error term between the matching points is recorded as:
Figure BDA0002476918370000081
k belongs to (0, n) point cloud information of n different view angles;
pi∈Vkrepresenting the ith matching point pair in the kth point cloud, wherein the corresponding matching point pair is as follows: model point PiMatching point Q to the point cloudi；
pj∈VkRepresenting the jth matching point pair in the kth point cloud, wherein the corresponding matching point pair is as follows: point cloud point PjTo model matching point Qi；
Figure BDA0002476918370000086
A rigid transformation matrix representing the Kth point cloud data;
error calculation using the smoothing (or other smoothing term):
Figure RE-GDA0002536927170000102
wherein: m represents the number of points P in the vicinity, P _ liRepresenting the ith point of proximity.
Calculating the contour matching error:
Figure BDA0002476918370000083
wherein P isiAnd QiRepresenting corresponding contour points, and m representing the number of the contour points;
calculating an optical flow matching error:
Figure BDA0002476918370000084
wherein P isiAnd QiRepresenting the corresponding optical flow coefficient, m representing the number of points;
a priori knowledge errorItem EotherPlacing model results with uncontrollable modeling parameters by using prior knowledge;
the regularization term of coefficient β, denoted Fβ＝||β||；
Regular term of coefficient theta, denoted as Eθ＝||θ||；
Regular term of coefficient D, denoted ED＝||D||；
Wβ、Wθ、WD、Wflor、
Figure BDA0002476918370000085
Used for adjusting each weight coefficient;
calculating by optimization solution:
Figure BDA0002476918370000091
outputting solving coefficients β, theta and D;
further, β, theta and D are fixed, and SVD optimization solution is used
Figure BDA0002476918370000092
The method specifically comprises the following steps:
firstly, searching the corresponding relation between the human body model and the point clouds, namely transforming each point cloud according to the current rotation and translation, and then, carrying out transformation on each point cloud point qiFinding a match P on a modeliAnd for each point P on the modeljFinding matches q on all point cloudsj。
The optimization objective function is:
Figure BDA0002476918370000093
wherein, VkRefers to the set of all points in the point cloud k. K refers to the kth point cloud;
(k＝0，1，...N)；
pi∈Vkrepresenting the ith matching point pair in the kth point cloud, wherein the corresponding matching point pair is as follows: model point PiTo the closest point Q of the point cloudi；
pi∈VkRepresenting the jth matching point pair in the kth point cloud, wherein the corresponding matching point pair is as follows: point cloud point PiTo the nearest point Q of the modeli；
Then, the correspondence relationship is fixed, and the rotation R is optimized for each point cloud k (k is 0, 1, … N)kAnd translation Tk. The objective function can be further simplified as:
Figure BDA0002476918370000094
wherein: m isiFinger model point PiThe number of occurrences in the original objective function;
rithe average value of all corresponding model points in the original objective function is referred to;
finally, SVD decomposition is carried out, and an optimization result B is outputk＝VUTAccording to RkCalculating Tk。
As an embodiment of the present invention, the obtaining of the matching points of the point cloud data according to the preliminary three-dimensional human body model and constructing all the matching points into matching point pairs specifically includes:
screening out matching points by calculating the distance between the point cloud data and the preliminary human body three-dimensional model and the direction vector included angle;
and forming a matching point pair by the point cloud data and the matching points of the point cloud data on the preliminary human body three-dimensional model.
Specifically, find point cloud point QjMatching point P to modeljAnd model point PiMatching point Q to the point cloudi. Then by calculating QiAnd Pi，QjAnd PjThe matching points are screened according to the distance between the matching points and the included angle of the direction vector;
Figure BDA0002476918370000101
wherein dis represents the distance between the cloud point and the matching point, and α represents the included angle between the two direction vectors;
Figure BDA0002476918370000102
respectively, are the direction vectors of the P and Q points.
As an embodiment of the present invention, the optimizing the coefficients of the preliminary human three-dimensional model and the rigid transformation matrix of the point cloud data according to the matching point pairs specifically includes:
fixing a rigid transformation matrix of the point cloud data according to the matching point pairs to obtain a coefficient of a preliminary human body three-dimensional model;
and updating the matching point pairs according to the coefficient of the preliminary human body three-dimensional model, fixing the coefficient of the preliminary human body three-dimensional model, and acquiring a rigid transformation matrix of the updated point cloud data.
Specifically, alternate repetitive fixation
Figure BDA0002476918370000103
Solving β theta, D and fixed β theta, D through objective function, and optimizing solving through SVD decomposition
Figure BDA0002476918370000104
Obtain β, theta, D and D meeting the error requirement
Figure BDA0002476918370000105
As an embodiment of the present invention, the fixing a rigid transformation matrix of the point cloud data according to the matching point pairs to obtain a coefficient of a preliminary human body three-dimensional model specifically includes:
fixing a rigid transformation matrix of the point cloud data according to the matching point pairs;
and obtaining the coefficient of the preliminary human body three-dimensional model by minimizing the error between the matching point and the point cloud data.
Specifically, matching points of the point cloud data are found in the preliminary human body three-dimensional model, and matching point pairs are generated by the point cloud data and the matching points. The coefficient of the preliminary human body model and the rigid transformation matrix of the point cloud data are optimized by firstly fixing point cloud transformation parameters and solving beta, theta and D through an objective function.
The method specifically comprises the following steps: calculating fine-grained optimization residual errors, wherein the calculation formula is as follows:
Eloss＝Egeo+Wsmooth*Esmooth+Wcontour*Econtour+Wflor*Eflor+Wother*Eother+Wβ*Eβ+Wθ*Eθ+WD*ED
wherein, the error term between the matching points is recorded as:
Figure BDA0002476918370000111
k belongs to (0, n) point cloud information of n different view angles;
pi∈Vkrepresenting the ith matching point pair in the kth point cloud, wherein the corresponding matching point pair is as follows: model point PiMatching point Q to the point cloudi；
pj∈VkRepresenting the jth matching point pair in the kth point cloud, wherein the corresponding matching point pair is as follows: point cloud point PiTo model matching point Qi；
Figure BDA0002476918370000112
A rigid transformation matrix representing the Kth point cloud data;
error calculation using the smoothing (or other smoothing term):
Figure RE-GDA0002536927170000142
wherein: m represents the number of points P in the vicinity, P _ liRepresenting the ith point of proximity.
Calculating the contour matching error:
Figure BDA0002476918370000114
wherein P isiAnd QiRepresenting corresponding contour points, and m representing the number of the contour points;
calculating an optical flow matching error:
Figure BDA0002476918370000115
wherein P isiAnd QiRepresenting the corresponding optical flow coefficient, m representing the number of points;
a priori knowledge error term EotherPlacing model results with uncontrollable modeling parameters by using prior knowledge;
the regularization term of coefficient β, denoted as Eβ＝||β||；
Regular term of coefficient theta, denoted as Eθ＝||θ||；
Regular term of coefficient D, denoted ED＝||D||；
Wβ、Wθ、WD、Wflor、
Figure BDA0002476918370000121
Used for adjusting each weight coefficient;
calculating by optimization solution:
Figure BDA0002476918370000122
the solution coefficients β, θ, D are output.
Fig. 2 is a schematic structural diagram of a human body measurement device provided in an embodiment of the present invention, including:
an acquisition module: the system comprises a human body posture estimation library, a multi-view image data acquisition module, a human body posture estimation module and a human body posture estimation module, wherein the multi-view image data acquisition module is used for acquiring multi-view image data of a human body to be measured and acquiring human body posture information based on the human body posture estimation;
specifically, the multi-view image data is image data of a human body to be measured photographed from different views, and the human body posture information is posture estimation of the human body image data of the human body to be measured from different views according to a human body posture library.
A conversion module: the system comprises a depth camera, a multi-view image data acquisition unit, a data acquisition unit and a data processing unit, wherein the depth camera is used for acquiring multi-view image data;
specifically, multi-view image data (m, n) is converted into point cloud data (W)x，Wy，Wz) The formula of (1) is:
Figure BDA0002476918370000123
wherein f isx，fyFocal lengths of the depth camera in the x and y directions, c, respectivelyx，cyThe optical centers of the depth camera in the x and y directions, respectively.
A processing module: the system comprises a point cloud data acquisition unit, a rigidity transformation matrix acquisition unit, a point cloud data acquisition unit and a data processing unit, wherein the point cloud data acquisition unit is used for acquiring rigidity transformation matrixes and a primary human body three-dimensional model according to coefficients of the primary human body three-dimensional model and the point cloud data;
a measurement module: the device is used for obtaining an accurate human body three-dimensional model based on the point cloud data, the human body posture information and the preliminary human body three-dimensional model, and realizing the measurement of the size of the key part based on the accurate human body three-dimensional model.
Specifically, the human three-dimensional model is generated by training through a deep learning VAE method, where β, θ, and D respectively represent a parameter related to human identity, a parameter related to human motion, and an offset of each vertex. The human body model can be generated by sending the parameters beta, theta and D into a network in the deep learning VAE, and the specific steps are as follows:
Model＝HumanNet(β、θ、D、T)
wherein: HumanNet represents a trained human body VAE parameterized template;
t represents a mean model;
the Model represents the generated human body Model.
The measurement module realizes the measurement of the size of the key part of the accurate human body three-dimensional model by calibrating the position to be measured on the accurate human body three-dimensional model and using a geodesic distance calculation method to calculate the shortest geodesic distance between two points on the triangular grid model as the final distance output method.
As an embodiment of the present invention, the processing module is further configured to:
acquiring coarse grain optimization residual errors according to different human body posture information in multi-view image data;
and minimizing the coarse-grained optimization residual error, acquiring a rigid transformation matrix of the point cloud data, and generating a preliminary human body three-dimensional model according to the coefficient of the preliminary human body three-dimensional model and the point cloud data.
The above-described embodiments of the apparatus are merely illustrative, and the units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the solution of the present embodiment. One of ordinary skill in the art can understand and implement it without inventive effort.
Through the above description of the embodiments, those skilled in the art will clearly understand that each embodiment can be implemented by software plus a necessary general hardware platform, and of course, can also be implemented by hardware. With this understanding in mind, the above technical solutions may be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM/RAM, magnetic disk, optical disk, etc., and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute the method according to the embodiments or some parts of the embodiments.
Finally, it should be noted that: the above examples are only intended to illustrate the technical solution of the present invention, but not to limit it; although the present invention has been described in detail with reference to the foregoing embodiments, it will be understood by those of ordinary skill in the art that: the technical solutions described in the foregoing embodiments may still be modified, or some technical features may be equivalently replaced; and such modifications or substitutions do not depart from the spirit and scope of the corresponding technical solutions of the embodiments of the present invention.

Claims (10)

1. A method of anthropometric measurement, comprising:
acquiring multi-view image data of a human body to be measured, and acquiring human body posture information based on a human body posture estimation library;
converting the multi-view image data into point cloud data through internal reference of a depth camera;
acquiring a rigid transformation matrix and a preliminary human body three-dimensional model according to the coefficient of the preliminary human body three-dimensional model and the point cloud data;
and acquiring an accurate human body three-dimensional model based on the point cloud data, the human body posture information and the preliminary human body three-dimensional model, and realizing the measurement of the size of the key part based on the accurate human body three-dimensional model.
2. The method of claim 1, wherein the converting the multi-view image data into point cloud data via internal parameters of a depth camera further comprises:
projecting each pixel in the multi-view image to a world coordinate system according to an internal reference of a depth camera;
and reconstructing the coordinates of each pixel in the world coordinate system into point cloud data by using a Poisson reconstruction algorithm.
3. The method of claim 1, wherein obtaining a rigid transformation matrix and a preliminary three-dimensional model of the human body from the coefficients of the preliminary three-dimensional model of the human body and the point cloud data further comprises:
acquiring coarse grain optimization residual errors according to different human body posture information in multi-view image data;
and minimizing the coarse-grained optimization residual error, acquiring a rigid transformation matrix of the point cloud data, and generating a preliminary human body three-dimensional model according to the coefficient of the preliminary human body three-dimensional model and the point cloud data.
4. The method according to claim 3, wherein the obtaining of the coarse-grained optimized residual according to different human posture information in the multi-view image data specifically comprises:
calculating the sum of regular terms of different human posture information in all the multi-view image data E1;
and summing the product of the coefficient and the weight of the preliminary human body three-dimensional model and the E1 to obtain coarse-grained optimization residual error.
5. The method according to claim 1, wherein the obtaining of the accurate three-dimensional human body model based on the point cloud data, the human body posture information and the preliminary three-dimensional human body model and the measurement of the critical dimension based on the accurate three-dimensional human body model specifically comprise:
acquiring matching points of the point cloud data according to the preliminary human body three-dimensional model, and constructing all the matching points into matching point pairs;
optimizing the coefficient of the preliminary human body three-dimensional model and a rigid transformation matrix of the point cloud data according to the matching point pairs;
and obtaining an accurate human body three-dimensional model according to the optimized coefficient of the initial human body three-dimensional model and the rigid transformation matrix of the point cloud data, and realizing the measurement of the size of the key part based on the accurate human body three-dimensional model.
6. The method according to claim 5, wherein the obtaining of the matching points of the point cloud data according to the preliminary three-dimensional human body model and the constructing of all the matching points into matching point pairs specifically comprises:
screening out matching points by calculating the distance between the point cloud data and the preliminary human body three-dimensional model and the direction vector included angle;
and forming a matching point pair by the point cloud data and the matching points of the point cloud data on the preliminary human body three-dimensional model.
7. The method according to claim 5, wherein the optimizing the coefficients of the preliminary three-dimensional model of the human body and the rigid transformation matrix of the point cloud data according to the pairs of matching points comprises:
fixing a rigid transformation matrix of the point cloud data according to the matching point pairs to obtain a coefficient of a preliminary human body three-dimensional model;
and updating the matching point pairs according to the coefficient of the preliminary human body three-dimensional model, fixing the coefficient of the preliminary human body three-dimensional model, and acquiring a rigid transformation matrix of the updated point cloud data.
8. The method according to claim 7, wherein the fixing the rigid transformation matrix of the point cloud data according to the matching point pairs to obtain coefficients of a preliminary three-dimensional model of the human body comprises:
fixing a rigid transformation matrix of the point cloud data according to the matching point pairs;
and obtaining the coefficient of the preliminary human body three-dimensional model by minimizing the error between the matching point and the point cloud data.
9. A human body dimension measuring device, comprising:
an acquisition module: the system comprises a human body posture estimation library, a multi-view image data acquisition module, a human body posture estimation module and a human body posture estimation module, wherein the multi-view image data acquisition module is used for acquiring multi-view image data of a human body to be measured and acquiring human body posture information based on the human body posture estimation;
a conversion module: the system comprises a depth camera, a multi-view image data acquisition unit, a data acquisition unit and a data processing unit, wherein the depth camera is used for acquiring multi-view image data;
a processing module: the system comprises a point cloud data acquisition unit, a rigid transformation matrix acquisition unit, a point cloud data acquisition unit and a data processing unit, wherein the point cloud data acquisition unit is used for acquiring a point cloud data of a point cloud of a human body;
a measurement module: the device is used for obtaining an accurate human body three-dimensional model based on the point cloud data, the human body posture information and the preliminary human body three-dimensional model, and realizing the measurement of the size of the key part based on the accurate human body three-dimensional model.
10. The apparatus of claim 9, wherein the processing module is further configured to:
acquiring coarse grain optimization residual errors according to different human body posture information in multi-view image data;
and minimizing the coarse-grained optimization residual error, acquiring a rigid transformation matrix of the point cloud data, and generating a preliminary human body three-dimensional model according to the coefficient of the preliminary human body three-dimensional model and the point cloud data.
CN202010367206.3A 2020-04-30 2020-04-30 Human body measuring method and device Active CN111612887B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010367206.3A CN111612887B (en) 2020-04-30 2020-04-30 Human body measuring method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010367206.3A CN111612887B (en) 2020-04-30 2020-04-30 Human body measuring method and device

Publications (2)

Publication Number Publication Date
CN111612887A true CN111612887A (en) 2020-09-01
CN111612887B CN111612887B (en) 2021-11-09

Family

ID=72199291

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010367206.3A Active CN111612887B (en) 2020-04-30 2020-04-30 Human body measuring method and device

Country Status (1)

Country Link
CN (1) CN111612887B (en)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112435345A (en) * 2020-12-14 2021-03-02 武汉纺织大学 Human body three-dimensional measurement method and system based on deep learning
CN112884638A (en) * 2021-02-02 2021-06-01 北京东方国信科技股份有限公司 Virtual fitting method and device

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012011068A1 (en) * 2010-07-22 2012-01-26 Cad Modelling Ergonomics S.R.L. Method and apparatus for identification and anthropometric classification of human body
CN106780619A (en) * 2016-11-25 2017-05-31 青岛大学 A kind of human body dimension measurement method based on Kinect depth cameras
CN108053437A (en) * 2017-11-29 2018-05-18 深圳奥比中光科技有限公司 Three-dimensional model acquiring method and device based on figure
CN108629831A (en) * 2018-04-10 2018-10-09 清华大学 3 D human body method for reconstructing and system based on parametric human body template and inertia measurement
CN108665537A (en) * 2018-05-15 2018-10-16 清华大学 The three-dimensional rebuilding method and system of combined optimization human body figure and display model
CN109872354A (en) * 2019-01-28 2019-06-11 深圳市易尚展示股份有限公司 Multi-angle of view point cloud registration method and system based on nonlinear optimization
CN110598590A (en) * 2019-08-28 2019-12-20 清华大学 Close interaction human body posture estimation method and device based on multi-view camera

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012011068A1 (en) * 2010-07-22 2012-01-26 Cad Modelling Ergonomics S.R.L. Method and apparatus for identification and anthropometric classification of human body
CN106780619A (en) * 2016-11-25 2017-05-31 青岛大学 A kind of human body dimension measurement method based on Kinect depth cameras
CN108053437A (en) * 2017-11-29 2018-05-18 深圳奥比中光科技有限公司 Three-dimensional model acquiring method and device based on figure
CN108629831A (en) * 2018-04-10 2018-10-09 清华大学 3 D human body method for reconstructing and system based on parametric human body template and inertia measurement
CN108665537A (en) * 2018-05-15 2018-10-16 清华大学 The three-dimensional rebuilding method and system of combined optimization human body figure and display model
CN109872354A (en) * 2019-01-28 2019-06-11 深圳市易尚展示股份有限公司 Multi-angle of view point cloud registration method and system based on nonlinear optimization
CN110598590A (en) * 2019-08-28 2019-12-20 清华大学 Close interaction human body posture estimation method and device based on multi-view camera

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
L.ROSALES-ROSALES 等: "Anthropometrical study of the human glenoid in a normal Spanish populationEstudio antropométrico de la glena humana en una población española normal", 《REVISTA ESPAÑOLA DE CIRUGÍA ORTOPÉDICA Y TRAUMATOLOGÍA (ENGLISH EDITION)》 *
鲍陈 等: "基于散乱点云的三维人体自动测量", 《纺织学报》 *

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112435345A (en) * 2020-12-14 2021-03-02 武汉纺织大学 Human body three-dimensional measurement method and system based on deep learning
CN112884638A (en) * 2021-02-02 2021-06-01 北京东方国信科技股份有限公司 Virtual fitting method and device

Also Published As

Publication number Publication date
CN111612887B (en) 2021-11-09

Similar Documents

Publication Publication Date Title
CN110189399B (en) Indoor three-dimensional layout reconstruction method and system
CN110866969B (en) Engine blade reconstruction method based on neural network and point cloud registration
CN111679291B (en) Inspection robot target positioning configuration method based on three-dimensional laser radar
CN108053437B (en) Three-dimensional model obtaining method and device based on posture
CN104537707A (en) Image space type stereo vision on-line movement real-time measurement system
CN112734890B (en) Face replacement method and device based on three-dimensional reconstruction
CN106408653B (en) Real-time robust cluster adjustment method for large-scale three-dimensional reconstruction
CN114666564B (en) Method for synthesizing virtual viewpoint image based on implicit neural scene representation
CN111612887B (en) Human body measuring method and device
CN109858437B (en) Automatic luggage volume classification method based on generation query network
WO2023082306A1 (en) Image processing method and apparatus, and electronic device and computer-readable storage medium
CN111260702B (en) Laser three-dimensional point cloud and CT three-dimensional point cloud registration method
CN112330813A (en) Wearing three-dimensional human body model reconstruction method based on monocular depth camera
CN113936090A (en) Three-dimensional human body reconstruction method and device, electronic equipment and storage medium
CN116385660A (en) Indoor single view scene semantic reconstruction method and system
CN111553954A (en) Direct method monocular SLAM-based online luminosity calibration method
CN116797733A (en) Real-time three-dimensional object dynamic reconstruction method
CN116309879A (en) Robot-assisted multi-view three-dimensional scanning measurement method
CN116310194A (en) Three-dimensional model reconstruction method, system, equipment and storage medium for power distribution station room
CN116079727A (en) Humanoid robot motion simulation method and device based on 3D human body posture estimation
JP2007034964A (en) Method and device for restoring movement of camera viewpoint and three-dimensional information and estimating lens distortion parameter, and program for restoring movement of camera viewpoint and three-dimensional information and estimating lens distortion parameter
CN110689513B (en) Color image fusion method and device and terminal equipment
CN109859268B (en) Object shielded part imaging method based on query network generation
CN107316343A (en) A kind of model treatment method and apparatus based on data-driven
KR101533494B1 (en) Method and apparatus for generating 3d video based on template mode

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant
TR01 Transfer of patent right
TR01 Transfer of patent right

Effective date of registration: 20230607

Address after: Room 611-217, R & D center building, China (Hefei) international intelligent voice Industrial Park, 3333 Xiyou Road, high tech Zone, Hefei, Anhui 230001

Patentee after: Hefei lushenshi Technology Co.,Ltd.

Address before: Room 3032, gate 6, block B, 768 Creative Industry Park, 5 Xueyuan Road, Haidian District, Beijing 100083

Patentee before: BEIJING DILUSENSE TECHNOLOGY CO.,LTD.

Patentee before: Hefei lushenshi Technology Co.,Ltd.