TWI542320B - Human weight estimating method by using depth images and skeleton characteristic - Google Patents

Description

Method for estimating body weight with depth map image and skeleton feature points

The invention relates to a method for estimating the weight of a human body, in particular to a method for estimating the weight of a human body by using a depth map image and a skeleton feature point.

With the changes in social economy, medicine and lifestyle, people's living standards have improved, and their relative health has become more and more important. In addition to some major diseases that immediately threaten human health or life, chronic and degenerative diseases can also affect people's quality of life; therefore, in order to slow the pace of chronic and degenerative diseases, daily health care is relatively important. Weight control is one of the daily health care. It can be seen that a personalized weight monitoring system can properly record and analyze the human condition and achieve self-health maintenance.

Common weight monitoring systems include weight machines, body fat machines, etc. In the case of weight machines, users must stand on the weight machine to obtain their body weight values, and then calculate the body mass index (Body Mass Index, by formula). BMI). In Taiwan, the Department of Health of the Executive Yuan published the adult obesity standard in Taiwan according to its related research. The normal standard range is 18.5≦BMI<24, the underweight is BMI<18.5, and the overweight is 24≦BMI<27, light. The degree of obesity was 27≦BMI<30, moderate obesity was 30≦BMI<35, and severe obesity was BMI≧35. Unlike the weight machine, the body fat machine is based on the principle that the human body is conductive and the fat is not conductive. It passes through the human body with a weak current (<800mA), and uses Bioelectrical Impedance Analysis (BIA) with accurate regression analysis. The body fat percentage of the human body.

Statistically speaking, an indicator has a statistically significant degree as long as it can achieve 80% accuracy. Therefore, for public health research that requires a large number of samples, the BMI value is still a very useful tool. In contrast, the accuracy of the body fat rate requires cumbersome steps. Therefore, once the sample is too large, it is not easy to perform; however, if personal health care is concerned, the monitoring of BMI and body fat rate is suitable. However, whether it is a weight machine or a body fat machine, it must be in contact with the human body in order to accurately measure the body weight.

Therefore, in October 2010, Select Research in the UK published a new obesity measurement system using the Body Volume Index (BVI). In the new measurement system of BVI, the 3D scanning instrument is used to scan the human body, which not only can complete the human body scan in a short time (less than 6 seconds), but also notice the difference between human muscle and fat; For the first time, the human body's body shape difference is taken into consideration to accurately determine the distribution of body weight. However, the price of 3D scanning instruments is too high, and consumers generally have no way to buy them themselves and use them at home.

Through the above, it can be known that the non-contact body weight measuring device different from the similar weight machine and body fat machine is not currently sold or provided on the market; at the same time, the scanning body weight which is relatively inexpensive or sold is not available. Measuring device. In view of this, the inventors of this case tried to study it and finally developed a method for estimating the weight of the human body using depth map images and skeleton feature points.

The main object of the present invention is to provide a method for estimating body weight by using depth map images and skeleton feature points, wherein the method combines Microsoft Kinect® Exemplar skeleton detection technology and 3D depth image for weight estimation. Through this estimation system, people can measure the weight, height and BMI in a contactless manner, and simultaneously upload them to the cloud database through the computing host (ie, computer or mobile phone) to record their own health. Control. In addition, when the technology of the present invention and the cloud database are combined with each other, the technology of the present invention can be expanded to be applied to a playground, a dance classroom, or a fitness center; thus, the user can complete a sports amusement facility. After the dance fitness program, the body weight and BMI values are estimated instantaneously, which can improve the efficiency of human health detection.

Therefore, in order to achieve the above object of the present invention, the inventors of the present invention have proposed a method for estimating the weight of a human body by using a depth map image and a skeleton feature point, which comprises the following steps: (1) obtaining one by using the integrated camera. (1) taking a plurality of key skeleton feature points from the plurality of feature points; (3) performing coordinate transformation on the plurality of key skeleton feature points by a standard conversion function, Obtaining a corresponding plurality of screen pixel coordinates; (4) causing the body to be tested to freely rotate, to obtain a front foreground image and at least one side foreground image of the human body to be tested by using the somatosensory camera; (5) a human body height pixel difference function and using the front foreground image and the at least one side foreground image to calculate one of the human body to be tested a human body height pixel; (6) calculating a human body voxel value of the human body to be tested by a human body value function; and (7) by using a regression analysis method by the human body voxel value and the human body height pixel Calculate one of the body to be tested to estimate the body weight, an estimated height and an estimated BMI value.

<present invention>

10‧‧‧ Back-end computing host

11‧‧‧Sports camera

2‧‧‧The body to be tested

S01~S07‧‧‧ method steps

S11~S13‧‧‧ method steps

( , ) ‧ ‧ head features

( , )‧‧‧left shoulder feature points

( , )‧‧‧ right shoulder feature points

( , )‧‧‧ Left ankle feature points

( , ) ‧ ‧ right ankle feature points

S41~S43‧‧‧ method steps

( x _H , y _H ) ‧ ‧ head pixel coordinates

( , )‧‧‧left ankle pixel coordinates

( , )‧‧‧right ankle pixel coordinates

Y(y)‧‧‧ Human height pixels

G‧‧‧Average distance

The first figure is a device architecture diagram for estimating the weight of a human body by using a depth map image and a skeleton feature point; the second figure is a flow chart of a method for estimating the weight of a human body by using a depth map image and a skeleton feature point; The third figure is a detailed step chart of the step (S01); the fourth picture is the depth image of the body to be tested; the fifth picture is the detailed step chart of the step (S04); the sixth picture is the image of the body to be tested The seventh picture is the image of the human body to be tested; the eighth picture is the image of the foreground image of the human body to be tested; and the regression diagram of the weight of the ninth figure and the voxel value of the human body.

In order to more clearly describe a method for estimating the body weight of a depth map image and a skeleton feature point proposed by the present invention, a preferred embodiment of the present invention will be described in detail below with reference to the drawings.

Please refer to the first figure, which is a device architecture diagram for estimating the body weight with depth map images and skeleton feature points. As shown in the first figure, the device architecture adopted by the method of the present invention includes only a back-end computing host 10 and an integrated camera 11; wherein the somatosensory camera 11 used in the device architecture is Microsoft Kinect®, which is used by Taking the color image map and the depth image of the human body 2 to be tested, the back end computing host 10 calculates the human body voxel value of the human body 2 to be tested by performing image processing and obtaining skeleton feature points, and then, Then, the estimated body weight of the human body 2 to be measured, the estimated height and the estimated BMI value are derived from the human body voxel value.

Please refer to the first figure, and please refer to the second figure at the same time, which is a flowchart of the method for estimating the body weight of the depth map image and the skeleton feature point of the present invention; as shown in the second figure, the present invention provides The methodology consists of seven main steps.

The method first performs the step (S01), and obtains a plurality of skeleton feature points of the human body to be tested by using the integrated camera; wherein, the detailed steps of the step (S01) shown in the third figure are steps, steps ( S01) further includes three detailed steps, as follows: First, the steps (S011) and the step (S012) are performed, and the color camera image of the human body 2 to be tested is taken by the somatosensory camera 11 and executed on the color image map. An image blurring process; then, step (S013) is performed to obtain 20 skeleton feature points from the color image map.

After the step (S01) is completed, the method is followed by step (S02), and five key skeleton feature points are taken out from the 20 feature points; then, the step (S03) is performed, and the target conversion function is used as a label conversion function. The five key skeleton feature points are coordinate converted to obtain corresponding five pixel pixel coordinates. In the present invention, as shown in the depth image of the human body to be tested in the fourth figure, the five key skeleton feature points are respectively a single heart feature point ( , ), a left shoulder feature point ( , ), a right shoulder feature point ( , ), a left ankle feature point ( , ), with a right ankle feature point ( , ). And, the coordinate conversion function is: xH=λx* , , ,as well as Where λx and λy are conversion parameters of the x-axis and the y-axis of the screen pixel, respectively, and the values are 320 and 240, respectively. In this way, the five pixel coordinates can be obtained to include a head pixel coordinate ( x _H , y _H ) and a left shoulder pixel coordinate ( , ), a right shoulder pixel coordinate ( , ), a left ankle pixel coordinates ( , ), with a right ankle pixel coordinates ( , ).

Please refer to the fifth and sixth figures, which are respectively the flow chart of the detailed steps of step (S04) and the image of the human body to be tested. In the method of the present invention, after obtaining the five screen pixel coordinates, the step (S04) is performed to rotate the body 2 to be tested to obtain the front side of the body 2 to be tested by the somatosensory camera 11. A foreground image and at least one side foreground image. Wherein, as shown in the fifth figure, the step (S04) further includes three detailed steps. First, the first detailed step (ie, step (S041)) is performed to make the body 2 to be tested free to rotate, so that the plurality of depth images of the human body 2 to be tested are acquired by the somatosensory camera 11. Then performing the step (S042), using an automatic detection algorithm, and using the head pixel coordinates ( x _H , y _H ), the left shoulder pixel coordinates ( , ), the right shoulder pixel coordinates ( , ), the left ankle pixel coordinates ( , ), with the right ankle pixel coordinates ( , And further obtaining a front depth image and at least one side depth image from the plurality of depth images.

According to the above, the automatic detection algorithm described in step (42) is Where Ds is expressed as the shoulder distance of the human body to be tested, and versus They are taken from the right shoulder pixel coordinate and the left shoulder pixel coordinate respectively. Wherein, if the calculated shoulder distance Ds in a depth image is the maximum value, the depth image is the front depth image; otherwise, if the shoulder distance calculated in a depth image is When Ds is the minimum value, the depth image is the side depth image.

After the front depth image and the at least one side depth image are obtained, the method continues with the step (S043), and performs smoothing, for example, background subtraction processing on the front depth image and the at least one side depth image. Filtering processing to obtain corresponding front foreground images and side foreground images. As shown in the seventh figure, the image of the human body to be tested is shown in (a) as the color image of the human body 2 to be tested, and (b) is the front depth of the human body 2 to be tested. The image, and the figure (c) is shown as the front foreground image of the obtained human body 2 to be tested.

After the step (S04) is completed, the method continues to perform the step (S05), and the human foreground height pixel difference function is used together with the front foreground image and the at least one side foreground image to calculate the to-be-tested One of the human body 2 height pixels. As shown in the image of the foreground image of the human body to be tested in the eighth figure, the height pixel system of the human body is defined as Where G is the average distance between the head pixel coordinate and the head of the body to be tested, and yH, versus The pixel coordinates are taken from the head pixel, the left ankle pixel coordinates, and the right ankle pixel coordinates.

After the completion of the human body height pixel Y(y), the method performs the step (S06), and calculates a human body voxel value of the human body 2 to be tested by a human body voxel value function. Wherein, the human body voxel value function is Where βvoxel is expressed as a human body voxel value, Expressed as the x-axis total pixel of the front depth image of the human body to be tested, It is expressed as the x-axis total pixel point of the side depth image of the human body to be tested, and Δy is represented as the number of height pixel points on the x-axis of the depth image. Finally, in step (S07), the human body voxel value and the human body height pixel can be used to calculate the estimated body weight, an estimated height and an estimated BMI by means of regression analysis. value.

In the present invention, the software used in the regression analysis is Mcrosoft Excel® 2010. As shown in the following table (1), the present invention uses 15 subjects as training samples, and uses Excel software to perform regression analysis on the obtained 15 human body voxel values and human body height pixel data.

From the regression diagram of the weight of the ninth figure and the voxel value of the human body, we can find that the data points are concentrated in the vicinity of the regression line, which means that the weight has a linear relationship with the human body voxel value (Voxel). Moreover, the decision coefficient (R2) of the linear regression calculated by Excel software is 0.89, indicating that the regression relationship is strong, that is, the interpretation ability of the regression mode is high; in other words, among the total variance of the weight, 89% have been explained by the Voxel value. Among them, the obtained regression equation is as follows: M=10.00395+3.19343Bvoxelx10-5.

Thus, from the above regression equation, we can calculate the estimated body weight of the subject and organize it in the following table (2).

Among them, the estimated height in Table (2) is obtained by multiplying the human height pixel Y(y) by 0.44, and 0.44 is the parameter obtained by the rule of thumb; then, by Body Mass Index (BMI) The calculation formula can easily calculate the BMI value.

Thus, the method for estimating the body weight of the depth map image and the skeleton feature point of the present invention has been completely and clearly disclosed by the above detailed description, and it can be seen from the above that the present invention has the following Advantages:

1. The present invention proposes a method for estimating body weight using a 3D somatosensory depth camera, which combines Microsoft Kinect® Exemplar skeleton detection technology and 3D depth image for weight estimation, through which the estimation system is People can measure the weight, height and BMI in a contactless way, and upload them to the cloud database through the computing host (ie, computer or mobile phone) for their own health records and control.

2. According to the above first point, when the technology of the present invention and the cloud database are combined with each other, the technology of the present invention can be expanded to be applied to a playground, a dance classroom or a fitness center; thus, the user can complete After a sporty ride or dance fitness program, the body weight and BMI values are estimated immediately, which improves the efficiency of human health testing.

It is to be understood that the foregoing detailed description of the embodiments of the present invention is not intended to Both should be included in the scope of the patent in this case.

S01~S07‧‧‧ method steps

Claims (6)

The method for estimating the weight of a human body by using a depth map image and a skeleton feature point includes the following steps: (11) obtaining a color image of a human body to be tested by using the integrated camera; (12) the color image The image performs an image blurring process; (13) obtains 20 skeleton feature points from the color image map; (2) extracts five key skeleton feature points from the 20 skeleton feature points, respectively, which are one core feature points ( , ), a left shoulder feature point ( , ), a right shoulder feature point ( , ), a left ankle feature point ( , ), with a right ankle feature point ( , And the plurality of screen pixel coordinates include a head pixel coordinate ( x _H , y _H ) and a left shoulder pixel coordinate ( , ), a right shoulder pixel coordinate ( , ), a left ankle pixel coordinates ( , ), with a right ankle pixel coordinates ( , (3) coordinate conversion of the five key skeleton feature points by a standard conversion function to obtain a corresponding plurality of screen pixel coordinates; (4) freely rotating the body to be tested to utilize the somatosensory camera Obtaining a front foreground image and at least one side foreground image of the human body to be tested; (5) calculating the front foreground image and the at least one side foreground image by using a human height pixel difference function a human body height pixel of the human body to be tested; wherein the human body height pixel difference function is Where G is the average distance between the head pixel coordinate and the head of the body to be tested, and y _H , versus The system is taken from the head pixel coordinate, the left ankle pixel coordinate and the right ankle pixel coordinate; (6) calculating a human body voxel value of the human body to be tested by a human body value function; and (7) Calculating a body weight, an estimated height, and an estimated BMI value of the body to be tested by the human body voxel value and the body height pixel by means of regression analysis. The method for estimating the weight of a human body by using a depth map image and a skeleton feature point according to the first aspect of the patent application, wherein the step includes the following detailed steps: (41) allowing the body to be tested to freely rotate to utilize the The somatosensory camera obtains a plurality of depth images of the human body to be tested; (42) utilizing an automatic detection algorithm, and using the head pixel coordinates ( x _H , y _H ), the left shoulder pixel coordinates ( , ), the right shoulder pixel coordinates ( , ), the left ankle pixel coordinates ( , ), with the right ankle pixel coordinates ( , And obtaining a front depth image and at least one side depth image from the plurality of depth images; and (43) performing a smoothing filtering process on the front depth image and the at least one side depth image, And obtaining the corresponding front foreground image and the at least one side foreground image. The method for estimating the weight of a human body by using a depth map image and a skeleton feature point as described in claim 2, wherein the automatic detection algorithm described in the step is Where Ds is expressed as the shoulder distance of the human body to be tested, and versus They are taken from the right shoulder pixel coordinate and the left shoulder pixel coordinate respectively. The method for estimating the weight of a human body by using a depth map image and a skeleton feature point according to the second aspect of the patent application, wherein the smoothing filtering process described in the step is background subtraction processing. The method for estimating the weight of a human body by using a depth map image and a skeleton feature point according to the first aspect of the patent application, wherein the coordinate conversion function described in the step includes: ;as well as Where λ _x and λ _y are conversion parameters of the x-axis and the y-axis of the screen pixel, respectively, and the values are 320 and 240, respectively. The method for estimating the weight of a human body by using a depth map image and a skeleton feature point according to item 2 of the patent application scope, wherein the human body voxel value function described in the step is ,among them, Expressed as the x-axis total pixel of the front depth image of the human body to be tested, It is expressed as the x-axis total pixel point of the side depth image of the human body to be tested, and Δy is represented as the number of height pixel points on the x-axis of the depth image.