CN111493881B - Calorie estimation system and estimation method - Google Patents

Calorie estimation system and estimation method Download PDF

Info

Publication number
CN111493881B
CN111493881B CN202010339397.2A CN202010339397A CN111493881B CN 111493881 B CN111493881 B CN 111493881B CN 202010339397 A CN202010339397 A CN 202010339397A CN 111493881 B CN111493881 B CN 111493881B
Authority
CN
China
Prior art keywords
key
information
calorie
unit
motion amount
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010339397.2A
Other languages
Chinese (zh)
Other versions
CN111493881A (en
Inventor
蒋伟
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Everstep Technology Shanghai Co ltd
Original Assignee
Everstep Technology Shanghai Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Everstep Technology Shanghai Co ltd filed Critical Everstep Technology Shanghai Co ltd
Priority to CN202010339397.2A priority Critical patent/CN111493881B/en
Publication of CN111493881A publication Critical patent/CN111493881A/en
Application granted granted Critical
Publication of CN111493881B publication Critical patent/CN111493881B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1118—Determining activity level
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/02—Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024—Detecting, measuring or recording pulse rate or heart rate
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/103—Detecting, measuring or recording devices for testing the shape, pattern, colour, size or movement of the body or parts thereof, for diagnostic purposes
    • A61B5/11—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb
    • A61B5/1126—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique
    • A61B5/1128—Measuring movement of the entire body or parts thereof, e.g. head or hand tremor, mobility of a limb using a particular sensing technique using image analysis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/48—Other medical applications
    • A61B5/4866—Evaluating metabolism

Abstract

The invention provides a calorie estimation system, which comprises a storage unit, an extraction unit, an identification unit, an exercise amount calculation unit and a calorie calculation unit. In the calorie estimation system, the extraction unit extracts a plurality of frames of image information from the storage unit, and then the calorie consumption is calculated by the identification unit, the exercise amount calculation unit and the calorie calculation unit, so that the influence on the degree of freedom and the fluency of human body actions and the influence on the accuracy of energy consumption estimation due to the influence on the process of calorie estimation by using wearable equipment is avoided. The invention also provides an estimation method implemented by applying the calorie estimation system.

Description

Calorie estimation system and estimation method
Technical Field
The invention relates to the technical field of motion recognition and analysis, in particular to a calorie estimation system and a calorie estimation method.
Background
With the improvement of living standard, the demand of people on daily health state monitoring is increasingly urgent, so that the human activity assessment becomes a new research hotspot in the fields of pattern recognition and machine learning. The human activity evaluation mainly comprises two parts of action recognition and calorie consumption estimation, and the essence of the evaluation is to realize classification recognition and energy consumption estimation of different actions of the human body by utilizing the correlation between the collected motion energy signals and the daily actions of the human body.
Chinese patent publication No. CN105561567B discloses a step-counting and exercise state evaluation device, which collects exercise posture characteristics through an exercise sensor, collects muscle activity levels through a multi-channel surface electromyography sensor, and counts gaits in different exercise modes and calculates calorie consumption by combining the exercise posture and the muscle activity degree. However, the battery energy of the calorie estimation device based on the wearable device is limited, for example, the wearable device is usually heavy in weight to satisfy the sustainability monitoring for a long time, which affects the freedom and fluency of the human body movement, thereby affecting the accuracy of the energy consumption estimation.
Therefore, there is a need to develop a new calorie estimation system and estimation method to solve the above problems in the prior art.
Disclosure of Invention
The invention aims to provide a calorie estimation system and an estimation method applying the calorie estimation system, so as to avoid influencing the degree of freedom and fluency of human body actions due to the process of using a wearable device for calorie estimation, and further influence the accuracy of energy consumption estimation.
To achieve the above object, the calorie estimation system of the present invention includes a storage unit, an extraction unit, an identification unit, an exercise amount calculation unit, and a calorie calculation unit; the storage unit stores user information and a plurality of frames of image information; the extracting unit extracts the plurality of frames of image information from the storage unit and sends the frames of image information to the identifying unit; the identification unit acquires a plurality of pieces of key information in the frame image data and sends the plurality of pieces of key information to the motion amount calculation unit, wherein the key information comprises position information and key part information of skeleton key points; the motion amount calculation unit calculates the frame average motion amount of the plurality of frames of image information according to the plurality of pieces of key information and sends the frame average motion amount to the calorie calculation unit; the calorie calculating unit calculates a calorie consumption amount according to the user information and the mapping information of the frame average motion amount.
The estimation method of the present invention includes the steps of:
extracting a plurality of frames of image information from the storage unit through the extraction unit; acquiring a plurality of pieces of key information in the frame image data through the identification unit, wherein the key information comprises position information and key part information of skeleton key points; calculating the frame average motion amount of the plurality of frames of image information according to the plurality of key information through the motion amount calculating unit; calculating, by the calorie calculating unit, a calorie consumption amount according to the user information and the mapping information of the frame average motion amount.
The calorie estimation system and the estimation method of the present invention have the beneficial effects of: the extraction unit of the calorie estimation system extracts a plurality of frames of image information from the image information, and then the calorie consumption is calculated through the identification unit, the exercise amount calculation unit and the calorie calculation unit, so that the influence on the degree of freedom and the fluency of human body actions and the influence on the accuracy of energy consumption estimation due to the fact that the process of calorie estimation by using wearable equipment is influenced are avoided.
Preferably, the mapping information includes a heart rate value, the storage unit further stores a mapping coefficient, the calorie calculation unit calls the mapping coefficient from the storage unit to convert the frame average motion amount into the heart rate value, and the mapping coefficient is a ratio of a difference between the maximum motion heart rate value and the resting heart rate value to a maximum frame average motion amount. The beneficial effects are that: the accuracy of calculation is improved.
Further preferably, the heart rate value is not less than a resting heart rate value and not more than a maximum moving heart rate value.
Preferably, the storage unit further stores data of a correspondence between the user information and the resting heart rate value, and the calorie calculating unit further calls the user information from the storage unit to calculate the maximum exercise heart rate value and calls the corresponding resting heart rate value from the data of the correspondence. The beneficial effects are that: the accuracy of calculation is improved.
Further preferably, the user information includes gender data and age data, and the calorie calculating unit calculates the maximum exercise heart rate value according to the age data, and calls the corresponding resting heart rate value from the corresponding relationship data according to the gender data and the age.
Further preferably, the maximum motion heart rate value is a difference between a first constant and the age data, and the first constant is 220.
Preferably, the motion amount calculation unit calculates the average motion amount of all skeleton key points and the average motion angle of all key parts according to the plurality of pieces of key information, and then calculates the frame average motion amount of the plurality of pieces of frame image information according to the average motion amount of all skeleton key points and the average motion angle of all key parts. The beneficial effects are that: the accuracy of calculation is improved.
Further preferably, the motion amount calculation unit calculates the average motion amount of all skeleton key points according to the euclidean distance between two frames of the same skeleton key point and the weight of the same skeleton key point.
Further preferably, the motion amount calculation unit calculates the average motion angle of all key portions based on the rotation angle of the same key portion between two frames and the weight of the same key portion.
Further preferably, the system further comprises an assignment unit, the position information of the skeleton key points includes distribution position information of the skeleton key points, the identification unit sends the key position information and the distribution position information of the skeleton key points to the assignment unit, and the assignment unit assigns weights to the skeleton key points and the key positions according to the distribution position information of the skeleton key points and the distribution position information of the key positions.
Further preferably, the assignment unit assigns the weights so that the weights of the skeleton key points distributed on the trunk are greater than the weights of the skeleton key points distributed on the limbs, and the weights of the key parts located on the trunk are greater than the weights of the key parts located on the limbs.
Further preferably, the motion amount calculation unit converts the frame average motion amount of the plurality of frames of image information by using a correction coefficient to obtain the corrected frame average motion amount of the plurality of frames of image information mapped to a uniform coordinate system, where the correction coefficient is a ratio of an actual height of the user to an average trunk height of the plurality of frames of image information.
Further preferably, the training device further comprises a sorting unit, the identifying unit sends the plurality of frames of image information to the sorting unit, the sorting unit sorts the plurality of frames of image information in descending order according to the trunk heights, and provides the frame image information corresponding to the first to mth trunk heights to the movement amount calculating unit, so that the movement amount calculating unit calculates the average trunk height of the plurality of frames of image information.
Preferably, the skeleton key points include a first key point, a second key point and a third key point, the first key point marks an upper limit position of the trunk part, the second key point and the third key point mark a lower limit position of the trunk part, and the trunk height of each frame is a euclidean distance between the first key point and a center point of the second key point and a center point of the third key point.
Further preferably, the first key point is located on the neck, the second key point is located on the right hip, and the third key point is located on the left hip.
Drawings
FIG. 1 is a block diagram of a calorie estimation system according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a position relationship of skeleton key points on a human body model according to an embodiment of the present invention;
FIG. 3 is a block diagram of another calorie estimation system according to an embodiment of the present invention;
fig. 4 is a block diagram illustrating a structure of another calorie estimation system according to an embodiment of the present invention.
Detailed Description
In order to make the objects, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention are clearly and completely described below with reference to the accompanying drawings, and it is obvious that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention. Unless defined otherwise, technical or scientific terms used herein shall have the ordinary meaning as understood by one of ordinary skill in the art to which this invention belongs. As used herein, the word "comprising" and similar words are intended to mean that the element or item listed before the word covers the element or item listed after the word and its equivalents, but does not exclude other elements or items.
In view of the problems in the prior art, embodiments of the present invention provide a calorie estimation system and an estimation method using the calorie estimation system.
Fig. 1 is a block diagram showing a calorie estimation system according to an embodiment of the present invention.
Referring to fig. 1, the calorie estimation system 100 includes a storage unit 101, an extraction unit 102, an identification unit 103, an exercise amount calculation unit 104, and a calorie calculation unit 105.
Wherein the storage unit 101 stores image information. The image information is derived from a motion video. Specifically, the motion video reflects the continuous motion of the user performing the limb motion.
In some embodiments of the present invention, the motion video is any one of a real-time motion video or a recorded video reflecting a user's designated action.
Referring to fig. 1, the estimation method includes the steps of:
extracting a plurality of frames of image information from the storage unit 101 through the extraction unit 102; acquiring a plurality of pieces of key information in the frame image data through the identification unit 103, wherein the key information comprises position information and key part information of skeleton key points; calculating, by the motion amount calculation unit 104, a frame average motion amount of the plurality of frames of image information according to the plurality of key information; the calorie consumption amount is calculated by the calorie calculation unit 105 according to the user information and the mapping information of the frame average motion amount.
Referring to fig. 1, the extracting unit 102 extracts a plurality of frames of image information from the image information and sends the extracted frames of image information to the identifying unit 103. Specifically, the extraction unit 102 divides the motion video into a plurality of frame images in units of frames to respectively extract information of the plurality of frame images, so that the user does not need to wear a wearable device to perform calorie estimation, and accuracy of energy consumption estimation is not affected by affecting the degree of freedom and fluency of human body motion.
In some embodiments of the present invention, the extracting unit 102 is disposed at any one of a mobile terminal, a PC terminal and a server terminal.
In some specific embodiments of the present invention, the mobile terminal is based on an android and iOS platform, and the extracting unit 102 is configured to directly obtain the motion video for a software development kit SDK loaded by a camera of the mobile terminal.
In some specific embodiments of the present invention, the extracting unit 102 is an OpenCV configured for the PC terminal, and the PC terminal runs any one of Linux, Windows, and Mac OS operating systems.
In some specific embodiments of the present invention, the extracting unit 102 is a video processing tool FFmepg configured for the server.
Referring to fig. 1, the recognition unit 103 acquires a plurality of pieces of key information in the frame image data, and sends the key information to the motion amount calculation unit 104, where the key information includes position information and key portion information of a skeleton key point.
In some embodiments of the present invention, the identification unit 103 is an openpos identification system.
Specifically, the identification unit 103 detects position information of a skeleton key point in the frame image information to obtain relationship data between the skeleton key point and the position information of each frame image. The position information of the skeleton key points comprises coordinate information of the skeleton key points in each frame of image, and the coordinate information comprises x-axis coordinate values and y-axis coordinate values.
FIG. 2 is a schematic diagram of the position relationship of some skeletal keypoints on a human model according to some embodiments of the present invention.
Referring to fig. 1 and 2, the recognition unit 103 marks the following key points on a human body model (not labeled in the figures): nose keypoint 0, neck keypoint 1, right shoulder keypoint 2, right elbow keypoint 3, right wrist keypoint 4, left shoulder keypoint 5, left elbow keypoint 6, left wrist keypoint 7, right hip keypoint 8, right knee keypoint 9, right ankle keypoint 10, left hip keypoint 11, left knee keypoint 12, left ankle keypoint 13, right eye keypoint 14, left eye keypoint 15, right ear keypoint 16, and left ear keypoint 17.
In addition, the recognition unit 103 defines the following 13 key parts, which are respectively: 4 sites starting at the nose keypoint 0 and ending at the neck keypoint 1, starting at the neck keypoint 1 and ending at the right shoulder keypoint 2, the right shoulder keypoint 5, the right hip keypoint 8 and the left hip keypoint respectively, a site starting at the right shoulder keypoint 2 and ending at the right elbow keypoint 3, a site starting at the right elbow keypoint 3 and ending at the right wrist keypoint 4, a site starting at the left shoulder keypoint 5 and ending at the left elbow keypoint 6, a site starting at the left hand keypoint 6 and ending at the left wrist keypoint 7, a site starting at the right hip keypoint 8 and ending at the right knee keypoint 9, a site starting at the right elbow keypoint 9 and ending at the right ankle keypoint 10, a site starting at the left hip keypoint 11 and ending at the left hip keypoint 12, and a location beginning at the left knee keypoint 12 and ending at the left ankle keypoint 13.
In some embodiments of the present invention, the key portion information includes coordinate information of the key portion in each frame of image, and the coordinate information of the key portion is expressed by using a vector of a start point coordinate and a vector of an end point coordinate of the key portion.
Taking the position starting from the nose key point 0 and ending at the neck key point 1 as an example, when the coordinate information of the nose key point 0 is (x1, y1), the coordinate information of the neck key point 1 is (x2, y2), and the coordinate information of the position starting from the nose key point 0 and ending at the neck key point 1 is (x1, y1, x2, y 2).
Referring to fig. 1, the motion amount calculation unit 104 calculates a frame average motion amount of the plurality of frames of image information from the plurality of key information, and transmits the frame average motion amount to the calorie calculation unit 105.
Specifically, the motion amount calculation unit 104 calculates the average motion amount of all skeleton key points and the average motion angle of all key portions according to the plurality of pieces of key information, and then calculates the frame average motion amount of the plurality of pieces of frame image information according to the average motion amount of all skeleton key points and the average motion angle of all key portions.
Fig. 3 is a block diagram illustrating another calorie estimation system according to an embodiment of the present invention.
Referring to fig. 1 and 3, the calorie estimation system shown in fig. 3 is different from the calorie estimation system 100 shown in fig. 1 in that the calorie estimation system shown in fig. 3 further includes an assignment unit 301.
Specifically, the position information of the skeleton key points generated by the identification unit 103 includes distribution location information of the skeleton key points, the identification unit 103 sends the key location information and the distribution location information of the skeleton key points to the assignment unit 301, and the assignment unit 301 assigns weights to the skeleton key points and the key locations according to the distribution location information of the skeleton key points and the information of the key locations, respectively.
In some embodiments of the present invention, the motion amount calculating unit 104 calculates the average motion amount of all skeleton key points as follows:
first, the motion amount calculation unit 104 calculates the euclidean distance δ between adjacent different frames for the same skeleton key point by formula 1.
Formula 1 specifically is:
Figure BDA0002468017340000081
wherein x1 and x2 are x-axis coordinate values of the same skeleton key point in a first frame and a second frame respectively, y1 and y2 are y-axis coordinate values of the same skeleton key point in the first frame and the second frame respectively, the first frame and the second frame are two adjacent frames, and the time of the second frame is later than that of the first frame.
Then, on the one hand, the motion amount calculation unit 104 calculates the average motion amount of all skeleton key points from the euclidean distance between two frames of the same skeleton key point and the weight of the same skeleton key point. Specifically, the motion amount calculation unit 104 calculates a plurality of euclidean distances δ of all skeleton key points according to formula 1, obtains the weight assignment result from the assignment unit 301, and calculates the average motion amount α of all skeleton key points according to formula 2.
Formula 2 is specifically:
Figure BDA0002468017340000091
where n1 is the number of skeletal key points involved in the calculation, δiEuclidean distance, w, of each skeleton key point in two adjacent framesiAnd assigning a weight value of each skeleton key point obtained through the weight.
In some embodiments of the present invention, the assigning unit 301 assigns the weights so that the weights of the skeleton key points distributed on the trunk are greater than the weights of the skeleton key points distributed on the limbs.
Specifically, referring to fig. 2, the skeleton key points distributed on the trunk are the neck key point 1, the right shoulder key point 2, the left shoulder key point 5, the right hip key point 8, and the left hip key point 11; the skeleton key points distributed on the four limbs are the right elbow key point 3, the right wrist key point 4, the left elbow key point 6, the left wrist key point 7, the right knee key point 9, the right ankle key point 10, the left knee key point 12 and the left ankle key point 13.
In some embodiments of the present invention, since for a human body, parts with large motion amplitude are mainly distributed on the trunk and the limbs, and the calories consumed by these parts account for most of the total calories consumed by the human body, the skeleton key points involved in the calculation of formula 2 are the skeleton key points distributed on the trunk and the skeleton key points distributed on the limbs. The assigning unit 301 assigns a first weight value to each skeleton key point distributed on the trunk, assigns a second weight value to each skeleton key point distributed on the four limbs, and makes the first weight value larger than the second weight value, and at the same time, the first weight value and the second weight value are added to be 1.
In other embodiments of the present invention, the assigning unit 301 makes the weight values of the remaining skeleton key points equal, that is, the weight values of the right-eye key point 14, the left-eye key point 15, the right-ear key point 16, and the left-ear key point 17 are equal. Specifically, the assigning unit 301 further assigns a third weight value to the remaining skeleton key points, and the sum of the first weight value, the second weight value, and the third weight value is 1. Since the remaining skeleton key points contribute little to calories consumed by human body exercise, the third weight value is controlled to be not more than 0.05.
Then, on the other hand, the motion amount calculation unit 104 calculates the average motion angle of all key parts from the rotation angle of the same key part between two frames and the weight of the same key part.
Specifically, the motion amount calculating unit 104 calculates a vector angle θ between two adjacent frames at the same key location by using a formula 3, and then calculates the average motion angle β of all key locations by using a formula 4 according to the obtained vector angles θ and the weights of different key locations.
Equation 3 specifically is:
Figure BDA0002468017340000101
wherein the content of the first and second substances,
Figure BDA0002468017340000102
the vector of the same key part in the first frame,
Figure BDA0002468017340000103
and the vector of the same key part in the second frame is defined, the first frame and the second frame are two adjacent frames, and the time of the second frame is later than that of the first frame.
Equation 4 specifically is:
Figure BDA0002468017340000104
where n2 is the number of key sites involved in the calculation, θiIs the vector included angle, w ', of each key part in two adjacent frames'iAnd assigning a weight value of each key part obtained by the weight assignment.
In some embodiments of the present invention, the assigning unit 301 assigns the weights such that the weights of the key parts located on the trunk are greater than the weights of the key parts located on the limbs.
Specifically, referring to fig. 2, the key parts located on the trunk are: starting from the neck keypoint 1 and ending in 4 positions of the right shoulder keypoint 2 and the left shoulder keypoint 5, the right hip keypoint 8 and the left hip keypoint, respectively.
The key parts positioned at the four limbs are as follows: a location beginning at the right shoulder keypoint 2 and ending at the right hand elbow keypoint 3, a location beginning at the right hand elbow keypoint 3 and ending at the right wrist keypoint 4, a location beginning at the left shoulder keypoint 5 and ending at the left hand elbow keypoint 6, a location beginning at the left hand elbow keypoint 6 and ending at the left hand wrist keypoint 7, a location beginning at the right hip keypoint 8 and ending at the right knee keypoint 9, a location beginning at the right knee keypoint 9 and ending at the right ankle keypoint 10, a location beginning at the left hip keypoint 11 and ending at the left knee keypoint 12, and a location beginning at the left knee keypoint 12 and ending at the left ankle keypoint 13.
In some embodiments of the present invention, the key parts involved in the calculation of equations 3 and 4 are key parts distributed on the trunk and key parts distributed on the limbs. The assigning unit 301 assigns a fourth weight value to each of the key parts distributed on the trunk, assigns a fifth weight value to each of the key parts distributed on the limbs, and makes the fourth weight value larger than the fifth weight value, and at the same time, the fourth weight value and the fifth weight value are added to be 1.
Finally, the motion amount calculation unit 104 calculates the inter-frame image motion amount E between two adjacent frames according to the average motion amount α of all skeleton key points and the average motion angle β of all key parts by using formula 5, and calculates the frame average motion amount E of the plurality of frames of image information by using formula 6 according to the inter-frame image motion amount E.
Equation 5 specifically is:
ε＝α×β
equation 6 specifically is:
Figure BDA0002468017340000111
where n3 is the number of frames participating in the calculation obtained by subtracting 1 from the number of frames of the moving video.
In some embodiments of the present invention, the user himself/herself shoots the motion video, and since different users shoot different videos using different camera parameters, the control of the shooting distance and shooting angle will affect the calculation of the formulas 1 to 6. To eliminate such an influence, it is necessary to uniformly convert the coordinate system of the motion video.
In some embodiments of the invention, the unified transformation of the coordinate system of the motion video is realized through torso estimation and the actual height of the user. Specifically, the motion amount calculation unit 104 further converts the frame average motion amount of the plurality of frames of image information by using a correction coefficient, so as to obtain the corrected frame average motion amount of the plurality of frames of image information mapped to a uniform coordinate system, where the correction coefficient is a ratio of an actual height of the user to an average trunk height of the plurality of frames of image information.
Specifically, the motion amount calculation unit 104 calculates the corrected frame average motion amount e of the plurality of frames of image information by equation 7.
Equation 7 specifically is:
Figure BDA0002468017340000121
wherein h is the actual height of the user, and T is the average trunk height of the plurality of frames of image information.
In some embodiments of the present invention, referring to fig. 4, the actual height of the user is stored in the storage unit 101, and the exercise amount calculating unit 104 retrieves the actual height of the user from the storage unit 101.
Fig. 4 is a block diagram illustrating a structure of another calorie estimation system according to an embodiment of the present invention.
Referring to fig. 3 and 4, the calorie estimation system shown in fig. 4 also has a ranking unit 401, compared to the calorie estimation system shown in fig. 3. The recognition unit 103 sends the plurality of frames of image information to the sorting unit 401, and the sorting unit 401 sorts the plurality of frames of image information in order from high to low according to the trunk heights, and provides the frame image information corresponding to the first to mth trunk heights to the movement amount calculation unit 104, so that the movement amount calculation unit 104 calculates the average trunk height T.
In some embodiments of the present invention, the several frames of image information are all involved in the calculation of the average torso height T.
In some embodiments of the present invention, the motion amount calculation unit 104 calculates an average torso height T of the plurality of frames of image information by equation 8.
Equation 8 specifically is:
Figure BDA0002468017340000131
wherein, tiIs the height of the trunk of each frame, and m is the number of frames of the motion video participating in the calculation.
In some embodiments of the present invention, m is 1/6 of the total number of frames of the motion video.
In some embodiments of the present invention, the height of the trunk of each frame is obtained by a euclidean distance between a first key point and a center point of a second key point and a center point of a third key point in the skeleton key points. Specifically, the first key point marks the upper limit position of the trunk part, and the second key point and the third key point mark the lower limit position of the trunk part.
In some embodiments of the invention, the first keypoint is located at the neck, the second keypoint is located at the right hip, and the third keypoint is located at the left hip. Specifically, referring to fig. 2, the first key point is the neck key point 1, the second key point is the right hip key point 8, and the third key point is the left hip key point 11.
Referring to fig. 1, the calorie calculation unit 105 calls user information from the storage unit 101 to calculate a calorie consumption amount with the mapping information of the frame average motion amount. Specifically, the mapping information includes a heart rate value, and the heart rate value is not less than a resting heart rate value and not more than a maximum moving heart rate value.
In some embodiments of the present invention, the user information includes gender data and age data, and the storage unit 101 stores the user information and correspondence data between the user information and the resting heart rate value.
Table 1 is a table of correspondence between age data, gender data, and resting heart rate values of users according to some embodiments of the present invention.
TABLE 1
Age value 18～25 26～35 36～45 46～55 56～65 Greater than 65
Male sex 70～73 71～74 71～75 72～76 72～75 70～73
Female with a view to preventing the formation of wrinkles 74～78 73～76 74～78 74～77 74～77 73～76
The process of calculating the calorie consumption amount by the calorie calculating unit 105 specifically includes:
first, the storage unit 101 stores a mapping coefficient that the calorie calculation unit 105 calls to convert the frame average motion amount into the heart rate value, the mapping coefficient being a ratio of a difference between the maximum motion heart rate value and the resting heart rate value to a maximum frame average motion amount.
Specifically, the calorie calculating unit 105 calculates the maximum exercise heart rate value from the user information in the storage unit 101 and calls the corresponding rest heart rate value from the corresponding relationship data, and then establishes a mapping relationship between the modified frame average exercise amount e of the plurality of frames of image information and the heart rate value hr by using formula 9.
Equation 9 specifically is:
hr＝(hrmax-hrrest)*e/emax
wherein, hrmaxIs the maximum heart rate of movement value, hrrestResting heart rate values.
In some embodiments of the invention, when the user is a 30 year old male, since the e to hr mapping is a uniform mapping, the corresponding resting heart rate value obtained from Table 1 is 72.3, and then rounded to 72 as hr applied to equation 9rest。
In some embodiments of the present invention, the calorie calculation unit 105 calculates the maximum exercise heart rate value by calling the age data from the storage unit 101. The maximum motion heart rate value is a difference between a first constant and the age data, the first constant being 220.
Then, the calorie calculating unit 105 calculates a calorie consumption amount c by equation 10k。
Equation 10 specifically is:
Figure BDA0002468017340000151
wherein A is an age value, We is a weight value, t is a duration of exercise, and C1, C2, C3, and C4 are constants.
In some embodiments of the invention, when the user is male, C1 is 0.2017, C2 is 0.1988, C3 is 0.6309, and C4 is 55.0969. When the user is female, C1 is 0.074, C2 is 0.1263, C3 is 0.4472, and C4 is 20.4022. Specifically, A, We, t, C1, C2, C3, and C4 are stored in the memory cell 101.
Although the embodiments of the present invention have been described in detail hereinabove, it is apparent to those skilled in the art that various modifications and variations can be made to these embodiments. However, it is to be understood that such modifications and variations are within the scope and spirit of the present invention as set forth in the following claims. Moreover, the invention as described herein is capable of other embodiments and of being practiced or of being carried out in various ways.

Claims (15)

1. A calorie estimation system is characterized by comprising a storage unit, an extraction unit, an identification unit, an exercise amount calculation unit and a calorie calculation unit;
the storage unit stores user information and a plurality of frames of image information;
the extracting unit extracts the plurality of frames of image information from the storage unit and sends the frames of image information to the identifying unit;
the identification unit acquires a plurality of pieces of key information in the plurality of pieces of frame image information and sends the plurality of pieces of key information to the motion amount calculation unit, wherein the plurality of pieces of key information comprise key part information and position information of a skeleton key point;
the motion amount calculation unit calculates the frame average motion amount of the plurality of frames of image information according to the plurality of pieces of key information and sends the frame average motion amount to the calorie calculation unit;
the calorie calculating unit calculates a calorie consumption amount according to the user information and the mapping information of the frame average motion amount;
the mapping information includes a heart rate value, the storage unit further stores a mapping coefficient, the calorie calculation unit calls the mapping coefficient from the storage unit to convert the frame average motion amount into the heart rate value, and the mapping coefficient is a ratio of a difference between a maximum motion heart rate value and a resting heart rate value to a maximum frame average motion amount.
2. The calorie estimation system of claim 1, wherein the heart rate value is not less than the resting heart rate value and not greater than the maximum exercise heart rate value.
3. The calorie estimation system according to claim 1, wherein the storage unit further stores correspondence data of the user information and the resting heart rate value, and the calorie calculation unit further calls the user information from the storage unit to calculate the maximum exercise heart rate value and calls the corresponding resting heart rate value from the correspondence data, respectively.
4. The calorie estimation system according to claim 3, wherein the user information includes sex data and age data, the calorie calculation unit calculates the maximum exercise heart rate value from the age data, and calls the corresponding resting heart rate value from the correspondence data according to the sex data and the age.
5. The calorie estimation system of claim 4, wherein the maximum heart of motion value is a difference between a first constant and the age data, the first constant being 220.
6. The calorie estimation system according to claim 1, wherein the motion amount calculation unit calculates an average motion amount of all skeletal key points and an average motion angle of all key parts from the plurality of pieces of key information, respectively, and then calculates a frame average motion amount of the plurality of pieces of image information from the average motion amount of all skeletal key points and the average motion angle of all key parts.
7. The calorie estimation system according to claim 6, wherein the motion amount calculation unit calculates the average motion amount of all skeleton key points from the euclidean distance between two frames for the same skeleton key point and the weight of the same skeleton key point.
8. The calorie estimation system according to claim 7, wherein the motion amount calculation unit calculates the all key parts average motion angle according to a rotation angle of the same key part between two frames and a weight of the same key part.
9. The calorie estimation system according to claim 8, further comprising an assignment unit, wherein the position information of the skeleton key points includes distribution location information of the skeleton key points, the identification unit sends the key location information and the distribution location information of the skeleton key points to the assignment unit, and the assignment unit assigns weights to the skeleton key points and the key locations, respectively, according to the distribution location information of the skeleton key points and the key location information.
10. The calorie estimation system according to claim 9, wherein the assignment unit performs the weight assignment such that the weight of the skeleton key points distributed on the trunk is larger than the weight of the skeleton key points distributed on the limbs, and the weight of the key parts located on the trunk is larger than the weight of the key parts located on the limbs.
11. The calorie estimation system according to claim 6, wherein the motion amount calculation unit converts the frame average motion amount of the plurality of frames of image information by a correction coefficient, which is a ratio of an actual height of the user to an average trunk height of the plurality of frames of image information, to obtain a corrected frame average motion amount mapped to the plurality of frames of image information in a uniform coordinate system.
12. The calorie estimation system according to claim 11, further comprising a sorting unit that sends the plurality of frames of image information to the sorting unit, the sorting unit sorting the plurality of frames of image information in order of torso height from high to low, and supplying frame image information corresponding to first to mth torso heights to the exercise amount calculation unit for the exercise amount calculation unit to calculate an average torso height of the plurality of frames of image information.
13. The calorie estimation system of claim 12, wherein the skeleton keypoints comprise a first keypoint marking an upper limit position of a torso part, a second keypoint and a third keypoint marking a lower limit position of the torso part, the torso height per frame being the euclidean distance between the first keypoint and the center points of the second keypoint and the third keypoint.
14. The calorie estimation system of claim 13, wherein the first keypoint is located at the neck, the second keypoint is located at the right hip, and the third keypoint is located at the left hip.
15. An estimation method using the calorie estimation system according to any one of claims 1 to 14, comprising the steps of:
extracting a plurality of frames of image information from the storage unit through the extraction unit;
acquiring a plurality of pieces of key information in the plurality of pieces of frame image information through the identification unit, wherein the key information comprises position information and key part information of skeleton key points;
calculating the frame average motion amount of the plurality of frames of image information according to the plurality of key information through the motion amount calculating unit;
calculating, by the calorie calculating unit, a calorie consumption amount according to the user information and the mapping information of the frame average motion amount.
CN202010339397.2A 2020-04-26 2020-04-26 Calorie estimation system and estimation method Active CN111493881B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010339397.2A CN111493881B (en) 2020-04-26 2020-04-26 Calorie estimation system and estimation method

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010339397.2A CN111493881B (en) 2020-04-26 2020-04-26 Calorie estimation system and estimation method

Publications (2)

Publication Number Publication Date
CN111493881A CN111493881A (en) 2020-08-07
CN111493881B true CN111493881B (en) 2021-03-19

Family

ID=71849451

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010339397.2A Active CN111493881B (en) 2020-04-26 2020-04-26 Calorie estimation system and estimation method

Country Status (1)

Country Link
CN (1) CN111493881B (en)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101520815A (en) * 2008-07-05 2009-09-02 杭州义盛祥通信技术有限公司 Individual sports management system and management method
CN104268285A (en) * 2014-10-21 2015-01-07 厦门大学 Motion capture database searching method based on dynamic bending algorithm
CN110561432A (en) * 2019-08-30 2019-12-13 广东省智能制造研究所 safety cooperation method and device based on man-machine co-fusion
CN110874910A (en) * 2018-08-31 2020-03-10 杭州海康威视数字技术股份有限公司 Road surface alarm method, device, electronic equipment and readable storage medium

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO1998036400A1 (en) * 1997-02-13 1998-08-20 Assist Advanced Technologies Ltd. Sensors for monitoring periodic movement of a human subject
JP3862525B2 (en) * 2001-07-26 2006-12-27 株式会社第一興商 A device that estimates the energy consumption associated with exercise by observing stationary human body movements
CN1582851A (en) * 2004-06-03 2005-02-23 清华大学 Method for determining trace of human movement
US8750971B2 (en) * 2007-05-24 2014-06-10 Bao Tran Wireless stroke monitoring
CN105748039B (en) * 2016-02-03 2018-10-26 天脉聚源(北京)传媒科技有限公司 A kind of method and device calculating movement energy consumption
CN106845073A (en) * 2016-12-16 2017-06-13 中世泓利(北京)健康科技有限公司 The method of automation generation exercise prescription
CN110457995A (en) * 2019-06-26 2019-11-15 山东师范大学 A kind of human motion amount estimation method based on computer vision and system

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101520815A (en) * 2008-07-05 2009-09-02 杭州义盛祥通信技术有限公司 Individual sports management system and management method
CN104268285A (en) * 2014-10-21 2015-01-07 厦门大学 Motion capture database searching method based on dynamic bending algorithm
CN110874910A (en) * 2018-08-31 2020-03-10 杭州海康威视数字技术股份有限公司 Road surface alarm method, device, electronic equipment and readable storage medium
CN110561432A (en) * 2019-08-30 2019-12-13 广东省智能制造研究所 safety cooperation method and device based on man-machine co-fusion

Also Published As

Publication number Publication date
CN111493881A (en) 2020-08-07

Similar Documents

Publication Publication Date Title
CN106650687B (en) Posture correction method based on depth information and skeleton information
CN108198601B (en) Motion scoring method, device, equipment and storage medium
CN107349594B (en) A kind of action evaluation method of virtual Dance System
CN111263953A (en) Operation state evaluation system, operation state evaluation device, operation state evaluation server, operation state evaluation method, and operation state evaluation program
KR20220066366A (en) Predictive individual 3D body model
CN112464918B (en) Body-building action correcting method and device, computer equipment and storage medium
CN109919034A (en) A kind of identification of limb action with correct auxiliary training system and method
CN110298218B (en) Interactive fitness device and interactive fitness system
CN110600125B (en) Posture analysis assistant system based on artificial intelligence and transmission method
CN114067358A (en) Human body posture recognition method and system based on key point detection technology
CN110637324B (en) Three-dimensional data system and three-dimensional data processing method
CN109753868A (en) Appraisal procedure and device, the Intelligent bracelet of athletic performance
KR102356685B1 (en) Home training providing system based on online group and method thereof
CN112381035A (en) Motion similarity evaluation method based on motion trail of skeleton key points
CN112200074A (en) Attitude comparison method and terminal
CN109965851A (en) A kind of human motion fatigue detection system based on multi-physiological-parameter
CN112990011A (en) Body-building action recognition and evaluation method based on machine vision and deep learning
KR20140043174A (en) Simulator for horse riding and method for simulation of horse riding
CN113663312B (en) Micro-inertia-based non-apparatus body-building action quality evaluation method
CN111507301A (en) Video processing method, video processing device, computer equipment and storage medium
CN111493881B (en) Calorie estimation system and estimation method
Yamao et al. Development of human pose recognition system by using raspberry pi and posenet model
CN114191804A (en) Deep-learning-based method and device for judging whether deep-squatting posture is standard or not
CN108096788B (en) Game lower limb rehabilitation training system based on kinect
CN112818800A (en) Physical exercise evaluation method and system based on human skeleton point depth image

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant