CN113920578A - Intelligent home yoga coach information processing system, method, terminal and medium - Google Patents
Intelligent home yoga coach information processing system, method, terminal and medium Download PDFInfo
- Publication number
- CN113920578A CN113920578A CN202111049527.XA CN202111049527A CN113920578A CN 113920578 A CN113920578 A CN 113920578A CN 202111049527 A CN202111049527 A CN 202111049527A CN 113920578 A CN113920578 A CN 113920578A
- Authority
- CN
- China
- Prior art keywords
- information
- human body
- posture
- yoga
- intelligent home
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000000034 method Methods 0.000 title claims abstract description 26
- 230000010365 information processing Effects 0.000 title claims abstract description 21
- 238000012545 processing Methods 0.000 claims abstract description 24
- 235000011034 Rubus glaucus Nutrition 0.000 claims abstract description 21
- 235000009122 Rubus idaeus Nutrition 0.000 claims abstract description 21
- 240000007651 Rubus glaucus Species 0.000 claims abstract description 20
- 238000000605 extraction Methods 0.000 claims abstract description 16
- 238000012549 training Methods 0.000 claims abstract description 16
- 230000036544 posture Effects 0.000 claims description 55
- 238000013528 artificial neural network Methods 0.000 claims description 23
- 238000007637 random forest analysis Methods 0.000 claims description 21
- 210000000988 bone and bone Anatomy 0.000 claims description 13
- 238000001514 detection method Methods 0.000 claims description 11
- 238000003672 processing method Methods 0.000 claims description 9
- 230000006870 function Effects 0.000 claims description 8
- 238000004590 computer program Methods 0.000 claims description 7
- 239000013598 vector Substances 0.000 claims description 6
- 238000003384 imaging method Methods 0.000 claims description 5
- 238000011156 evaluation Methods 0.000 claims description 4
- 238000009826 distribution Methods 0.000 claims description 3
- 208000009144 Pure autonomic failure Diseases 0.000 claims 2
- 239000013312 porous aromatic framework Substances 0.000 claims 2
- 230000009471 action Effects 0.000 abstract description 7
- 230000000694 effects Effects 0.000 abstract description 6
- 238000005516 engineering process Methods 0.000 description 6
- 238000004422 calculation algorithm Methods 0.000 description 4
- 230000007547 defect Effects 0.000 description 4
- 238000010586 diagram Methods 0.000 description 4
- 238000004364 calculation method Methods 0.000 description 3
- 238000013461 design Methods 0.000 description 3
- 238000012986 modification Methods 0.000 description 3
- 230000004048 modification Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 238000004458 analytical method Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 241001520299 Phascolarctos cinereus Species 0.000 description 1
- 244000235659 Rubus idaeus Species 0.000 description 1
- 208000006011 Stroke Diseases 0.000 description 1
- 230000006978 adaptation Effects 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000013473 artificial intelligence Methods 0.000 description 1
- 238000003556 assay Methods 0.000 description 1
- 230000005540 biological transmission Effects 0.000 description 1
- 238000013145 classification model Methods 0.000 description 1
- 239000003086 colorant Substances 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013527 convolutional neural network Methods 0.000 description 1
- 238000013135 deep learning Methods 0.000 description 1
- 230000010354 integration Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000010801 machine learning Methods 0.000 description 1
- 238000013507 mapping Methods 0.000 description 1
- 230000008092 positive effect Effects 0.000 description 1
- 238000012546 transfer Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F18/00—Pattern recognition
- G06F18/20—Analysing
- G06F18/21—Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation
- G06F18/214—Generating training patterns; Bootstrap methods, e.g. bagging or boosting
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/04—Architecture, e.g. interconnection topology
- G06N3/045—Combinations of networks
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Life Sciences & Earth Sciences (AREA)
- Artificial Intelligence (AREA)
- General Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- Evolutionary Computation (AREA)
- Biophysics (AREA)
- Computational Linguistics (AREA)
- Software Systems (AREA)
- Mathematical Physics (AREA)
- Health & Medical Sciences (AREA)
- Biomedical Technology (AREA)
- Computing Systems (AREA)
- Molecular Biology (AREA)
- General Health & Medical Sciences (AREA)
- Evolutionary Biology (AREA)
- Bioinformatics & Cheminformatics (AREA)
- Bioinformatics & Computational Biology (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Image Analysis (AREA)
Abstract
The invention belongs to the technical field of intelligent home equipment, and discloses an intelligent home yoga coach information processing system, a method, a terminal and a medium, wherein a camera is used for collecting RGB (red, green and blue) images and deep data of a user to obtain original image data of a human posture, and the data is transmitted to a raspberry pie; extracting depth skeletonization information by processing the image data through the raspberry pi, and transmitting coordinate information into a cloud server for feature extraction; extracting physical characteristics of the human body by the server, obtaining posture classification results after the physical characteristics are classified by the classifier, comparing the posture classification results with a standard posture library, obtaining calibration information, and returning the calibration information to the web end; and (6) visually displaying. The invention reduces the cost of reporting the yoga class and reduces the time loss for going to the yoga class. Can accurate discernment yoga gesture. The invention is a universal model, and can even play a good teaching effect on Taijiquan, boxing action, dance gesture and the like only by providing enough training data.
Description
Technical Field
The invention belongs to the technical field of intelligent home equipment, and particularly relates to an intelligent home yoga coach information processing system, method, terminal and medium.
Background
At present, with the increasing living standard and health consciousness of people, yoga gradually begins to suffer from stroke among people. However, professional yoga courses have a relatively high cost and need to follow the shift and attend class on time, and these problems also result in a phenomenon that many people wish to learn yoga but the reality is not allowed.
In recent years, due to continuous development and progress of sensor technology and internet of things technology in recent years, the available fields of deep image recognition in life are increasingly wide, such as assisted learning, work commuting to social services and the like, and the technology gradually relates to various aspects of people in life.
Through the above analysis, the problems and defects of the prior art are as follows:
(1) in the prior art, the cost of home yoga trainer equipment is high
(2) In the prior art, a home yoga coach device cannot acquire skeletonized image information of a human body in real time, and the acquired image information is poor in processing effect.
(3) In the prior art, home yoga coaching devices classify different posture images in yoga practice in real time and can not provide basis for posture calibration.
The difficulty in solving the above problems and defects is:
(1) human skeletonization extraction processing is complex.
(2) Real-time skeletonized extraction has high requirements on computational power.
(3) Yoga postures are various, and are difficult to accurately identify and give modification suggestions.
The significance of solving the problems and the defects is as follows:
(1) the real images are abstracted into computer data, and a data basis is provided for the subsequent processing of projects.
(2) And the real-time skeletonization extraction can accurately and efficiently identify actions and give sufficient user feedback.
(3) And corresponding professional suggestions are given according to the user postures, so that the user is helped to correct the postures, and an ideal body building effect is achieved.
Disclosure of Invention
In order to overcome the problems in the related art, the disclosed embodiment of the invention provides an intelligent home yoga trainer information processing system and method.
The technical scheme is as follows: an intelligent home yoga coach information processing method comprises the following steps:
acquiring RGB images and deep data of a user by using a camera to obtain original image data of a human body posture, and transmitting the data into a raspberry group;
step two, extracting depth skeletonization information by processing the image data through the raspberry pi, wherein the depth skeletonization information comprises the following steps: the two-dimensional, deep and three-dimensional coordinates of the plurality of joint points transmit coordinate information into a cloud server for feature extraction;
step three, extracting physical characteristics of the human body, including the human body extension degree and the included angle information of each important joint, by the server; after the physical characteristics are classified by the classifier, a posture classification result is obtained and is compared with a standard posture library to obtain calibration information, and the calibration information is returned to the web end;
fourthly, predicting the posture by utilizing a random forest at the web end according to the characteristics, and checking the posture with the posture standard library to obtain an evaluation result and a suggestion; and visually displayed on the web side.
In an embodiment of the present invention, the extracting depth skeletonization information from the image data processed by the second raspberry pi processing step includes: determining two-dimensional coordinates of 18 skeleton points of a human body in the RGB image and depth values of corresponding points by using an openposition neural network; the two-dimensional coordinates and depth values of each joint point constitute the original image data of the current joint point.
In an embodiment of the invention, the openposition neural network is a bottom-up detection model, and is characterized in that all possible key points in a picture are firstly identified, the output in the model is a heat map with rich colors, the approximate position of a human body in an image is shown, and the connection relation among all the key points is identified; the output of the assay model is paf (Part Affinity Fields partial Affinity); and simultaneously detecting and connecting key points of the human body.
In an embodiment of the present invention, the method for determining two-dimensional coordinates of 18 skeletal points of a human body in an RGB image by using an openposition neural network includes:
1) one branch of the openposition neural network is used for predicting a grading map Confidence Maps (S), and the other branch is used for predicting a partial affinity PAFs skeleton direction vector;
2) carrying out neural network convergence by using a Loss Function of the Loss;
3) calculating a heat map and a bone orientation vector by adopting Gaussian distribution;
4) and performing joint connection and posture recognition prediction of the bone joints.
In one embodiment of the present invention, in step 1), the PAFs are represented by L, and the confidence level is represented by S; if any skeleton point is shielded or not detected, removing the point and not calculating;
a formula reveals a normal peak value of the skeleton point detection marking point; when the human skeleton point is slowly close to the standard point k, the peak value is gradually reached along with the lapse of the human skeleton point;
step 2) the Loss Function of the Loss Function is:
in the second step of the invention, the two-dimensional coordinates and depth value data of 18 skeletal points of each frame of human body are uploaded to a PC (personal computer) through a cloud end for three-dimensional reconstruction;
converting the 2D coordinates into 3D coordinates x, y and z by using camera internal parameters through a pinhole imaging model formula; the pinhole imaging model is:
in an embodiment of the present invention, in the third step, the extracting, by the server, the physical characteristics of the human body includes:
the bone angle, the human body extension degree and the relative proportion of the human body in the three-dimensional space are used. And training a posture classification random forest model by taking the random forest training set as a random forest training set and taking the corresponding human body posture as a label. When the method is used, the individual skeletonized data is used as a random forest model to be input, and a random forest classification result is obtained, namely a human posture classification result is obtained.
Another object of the present invention is to provide an intelligent home yoga trainer information processing system, comprising:
the camera is used for acquiring an original RGB image and a depth image of a human body and providing a basis and a basis for modeling a human body skeleton;
the computing bar is used for deploying an OpenPose classical neural network to realize the skeletonization real-time extraction of the three-dimensional human body;
the bear pie is used for reading WIFI information of the mobile equipment and transmitting the network information to the raspberry pie through the UART serial port;
the raspberry group receives network information of the little bear group to realize the configuration and network access of the mobile equipment; and receiving the human body data information of the neural network, and uploading the skeletonized information to a server.
And the cloud server is used for deploying a random forest model, providing a training set, training a human posture classifier, reading skeletonized information, extracting feature points and classifying postures.
Another object of the present invention is to provide an information data processing terminal, which includes a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, causes the processor to perform an intelligent home yoga trainer information processing method.
Another object of the present invention is to provide a computer-readable storage medium storing a computer program, which, when executed by a processor, causes the processor to perform the method for processing information of an intelligent home yoga trainer.
By combining all the technical schemes, the invention has the advantages and positive effects that:
the invention provides a deep learning-based human skeleton real-time extraction method, which comprises the following steps: and (3) constructing a human body model by utilizing RGB information and depth information acquired by the RealSense depth camera, and realizing the real-time extraction of human skeletonization. Real-time extraction enables the yoga action of the user to be recognized quickly and efficiently, and therefore timely action feedback is given to the user to correct the posture.
OpenVINO-based neural network edge deployment: the invention also aims to provide an OpenPose classical neural network deployed on the Intel computing stick, and the computing power is enhanced to realize data information extraction. On one hand, the real pictures are abstracted into computer data, and a data base is provided for the subsequent processing of projects. On the other hand, the computing bar makes up the defect of insufficient computing power, and the real-time skeletonization extraction becomes practical.
Accurate action identification based on random forests: and (4) collecting a large number of standard data sets, and training a human posture classifier by taking the standard data sets as a training set of a random forest model. The human posture classifier is based on an artificial intelligence classification model, continuously grows according to the input of a standard data set, and is efficient and accurate in recognition effect.
Cloud edge collaborative attitude detection based on Hua is cloud: the human posture classifier is deployed in a Huacheng cloud server to perform posture classification and posture calibration. Data processing is deployed in a cloud server, so that the situation of insufficient computing power of hardware equipment can be relieved, the data processing is processed on a cloud platform, the processed data are skeleton information after abstraction, people images and background images are not involved, and the individual privacy is fully protected.
Aiming at the problem, the invention provides a design idea of a 'home yoga trainer' by combining the development of the sensor and the technology of the Internet of things, and only one camera can play a role of training through machine learning and deep recognition. Not only the learning cost is reduced, but also the limitation of time and space is broken, so that the yoga exercise can be really popularized to the public, and the contribution is made to the health of people.
It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.
Drawings
The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and together with the description, serve to explain the principles of the disclosure.
Fig. 1 is a flowchart of an intelligent home yoga trainer information processing method according to an embodiment of the present invention.
FIG. 2 is a schematic diagram of an intelligent home yoga trainer information processing system according to an embodiment of the invention;
in the figure: 1. a camera; 2. a calculating rod; 3. a little bear pie; 4. a raspberry pie; 5. and (4) a cloud server.
Fig. 3 is a schematic diagram of an intelligent home yoga trainer information processing system according to an embodiment of the invention.
Fig. 4 is a schematic diagram of an intelligent home yoga trainer information processing method according to an embodiment of the present invention.
FIG. 5 is a flowchart of the original data acquisition of the joint according to an embodiment of the present invention.
Fig. 6 is a flowchart of three-dimensional reconstruction provided in an embodiment of the present invention.
Fig. 7 is a graph of experimental results provided by an embodiment of the present invention.
Detailed Description
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below. In the following description, numerous specific details are set forth in order to provide a thorough understanding of the present invention. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiments set forth herein, but rather should be construed as broadly as the present invention is capable of modification in various respects, all without departing from the spirit and scope of the present invention.
It will be understood that when an element is referred to as being "secured to" another element, it can be directly on the other element or intervening elements may also be present. When an element is referred to as being "connected" to another element, it can be directly connected to the other element or intervening elements may also be present. As used herein, the terms "vertical," "horizontal," "left," "right," and the like are for purposes of illustration only and are not intended to represent the only embodiments.
Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. As used herein, the term "and/or" includes any and all combinations of one or more of the associated listed items.
As shown in fig. 1, the method for processing information of an intelligent home yoga trainer according to an embodiment of the present invention includes:
s101, acquiring human body posture data: the image is a reg and depth superposition image, and is the original data acquired by the render D415, and the data is transmitted to the raspberry Pi 4 for processing. Raw data RGB images and deep data are collected from the user.
S102, processing data of the depth skeleton: the coordinates and the visualization of the joint points are obtained, the image data are processed by openposition to extract the deep skeletonized information, the information exists in a rebd form of eighteen joint points, namely, a reg two-dimensional and deep three-dimensional coordinate, and the coordinate information is transmitted to a cloud server 5 for feature extraction.
Openpos obtains a body skeletonized node, and raspberry pi 4 performs alignment processing to extract three-dimensional data.
S103, extracting three-dimensional features, wherein the physical features of the human body extracted by the server represent the extension degree of the human body and the included angle information of each important joint.
The RGBD three-dimensional skeletonized nodes are processed into XYz coordinate system data, and physical characteristics are extracted.
After the physical characteristics are classified by the classifier, a posture classification result is obtained and is compared with a standard posture library to obtain calibration information, and the calibration information is returned to the web end.
And S104, the web side returns information. And predicting the posture of the random forest according to the characteristics, and checking the posture with the posture standard library to obtain an evaluation result and a suggestion.
The technical solution of the present invention is further described below with reference to specific examples.
Example (b):
1. as shown in fig. 2, the intelligent home yoga trainer information processing system of the invention is a cloud edge cooperative integration real-time posture calibration scheme based on openpos and random forest algorithm, and uses the following equipment:
intel RealSense depth camera 1: the method comprises the steps of collecting an original RGB image and a depth image of a human body, and providing a basis and a basis for modeling of a human skeleton.
Intel calculator bar 2: an OpenPose classical neural network is deployed, computing power is enhanced, and 3-dimensional human skeletonization real-time extraction is achieved.
Little bear 3: utilize hong meng system NFC module, read mobile device WIFI information, and then pass through the UART serial ports with network information transfer for raspberry group 4.
Raspberry pie 4: and receiving the network information of the little bear 3, and realizing the configuration and network access of the mobile equipment. And receiving the human body data information of the neural network, and uploading the skeletonized information to a server.
Hua is cloud server 5: and deploying a random forest model, providing a training set, training a human posture classifier, reading skeletonized information, and extracting feature points to classify the postures.
In a preferred embodiment of the present invention, RealSense depth camera 1.
Yoga moves and makes thousands, and the action can inevitably lead to the fact human part to shelter from a few, will realize the accurate discernment to human gesture, just need shoot human camera 1 have higher RGB sensor resolution ratio and frame number and wider depth distance scope. Therefore, the invention selects the intel Realsense D435 depth camera 1 which is most suitable for the product in the market. The resolution and the frame number of the RGB sensor can reach 1920 multiplied by 1080 and 30fps, and the depth distance range can reach 0.105 m-10 m. More than 95% of the products on the market.
In a preferred embodiment of the present invention, the edge device includes:
the home yoga instruction of convenience, rapidness, economy and profession is provided for the user, so that the hardware equipment does not depend on the PC with a troublesome carrying process. From the analysis of the industrialization, the invention selects the small and convenient raspberry pie 4 as the edge device, in order to ensure the high-speed calculation and transmission of data and make up the problems of insufficient performance and insufficient computing power of the raspberry pie 4, the invention selects the raspberry pie 4 with enhanced computing power and is provided with the Intel neural network computing rod 2 to improve the computing power of the system, thereby realizing the real-time three-dimensional skeletonization extraction.
In a preferred embodiment of the present invention, the little bear pie 3 comprises:
from a usage requirement, it is difficult to require the user to learn the computer program startup process. In order to meet the requirements of convenient use and easiness and convenience, the Hongmong NFC module of the koala 3 is adopted, network information edited by the mobile equipment is read by touching one touch, the mobile equipment is configured to enter the network by matching with the raspberry type 4, and then the whole process of automatically pulling up a program by a script is run immediately.
Fig. 3 is a schematic diagram of an intelligent home yoga trainer information processing system according to an embodiment of the present invention.
2. In an embodiment of the present invention, a method for processing information of an intelligent home yoga trainer according to the present invention is shown in fig. 4. The method comprises the following steps:
(1) extracting human body skeletonization information:
in order to obtain the original data of the coordinates of 18 joint points of the human body, as shown in fig. 5, the invention determines the two-dimensional coordinates of 18 bone points of the human body in the RGB image and the depth values of the corresponding points by using an openposition neural network. The two-dimensional coordinates and depth values of each joint point constitute the raw data of the current joint point.
The OpenPose network model is the basis of the present invention. The human body posture recognition method is widely applied to the field of human body posture recognition. Multiple persons in one image can be effectively detected. Parts of the body and associations are learned by Part Affinity Fields. Part Affinity Fields (PAFs) method involves joining portions of body joints to form a body pose. The method utilizes global texture information and adopts a bottom-up method to achieve real-time performance and high precision. The human body posture detection model is different from a top-down network model, and the top-down detection model is mainly used for detecting key points of each person after determining the number of people in a picture. While opencast is a bottom-up detection model that first identifies all possible keypoints in the picture, the output in the model is heatmap (i.e., a colorful heatmap that can show the approximate position of the human body in the image), and the connection relationship between all keypoints, such connection between keypoints is called vector field or human keypoint Affinity field, and the output in the model is paf (Part Affinity Fields partial Affinity). This model performs simultaneous detection and connection of key points of the human body. The single posture estimation algorithm idea of OpenPose is characterized in that the whole algorithm flow is as follows:
1) one branch is used to predict the score Maps Confidence Maps (S) and one branch is used to predict the partial affinities PAFs.
2) And (4) ensuring the reliable convergence of the neural network by using a Loss Function of the Loss.
3) The heat map and bone orientation Vectormap were calculated using gaussian distributions.
4) And performing joint connection and posture recognition prediction of the bone joints.
The specific algorithm idea of OpenPose multi-person posture estimation. On the basis of the raw data, the specific parameters are corrected by the former ten-layer neural network and transmitted to the initial stage as a mapping F. The number of the PAFs is,
the invention is generally called a skeleton direction vector and represents the specific trend of the limb in the skeleton. The total is divided into two parts: blue and orange. Blue represents CNNs, which are typically used for reasoning and prediction by the engine, and after repeated training to increase S, which is generally referred to as confidence in the present invention. Orange stands for so-called convolution kernel, which can greatly reduce the workload of the invention and can improve the efficiency and the layer number of the refinement. In other words, the confidence level S mentioned above in the present invention is improved, that is, the specific position and posture of the body is recognized more accurately.
Equations (2-1), (2-2), and (2-3) reveal the neural network recognition attitude loss function. In the following formula, PAFs are denoted by L and confidence by S. If a bone point is occluded or not detected, then the point is planed out of calculation.
The normal peak of the skeleton point detection mark point is disclosed by the formula (2-4). The human bone point P of the present invention gradually reaches a peak value as it approaches the standard point k.
(2) Three-dimensional reconstruction as shown in fig. 6.
And uploading the two-dimensional coordinates and depth value data of 18 skeletal points of each frame of human body to a PC (personal computer) through a cloud end for three-dimensional reconstruction. The two-dimensional coordinate and depth value data for the 18 bone points are shown in table 1.
TABLE 1
2021-08-16 18:10:05：data-41：(270，189，2.70)，(270，216，2.74)，(248，219，2.83)，(206，223，2.86)，(169，219，2.80)，(293，216，2.78)，(330，219，2.87)，(364，216，2.90)，(255，298，2.79)，(259，351，2.78)，(263，396，2.90)，(285，298，2.73)，(289，347，2.87)，(289，392，3.01)，(263，182，2.69)，(274，182，2.69)，(259，189，2.72)，(282，186，3.18)，
With the two-dimensional coordinates and the depth values of the human skeleton points, the invention can convert the 2D coordinates into the 3D coordinates x, y and z by using parameters such as camera _ factor, camera _ cx, camera _ cy, camera _ fx and camera _ fy in the camera through a pinhole imaging model formula (2-5).
(3) Random forest human posture recognition:
in the technical design of the invention, the most important thing is the construction of the human body posture recognition model. Most of all human body posture recognition technologies on the market at present are based on recognition of RGB images, but if the work of the invention only uses a recognition technology based on two-dimensional images, the work has great limitation, and the angle between a human body and the camera 1, even the distance, can generate great interference on the result of model prediction. Therefore, the invention innovatively uses the bone angle (such as the included angle between the left shoulder and the left arm is called as the left shoulder angle) of the human body in the three-dimensional space as the basis for predicting the random forest model. After tens of thousands of data are collected, the accuracy of the random forest model trained by the calculated bone angle can reach more than 97%. The trained model is deployed to the Hua-is cloud, and by utilizing the processing speed of the cloud, most of calculation power can be saved, and the time from data acquisition to data result return can be reduced to the maximum extent.
Finally, the invention designs the Web page which is used as a port for user interaction and used for displaying information such as recognition gesture, gesture score, gesture suggestion and the like.
The home yoga system has the five advantages of economy, high efficiency, serviceability and integrity. The cost for reporting the yoga class is saved, and the time loss for going to the yoga class is reduced. The yoga gesture can be accurately identified, and detailed prompt information and historical information are compared. The service software has extremely low learning cost, and can be used as soon as possible. The system is quite complete in realization, and the local server and the web program are complete. The invention is a universal model, and can even play a good teaching effect on Taijiquan, boxing action, dance gesture and the like only by providing enough training data. The experimental effect graph is shown in fig. 7.
Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.
It will be understood that the present disclosure is not limited to the precise arrangements described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure should be limited only by the attached claims.
Claims (10)
1. The intelligent home yoga coach information processing method is characterized by comprising the following steps:
acquiring RGB images and deep data of a user by using a camera to obtain original image data of a human body posture, and transmitting the data into a raspberry group;
step two, processing the image data through a raspberry group, and extracting depth skeletonization information, wherein the depth skeletonization information comprises: the two-dimensional, deep and three-dimensional coordinates of the plurality of joint points transmit coordinate information into a cloud server for feature extraction;
step three, extracting physical characteristics of the human body, including the human body extension degree and the joint included angle information, by the server; after the physical characteristics are classified by the classifier, a posture classification result is obtained and is compared with a standard posture library to obtain calibration information, and the calibration information is returned to the web end;
and fourthly, predicting the posture by using a random forest at the web end according to the characteristics, checking the posture with the posture standard library, obtaining an evaluation result and a suggestion, and visually displaying the evaluation result and the suggestion at the web end.
2. The method for processing information of an intelligent home yoga trainer according to claim 1, wherein in the second step, the extracting the depth skeletonization information from the raspberry pi processing image data comprises: determining two-dimensional coordinates of 18 skeleton points of a human body in the RGB image and depth values of corresponding points by using an openposition neural network; the two-dimensional coordinates and depth values of each joint point constitute the original image data of the current joint point.
3. The intelligent home yoga trainer information processing method according to claim 2, wherein the openpos neural network is a bottom-up detection model that first identifies all possible key points in the picture, outputs in the model are color rich heatmaps showing the approximate position of the human body in the image, and identifies the connection relationship between all key points; the output of the detection model is partial affinity; and simultaneously detecting and connecting key points of the human body.
4. The method for processing information of an intelligent home yoga trainer according to claim 2, wherein the method for determining the two-dimensional coordinates of 18 skeletal points of the human body in the RGB image by the openposition neural network comprises:
1) one branch of the openposition neural network is used for predicting the Confidence Maps of the scoring graph, and the other branch is used for predicting the skeleton direction vectors of partial affinity PAFs;
2) carrying out neural network convergence by using a Loss Function of the Loss;
3) calculating a heat map and a bone orientation vector by adopting Gaussian distribution;
4) and performing joint connection and posture recognition prediction of the bone joints.
5. The method for processing information of an intelligent home yoga trainer according to claim 4, wherein in step 1), the PAFs are represented by L, and the confidence level is represented by S; if any skeleton point is shielded or not detected, removing the point and not calculating;
a formula reveals a normal peak value of the skeleton point detection marking point; when the human skeleton point is slowly close to the standard point k, the peak value is gradually reached along with the lapse of the human skeleton point;
step 2) the Loss Function of the Loss Function is:
6. the intelligent home yoga trainer information processing method according to claim 2, wherein in the second step, the two-dimensional coordinates and depth value data of 18 skeletal points of the human body in each frame are uploaded to a PC through a cloud end for three-dimensional reconstruction;
converting the 2D coordinates into 3D coordinates x, y and z by using camera internal parameters through a pinhole imaging model formula; the pinhole imaging model is:
7. the method of claim 2, wherein the step three, the extracting physical characteristics of the human body by the server comprises:
a large number of human body posture skeleton angles, human body stretching degrees and relative proportion of bodies are used as random forest training sets, corresponding human body postures are labels, and posture classification random forest models are trained; when the method is used, the individual skeletonized data is used as a random forest model to be input, and a random forest classification result is obtained, namely a human posture classification result is obtained.
8. An intelligent home yoga trainer information processing system for implementing the intelligent home yoga trainer information processing method according to any one of claims 1 to 7, wherein the intelligent home yoga trainer information processing system comprises:
the camera is used for acquiring an original RGB image and a depth image of a human body and providing a basis and a basis for modeling a human body skeleton;
the computing bar is used for deploying an OpenPose classical neural network and realizing the skeletonization real-time extraction of the three-dimensional human body;
the bear pie is used for reading WIFI information of the mobile equipment and transmitting the network information to the raspberry pie through the UART serial port;
the raspberry group receives network information of the little bear group to realize the configuration and network access of the mobile equipment; receiving human body data information of a neural network, and uploading skeletonized information to a server;
and the cloud server is used for deploying a random forest model, providing a training set, training a human posture classifier, reading skeletonized information, extracting feature points and classifying postures.
9. An information data processing terminal, comprising a memory and a processor, wherein the memory stores a computer program, and the computer program, when executed by the processor, causes the processor to perform the intelligent home yoga trainer information processing method according to any one of claims 1 to 7.
10. A computer-readable storage medium storing a computer program which, when executed by a processor, causes the processor to perform the method of processing information for an intelligent home yoga trainer according to any one of claims 1 to 7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111049527.XA CN113920578A (en) | 2021-09-08 | 2021-09-08 | Intelligent home yoga coach information processing system, method, terminal and medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202111049527.XA CN113920578A (en) | 2021-09-08 | 2021-09-08 | Intelligent home yoga coach information processing system, method, terminal and medium |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN113920578A true CN113920578A (en) | 2022-01-11 |
Family
ID=79234236
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202111049527.XA Pending CN113920578A (en) | 2021-09-08 | 2021-09-08 | Intelligent home yoga coach information processing system, method, terminal and medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN113920578A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115497596A (en) * | 2022-11-18 | 2022-12-20 | 深圳聚邦云天科技有限公司 | Human body motion process posture correction method and system based on Internet of things |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110478862A (en) * | 2019-05-07 | 2019-11-22 | 深圳市云康创新网络科技有限公司 | A kind of exercise guide system and its guidance method |
| CN111652078A (en) * | 2020-05-11 | 2020-09-11 | 浙江大学 | Yoga action guidance system and method based on computer vision |
| CN112861624A (en) * | 2021-01-05 | 2021-05-28 | 哈尔滨工业大学（威海） | Human body posture detection method, system, storage medium, equipment and terminal |
-
2021
- 2021-09-08 CN CN202111049527.XA patent/CN113920578A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110478862A (en) * | 2019-05-07 | 2019-11-22 | 深圳市云康创新网络科技有限公司 | A kind of exercise guide system and its guidance method |
| CN111652078A (en) * | 2020-05-11 | 2020-09-11 | 浙江大学 | Yoga action guidance system and method based on computer vision |
| CN112861624A (en) * | 2021-01-05 | 2021-05-28 | 哈尔滨工业大学（威海） | Human body posture detection method, system, storage medium, equipment and terminal |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN115497596A (en) * | 2022-11-18 | 2022-12-20 | 深圳聚邦云天科技有限公司 | Human body motion process posture correction method and system based on Internet of things |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Zimmermann et al. | Freihand: A dataset for markerless capture of hand pose and shape from single rgb images | |
| US10679046B1 (en) | Machine learning systems and methods of estimating body shape from images | |
| WO2021175050A1 (en) | Three-dimensional reconstruction method and three-dimensional reconstruction device | |
| Jain et al. | Three-dimensional CNN-inspired deep learning architecture for Yoga pose recognition in the real-world environment | |
| WO2017133009A1 (en) | Method for positioning human joint using depth image of convolutional neural network | |
| CN108319957A (en) | A kind of large-scale point cloud semantic segmentation method based on overtrick figure | |
| CN113762133A (en) | Self-weight fitness auxiliary coaching system, method and terminal based on human body posture recognition | |
| CN111950412A (en) | Hierarchical dance action attitude estimation method with sequence multi-scale depth feature fusion | |
| CN105536205A (en) | Upper limb training system based on monocular video human body action sensing | |
| CN108564022A (en) | A kind of more personage's pose detection methods based on positioning classification Recurrent networks | |
| CN111460976A (en) | Data-driven real-time hand motion evaluation method based on RGB video | |
| CN105488541A (en) | Natural feature point identification method based on machine learning in augmented reality system | |
| Hu et al. | Human interaction recognition using spatial-temporal salient feature | |
| Amrutha et al. | Human Body Pose Estimation and Applications | |
| Zhang | Innovation of English teaching model based on machine learning neural network and image super resolution | |
| CN113920578A (en) | Intelligent home yoga coach information processing system, method, terminal and medium | |
| Liu et al. | Action recognition for sports video analysis using part-attention spatio-temporal graph convolutional network | |
| CN108682282A (en) | A kind of exchange method of the augmented reality version periodic table of chemical element based on ARKit frames | |
| CN117711066A (en) | Three-dimensional human body posture estimation method, device, equipment and medium | |
| CN107944340A (en) | A kind of combination is directly measured and the pedestrian of indirect measurement recognition methods again | |
| CN116363757A (en) | Skeleton and sensor bimodal human behavior recognition method based on self-attention intention convolution | |
| WO2023214093A1 (en) | Accurate 3d body shape regression using metric and/or semantic attributes | |
| CN108108648A (en) | A kind of new gesture recognition system device and method | |
| CN114120371A (en) | System and method for diagram recognition and action correction | |
| Li et al. | Application of human body gesture recognition algorithm based on deep learning in non-contact human body measurement |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |