CN115105059A

CN115105059A - Method and device for determining whole body posture of human body and intelligent shoes

Info

Publication number: CN115105059A
Application number: CN202210655554.XA
Authority: CN
Inventors: 陈梓嘉; 张一驰
Original assignee: Shenzhen Qianhai Xiangfang Future Technology Co ltd
Current assignee: Shenzhen Qianhai Xiangfang Future Technology Co ltd
Priority date: 2022-06-10
Filing date: 2022-06-10
Publication date: 2022-09-27
Also published as: WO2023236353A1

Abstract

The application provides a method for determining the posture of the whole body of a human body, a device for determining the posture of the whole body of the human body and an intelligent shoe, wherein the method comprises the following steps: firstly, foot pressure data and foot position data of a target object at each moment are obtained, wherein the foot position data comprise position data corresponding to the contact between the foot of the target object and the ground and position data corresponding to the non-contact between the foot of the target object and the ground; then, the whole body posture and the whole body skeleton point of the target object are determined based on at least one of the foot pressure data and the foot position data at each time. The whole body posture and the whole body skeleton points corresponding to the contact of the feet of the target object with the ground can be accurately obtained, the whole body posture and the whole body skeleton points corresponding to the non-contact of the feet with the ground can be accurately obtained, sensors are prevented from being arranged at other positions of the target object, the problem that the whole body posture of the human body cannot be simply and accurately predicted in the prior art is solved, and the accuracy of the determined whole body posture of the human body is high.

Description

Method and device for determining whole body posture of human body and intelligent shoes

Technical Field

The application relates to the field of human body postures, in particular to a method for determining a whole body posture of a human body, a determination device, a computer readable storage medium, a processor and an intelligent shoe.

Background

Human body skeleton posture restoration has wide application scenes in the fields of movie animation, live broadcast, health and the like, wherein the action and the action of a movie character are dynamically generated by human skeleton in the movie animation, a virtual portrait or a virtual idol is generated through skeleton data for live broadcast, the human body posture is captured for motion sensing game or interaction of an online conference, the health condition and the activity of old people are monitored, auxiliary rehabilitation, athlete training and the like, the human body posture data value is high, a plurality of technical routes are developed in recent years, and technically, the response speed, the accuracy, the robustness (anti-shielding capability), the easiness in equipment deployment (portability) and the like are mainly concerned.

At present, the posture of a human body is restored mainly by a computer vision method, an inertial sensor whole body tracking technology or a lighthouse laser positioning method, but a camera in the computer vision method needs to be fixed at a specific height and position, so that the degree of motion recognition not facing the camera is not high, and the camera is easily shielded, in addition, delay and corresponding speed are slow due to overlarge calculated amount in a picture calculation process, privacy of a user can be invaded, the experience of the user is low, posture information obtained by the inertial sensor whole body tracking technology is limited and accuracy is low, specifically, errors of the inertial sensor become larger along with the lapse of time in a use process, therefore, calibration needs to be carried out after a period of time, in addition, if more accurate human body postures need to be obtained, inertial sensor equipment needs to be bound on each trunk of the human body, the comfort of the user is low, two base stations need to be handled by the lighthouse laser positioning technology, and the user can only stand in a specific area.

Therefore, a method for simply and accurately predicting the posture of the whole body of a human body is needed.

The above information disclosed in this background section is only for enhancement of understanding of the background of the technology described herein and, therefore, certain information may be included in the background that does not form the prior art that is already known in this country to a person of ordinary skill in the art.

Disclosure of Invention

The present application mainly aims to provide a method for determining a posture of a whole body of a human body, a device for determining the posture of the whole body of the human body, a computer-readable storage medium, a processor and an intelligent shoe, so as to solve the problem that the posture of the whole body of the human body cannot be simply and accurately predicted in the prior art.

According to an aspect of an embodiment of the present invention, there is provided a method for determining a posture of a whole body of a human body, the method including: acquiring foot pressure data and foot position data of a target object at each moment, wherein the foot position data comprises position data corresponding to the contact between the foot of the target object and the ground and position data corresponding to the non-contact between the foot and the ground; and determining the whole body posture and the whole body bone points of the target object according to at least one of the foot pressure data and the foot position data at each moment.

Optionally, the obtaining foot pressure data and foot position data of the target object at each time includes: acquiring the foot pressure data and the corresponding foot posture data at each moment in real time; correcting the corresponding foot posture data at least according to the foot pressure data to obtain target foot posture data; determining the foot position data based at least on the target foot pose data.

Optionally, determining a whole body posture and a whole body skeleton point of the target object according to at least one of the foot pressure data and the foot position data at each time, including: determining the whole body posture and the whole body bone points from the foot pressure data at each time point under the condition that each foot of the target object is in contact with the ground; and under the condition that the feet are not in contact with the ground, determining the whole body posture and the whole body skeleton point according to the foot position data and the corresponding foot pressure data at each time.

Optionally, determining the whole body posture and the whole body skeletal points according to the foot position data and the corresponding foot pressure data at each of the time instants comprises: determining an initial whole body posture and initial whole body skeletal points of the target object according to the foot pressure data and the foot position data; performing predetermined processing on the initial whole-body posture and the initial whole-body bone points to obtain the whole-body posture and the whole-body bone points, wherein the predetermined processing at least comprises one of the following steps: kalman filtering processing, multi-layer perceptron processing and extended Kalman filter processing.

Optionally, modifying the corresponding foot posture data according to at least the foot pressure data to obtain target foot posture data, including: fitting first foot acceleration data by using a logistic regression algorithm or a linear regression algorithm to determine acceleration error data, wherein the first foot acceleration data is corresponding foot acceleration data when each foot is in contact with the ground, and the acceleration error data is error data representing target equipment for acquiring foot posture data; and when the foot is in contact with the ground and is not in contact with the ground, subtracting second foot acceleration data from the acceleration error data to obtain second target acceleration data, wherein the second foot acceleration data is the corresponding foot acceleration data when the foot is not in contact with the ground.

Optionally, determining the foot position data at least according to the target foot pose data comprises: acquiring the whole body posture and the whole body skeleton points corresponding to a first preset moment, wherein the first preset moment is the moment when each foot part is in contact with the ground; determining reference position data of the foot corresponding to the first preset time according to the whole body posture and the whole body skeleton points corresponding to the first preset time; acquiring first target foot posture data and second target foot posture data, wherein the first target foot posture data is the target foot posture data corresponding to the first preset time, the second target foot posture data is the target foot posture data corresponding to a second preset time, and the second preset time is the time when the foot is converted from the state of being in contact with the ground corresponding to the first preset time to the state of not being in contact with the ground; calculating a difference value between the first target foot attitude data and the reference position data, and correcting the second target foot attitude data according to the difference value; and determining the foot position data corresponding to the second preset time according to the corrected second target foot posture data.

Optionally, determining the full body posture and the full body skeletal points from the foot pressure data at each of the time instants with each of the feet of the target subject in contact with the ground comprises: obtaining a second deep learning model, wherein the second deep learning model is trained through machine learning by using multiple groups of first data, and each group of data in the multiple groups of first data comprises first historical foot pressure data when the foot of the target object is in contact with the ground, and corresponding first historical whole body bone points and first historical whole body postures; inputting the foot pressure data of the target object when all the feet of the target object are in contact with the ground into the second deep learning model to obtain the whole body posture and the whole body bone points of the target object when all the feet of the target object are in contact with the ground, and determining the whole body posture and the whole body bone points according to the foot position data and the corresponding foot pressure data at each time point under the condition that the feet are not in contact with the ground, wherein the method comprises the following steps: obtaining a third deep learning model, wherein the third deep learning model is trained through machine learning by using multiple sets of second data, and each set of data in the multiple sets of second data comprises historical foot position data, second historical foot pressure data, corresponding second historical whole body bone points and second historical whole body postures when the foot of the target object is not in contact with the ground; inputting the foot position data and the foot pressure data of the target object when the foot of the target object is not in contact with the ground into the third deep learning model, to obtain the whole body posture and the whole body bone point of the target object when the foot of the target object is not in contact with the ground.

Optionally, after obtaining the foot pressure data and the foot posture data of the target subject, the method further comprises: and under the condition that the foot pressure data meet a preset condition, sending out a thermal sensing signal and a vibration signal, wherein the thermal sensing signal is a signal for controlling the electric heating layer to send out preset heat, and the vibration signal is a signal for controlling the vibration module to generate vibration with a preset amplitude.

According to another aspect of the embodiments of the present invention, there is also provided an apparatus for determining a posture of a whole body of a human body, the apparatus including an obtaining unit and a determining unit, wherein the obtaining unit is configured to obtain foot pressure data and foot position data of a target object at each time, and the foot position data includes position data corresponding to a foot of the target object contacting with a ground and position data corresponding to the foot not contacting with the ground; the determination unit is configured to determine a whole body posture and a whole body skeleton point of the target object according to at least one of the foot pressure data and the foot position data at each time.

According to yet another aspect of embodiments of the present invention, there is also provided a computer-readable storage medium including a stored program, wherein the program is for executing any one of the methods.

According to yet another aspect of the embodiments of the present invention, there is also provided a processor for executing a program, where the program executes to perform any one of the methods.

According to another aspect of the embodiment of the invention, there is also provided a pair of intelligent shoes, each intelligent shoe comprises an inner layer and an outer layer, wherein the inner layer comprises a pressure sensor layer, and the pressure sensor layer is used for acquiring foot pressure data; the outer layer comprises processing circuitry for performing any of the methods and an IMU for obtaining foot position data comprising position data corresponding to a target subject's foot in contact with a ground surface and position data not corresponding to the foot in contact with the ground surface.

Optionally, the inner layer further comprises an electrothermal layer and a vibration module, wherein the electrothermal layer is located on one side of the pressure sensor layer and is used for emitting predetermined heat; the vibration module is located on one side, far away from the pressure sensor layer, of the electric heating layer and used for generating vibration with a preset amplitude.

In the method for determining the posture of the whole body of the human body, firstly, foot pressure data and foot position data of a target object at each moment are obtained, wherein the foot position data comprise position data corresponding to the contact between the foot of the target object and the ground and position data corresponding to the non-contact between the foot of the target object and the ground; then, a whole body posture and a whole body skeleton point of the target object are determined based on at least one of the foot pressure data and the foot position data at each time. Compare the problem of the whole body gesture of the prediction human that can not be simple and accurate among the prior art, the application the method for determining whole body gesture of human is through acquireing foot pressure data and foot position data, wherein, foot position data include the foot of target object with the position data that ground contact corresponds and the foot not with the position data that ground contact corresponds, again according to foot pressure data and foot position data confirm target object's whole body gesture and whole body bone point, guaranteed can obtain more accurately the corresponding whole body gesture when target object's foot and ground contact and whole body bone point, and obtain more accurately the foot not with ground contact corresponds whole body gesture and whole body bone point, guaranteed only according to foot pressure data and foot position data just can confirm the higher degree of accuracy's whole body gesture and whole body bone point The whole body gesture and the whole body skeleton point avoid needing to set up a plurality of sensors in other positions of target object among the prior art, guaranteed to confirm the process of whole body gesture of human is comparatively simple and the cost is lower, has guaranteed target user's experience sense is better, has solved the problem that can not simple and accurate prediction whole body gesture of human among the prior art, has guaranteed that the certainty the accuracy of whole body gesture of human is higher.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this application, illustrate embodiments of the application and, together with the description, serve to explain the application and are not intended to limit the application. In the drawings:

fig. 1 shows a flow chart of a method for determining a whole-body posture of a human body according to an embodiment of the present application;

fig. 2 shows a schematic diagram of a determination apparatus of a human body posture over the body according to an embodiment of the present application.

Detailed Description

It should be noted that the embodiments and features of the embodiments in the present application may be combined with each other without conflict. The present application will be described in detail below with reference to the embodiments with reference to the attached drawings.

In order to make the technical solutions better understood by those skilled in the art, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only partial embodiments of the present application, but not all embodiments. All other embodiments obtained by a person of ordinary skill in the art based on the embodiments in the present application without making any creative effort shall fall within the protection scope of the present application.

It should be noted that the terms "first," "second," and the like in the description and claims of this application and in the drawings described above are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It should be understood that the data so used may be interchanged under appropriate circumstances such that embodiments of the application described herein may be used. Furthermore, the terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover a non-exclusive inclusion, such that a process, method, system, article, or apparatus that comprises a list of steps or elements is not necessarily limited to those steps or elements expressly listed, but may include other steps or elements not expressly listed or inherent to such process, method, article, or apparatus.

It will be understood that when an element such as a layer, film, region, or substrate is referred to as being "on" another element, it can be directly on the other element or intervening elements may also be present. Also, in the specification and claims, when an element is described as being "connected" to another element, the element may be "directly connected" to the other element or "connected" to the other element through a third element.

As described in the background art, in order to solve the above problems, a method for determining a posture of a whole body of a human body, a device for determining the posture, a computer-readable storage medium, a processor, and a smart shoe are provided in an exemplary embodiment of the present application.

According to an embodiment of the present application, a method for determining a posture of a whole body of a human body is provided.

Fig. 1 is a flowchart of a method for determining a posture of a whole body of a human body according to an embodiment of the present application. As shown in fig. 1, the method comprises the steps of:

step S101, obtaining foot pressure data and foot position data of a target object at each moment, wherein the foot position data comprises position data corresponding to the contact between the foot of the target object and the ground and position data corresponding to the non-contact between the foot and the ground;

step S102 of specifying a whole body posture and a whole body skeleton point of the target object based on at least one of the foot pressure data and the foot position data at each time.

In the method for determining the posture of the whole body of the human body, first, foot pressure data and foot position data of a target object at each time are acquired, wherein the foot position data include position data corresponding to the contact of the foot of the target object with the ground and position data corresponding to the non-contact of the foot with the ground; then, the whole body posture and the whole body skeleton point of the target object are determined based on at least one of the foot pressure data and the foot position data at each time. Compared with the problem that the whole body posture of the human body cannot be simply and accurately predicted in the prior art, the method for determining the whole body posture of the human body of the application acquires the foot pressure data and the foot position data, wherein the foot position data comprises position data corresponding to the contact between the foot of the target object and the ground and position data corresponding to the non-contact between the foot of the target object and the ground, and determines the whole body posture and the whole body skeleton point of the target object according to the foot pressure data and the foot position data, so that the whole body posture and the whole body skeleton point corresponding to the contact between the foot of the target object and the ground can be accurately obtained, the whole body posture and the whole body skeleton point corresponding to the non-contact between the foot of the target object and the ground can be accurately obtained, and the high-accuracy determination can be realized only according to the foot pressure data and the foot position data Above-mentioned whole body gesture and above-mentioned whole body skeleton point have avoided needing to set up a plurality of sensors in other positions of above-mentioned target object among the prior art, have guaranteed that the process of confirming above-mentioned whole body gesture of human is comparatively simple and the cost is lower, have guaranteed that above-mentioned target user's experience feels better, have solved the problem that can not simple and accurate prediction whole body gesture of human among the prior art, have guaranteed that the accuracy of the whole body gesture of above-mentioned human of confirming is higher.

According to a specific embodiment of the present application, acquiring foot pressure data and foot position data of a target object at each time includes: acquiring the foot pressure data and the corresponding foot posture data at each moment in real time; correcting the corresponding foot posture data at least according to the foot pressure data to obtain target foot posture data; and determining the foot position data at least according to the target foot posture data. The foot pressure data and the corresponding foot posture data at each moment are acquired, and the corresponding foot posture data are corrected according to the foot pressure data, so that the accuracy of the obtained target foot posture data is high, the foot position data is determined according to the target foot posture data, the accuracy of the foot position data determined according to the target foot posture data is high, the determined whole body posture and the determined whole body skeleton point of the target object are high, and the determined whole body posture of the human body is high.

In a specific embodiment, since the target user may leave the ground when jumping, kicking, or walking, the accuracy of the whole body posture and the whole body skeleton point determined when the foot of the target user leaves the ground is ensured to be high by obtaining the foot posture data, determining the foot position data based on the target foot posture data, and obtaining the whole body posture and the whole body skeleton point when the foot of the target user leaves the ground based on the whole body posture and the whole body skeleton point before the foot of the target user leaves the ground.

Specifically, the foot posture data includes foot acceleration data and foot angular velocity data, and the foot position data is obtained by determining a moving distance of the foot of the target user from the ground based on the foot acceleration data and the foot angular velocity data, and calculating a specific position and a staying time of the foot in the air.

In a specific embodiment, in addition to determining the foot position data from the foot pose data, the foot position data may be determined by first sensing the real-time relative position of the feet to each other by an Electromagnetic sensor (EMF) in combination with a 9-axis IMU (Inertial Measurement Unit) and an electromagnet, specifically, the Electromagnetic sensor may be within an error of 1 cm from a distance of 30 cm; secondly, fitting by a plurality of ultrasonic ranging sensors and generators to sense the relative positions of the feet; thirdly, fitting through a plurality of millimeter wave radars to perceive the relative positions of the feet; fourthly, fitting by a plurality of infrared distance measuring sensors to sense the relative positions of the feet; fifthly, sensing the relative positions of the feet with each other by matrix fitting of a plurality of capacitive ranging sensors; sixth, the feet are fitted by a plurality of Radio Frequency Identification (RFID) sensors to sense the relative position of the feet to each other. The practical use of the electromagnet, the 9-axis IMU (MPU9250) used in the present application is provided with 3-axis magnetometers in a built-in manner, wherein the built-in magnetometer is AK8963, an electromagnet (placed in a shoe with an immovable foot) with a suction force of 50kg serves as a reference magnet, the acceleration and angular velocity data of the IMU are used for acquiring the real-time angle of the currently tracked object, the existing data is the three-axis electromagnetic field strength of the tracked object compared with the electromagnet and the angle information of the currently tracked object, the actual position of the current object compared with the electromagnet can be obtained through an algorithm of Particle filtering (Particle Filter), a CH201 sensor of a TDK is used during ultrasonic testing, the relative positioning of the object can be realized through three CH201, and of course, different sensor step relative information can be used according to the actual situation to achieve the actually required effect.

According to another specific embodiment of the present application, the determining a whole-body posture and a whole-body skeleton point of the target object based on at least one of the foot pressure data and the foot position data at the respective times includes: determining the whole body posture and the whole body skeleton point based on the foot pressure data at each of the time points when each of the feet of the target object is in contact with the ground; when the foot does not contact the ground surface, the whole body posture and the whole body skeleton point are determined based on the foot position data and the corresponding foot pressure data at each of the times. Determining the whole body posture and the whole body skeleton point based on the foot pressure data at each of the times by distinguishing whether each of the feet of the target object is in contact with the ground surface, and when each of the feet of the target object is in contact with the ground surface, determining the whole body posture and the whole body skeleton point based on the foot pressure data at each of the times, ensuring high accuracy of the whole body posture and the whole body skeleton point determined based on the foot pressure data, and when the foot is not in contact with the ground surface, since the foot away from the ground surface does not generate the foot pressure data, determining the whole body posture and the whole body skeleton point based on the foot position data and the corresponding foot pressure data at each of the times, further ensuring high accuracy of the whole body posture and the whole body skeleton point determined when the foot is not in contact with the ground surface, further ensuring the higher accuracy of the determined posture of the whole body of the human body.

In order to further ensure that the accuracy of the whole body posture of the human body is high, according to another specific embodiment of the present application, the determining the whole body posture and the whole body skeleton point based on the foot position data and the corresponding foot pressure data at each of the time points includes: determining an initial whole body posture and an initial whole body skeleton point of the target object according to the foot pressure data and the foot position data; performing predetermined processing on the initial whole-body posture and the initial whole-body skeleton point to obtain the whole-body posture and the whole-body skeleton point, wherein the predetermined processing includes at least one of: kalman filtering processing, multi-layer perceptron processing and extended Kalman filter processing. By performing the predetermined processing on the initial whole body skeleton points determined by the foot pressure data and the foot position data, the obtained whole body posture and the accuracy of the whole body skeleton points are ensured to be higher, and the accuracy of the determined whole body posture of the human body is further ensured to be higher.

Specifically, the kalman filter is an algorithm for performing optimal estimation on the system state by inputting and outputting observation data through a system by using a linear system state equation, because the observation data includes the influence of noise and interference in the system, the optimal estimation can also be regarded as a filtering process, and the data filtering is a data processing technology for removing noise and restoring real data, and because the kalman filter is convenient for the realization of computer programming and can update and process the data acquired on site in real time.

In a specific embodiment, the multi-layer Perceptron (MLP) is a feedforward artificial neural network model, which maps a plurality of input data sets onto a single output data set, and the extended kalman filter method considers the change of the human body posture as a state variable, considers a nonlinear model of the change of the human body posture, and can effectively suppress noise and improve accuracy by using the extended kalman filter method.

According to an embodiment of the present application, the correcting the corresponding foot posture data according to at least the foot pressure data to obtain target foot posture data includes: fitting first foot acceleration data by using a logistic regression algorithm or a linear regression algorithm to determine acceleration error data, wherein the first foot acceleration data is corresponding foot acceleration data when each foot is in contact with the ground, and the acceleration error data is error data representing target equipment for acquiring the foot posture data; when the foot is not in contact with the ground, the acceleration error data is subtracted from second foot acceleration data, which is the foot acceleration data corresponding to the case where the foot is not in contact with the ground, to obtain second target acceleration data. The method comprises the steps of fitting foot acceleration data corresponding to the feet in contact with the ground by using a logistic regression algorithm or a linear regression algorithm to determine acceleration error data, and subtracting the acceleration error data from the foot acceleration data corresponding to the feet not in contact with the ground when the feet are in contact with the ground and are not in contact with the ground, so that the accuracy of second target acceleration data obtained by removing the acceleration error data is high, the accuracy of the target foot posture data is high, the accuracy of the whole body posture and the whole body bone points determined according to the foot posture data is high, and the accuracy of the determined whole body posture of the human body is further high.

Specifically, the target device that collects the foot posture data cannot maintain accurate position sensing for a long time due to its physical characteristics, and is prone to drift, and the first foot acceleration data corresponding to each foot contacting the ground in the practical application process is small, so that the acceleration generated at this time is mainly determined by the accumulated error of the target device that collects the foot posture data, and the acceleration error data is determined by determining the foot acceleration data corresponding to each foot contacting the ground, and the acceleration error data is subtracted from the second foot acceleration data generated when the subsequent foot does not contact the ground, so as to perform correction each time the target user contacts the ground, and such high frequency correction can make the target device maintain accurate tracking of the foot in real time, the obtained second target acceleration data does not include error data caused by the target equipment, the accuracy of the second target acceleration data is high, the accuracy of the target foot posture data is high, the accuracy of the whole body posture and the whole body skeleton point determined according to the foot posture data is high, and the accuracy of the determined whole body posture of the human body is further high.

In a specific embodiment, an AHRS (Attitude and Heading Reference System) algorithm may be further used to obtain euler angle data relative to the ground at a current point, further correct the foot acceleration data of the IMU to obtain acceleration data relative to the ground, further integrate the three-axis acceleration data to obtain real-time three-axis velocity data relative to the ground, and further track the position of the tracked object according to the three-axis velocity.

According to another specific embodiment of the present application, determining the foot position data at least according to the target foot posture data includes: acquiring the posture of the whole body and the skeleton points of the whole body corresponding to a first preset time, wherein the first preset time is the time when each foot part contacts with the ground; determining reference position data of the foot part corresponding to the first predetermined time based on the whole body posture and the whole body skeleton point corresponding to the first predetermined time; acquiring first target foot posture data and second target foot posture data, wherein the first target foot posture data is the target foot posture data corresponding to the first predetermined time, the second target foot posture data is the target foot posture data corresponding to a second predetermined time, and the second predetermined time is a time when the foot is converted from a state of being in contact with the ground corresponding to the first predetermined time to a state of not being in contact with the ground; calculating a difference between the first target foot position data and the reference position data, and correcting the second target foot position data according to the difference; and determining the foot position data corresponding to the second predetermined time based on the corrected second target foot posture data. The accuracy of the whole body posture and the whole body skeleton point at the time when each of the feet is in contact with the ground is high by obtaining the whole body posture and the whole body skeleton point at the time when each of the feet is in contact with the ground, determining the reference position data of the feet based on the whole body posture and the whole body skeleton point at the time when each of the feet is in contact with the ground, ensuring the accuracy of the reference position data to be high, subtracting the reference position data from the first target foot posture data at the time when each of the feet is in contact with the ground, correcting the second target foot posture data at the time when the state where the feet are in contact with the ground is converted into the state where the feet are not in contact with the ground based on the difference, ensuring the accuracy of the foot position data determined based on the corrected second target foot posture data to be high, the accuracy of the foot position data is guaranteed to be high when the feet of the target object are not in contact with the ground, and the accuracy of the determined posture of the whole human body is further guaranteed to be high.

Specifically, a deep learning algorithm can be adopted to calculate the self-carried drift of the IMU, a multilayer convolutional neural network is used to perform convolution on IMU data (such as 200 frames) acquired within a period of time and acquire the drift of the real-time IMU as calibration, the drift data of the IMU is acquired in a specific manner that real-time displacement information of the IMU is acquired through a depth camera or an unmanned aerial vehicle and displacement information obtained by subtracting the IMU is subtracted, so that the drift position information of the IMU is obtained, then 3-axis acceleration, 3-axis angular velocity and 3-axis magnetic field data of the IMU of the previous N-frame time sequence are input and output, the predicted drift is removed from the displacement information predicted by the position tracking algorithm when the IMU is used in real time, and the drift of the IMU due to the self physical characteristics can be successfully reduced through the calibration manner.

According to still another specific embodiment of the present application, in a case where each of the feet of the target object is in contact with the ground surface, the determining the whole body posture and the whole body skeleton point based on the foot pressure data at each of the times includes: acquiring a second deep learning model, wherein the second deep learning model is trained through machine learning by using multiple groups of first data, and each group of data in the multiple groups of first data comprises first historical foot pressure data when the foot of the target object is in contact with the ground, and a corresponding first historical whole body bone point and a first historical whole body posture; inputting the foot pressure data when all of the feet of the target object are in contact with the ground surface into the second deep learning model, and obtaining the whole body posture and the whole body bone point when all of the feet of the target object are in contact with the ground surface, and when there is no contact between the feet and the ground surface, determining the whole body posture and the whole body bone point based on the foot position data at each of the time points and the corresponding foot pressure data, the method including: obtaining a third deep learning model, wherein the third deep learning model is trained through machine learning by using multiple sets of second data, and each set of data in the multiple sets of second data comprises historical foot position data, second historical foot pressure data, corresponding second historical whole body bone points and second historical whole body postures when the foot of the target object is not in contact with the ground; the foot position data and the foot pressure data when the foot of the target object is not in contact with the ground surface are input to the third deep learning model, and the posture of the whole body and the skeleton point of the whole body when the foot of the target object is not in contact with the ground surface are obtained. The method includes obtaining the whole body posture and the whole body skeleton point by discriminating whether each of the feet of the target object is in contact with the ground surface and processing the whole body posture and the whole body skeleton point based on different learning models, and specifically, when each of the feet of the target object is in contact with the ground surface, inputting foot pressure data when each of the feet of the target object is in contact with the ground surface into the second deep learning model, wherein the second deep learning model is trained through machine learning by a plurality of sets of first data, the first data being generated when each of the feet of the target object is in contact with the ground surface, and specifically including the first historical foot pressure data, the corresponding first historical whole body skeleton point, and the first historical whole body posture, thereby ensuring that the obtained whole body posture and the obtained whole body skeleton point are highly accurate when each of the feet of the target object is in contact with the ground surface, inputting the foot position data and the foot pressure data of the target object when the foot does not contact the ground surface, wherein the third deep learning model is trained by machine learning using a plurality of sets of the second data, the second data is generated when the foot of the target object is not in contact with the ground, and specifically includes the historical foot position data, the second historical foot pressure data, the corresponding second historical whole-body skeleton point, and the second historical whole-body posture, so that the accuracy of the whole-body posture and the whole-body skeleton point when the foot of the target object is not in contact with the ground, which are processed by the third deep learning model, is high, and the accuracy of the determined whole-body posture of the human body is further high.

Specifically, the determined range of the foot pressure data is 0-10 newtons, the accuracy is +/-10%, the density is 1 sensing point/square centimeter, more than 250 sensing point data are provided for a single foot, the interface pressure changes caused by different postures can be sufficiently reflected, the error of the position of the human joint point is not more than 3cm through 20 ten thousand data training of a prediction algorithm And comparison with an optical dynamic capture or LIDAR (Light Detection And Ranging) result.

In a specific embodiment, since the human body motion is a coherent change, the subsequent posture change interval can be predicted by a time sequence prediction algorithm such as RNN (Recurrent Neural Network) from the bone posture of the previous N (Nx32x3) frames.

According to an embodiment of the present application, after obtaining foot pressure data and foot posture data of the target object, the method further includes: and under the condition that the foot pressure data meet a preset condition, sending out a thermal sensing signal and a vibration signal, wherein the thermal sensing signal is a signal for controlling the electric heating layer to send out preset heat, and the vibration signal is a signal for controlling the vibration module to generate vibration with a preset amplitude. The thermal sensing signal and the vibration signal are sent out when the foot pressure data of the target user meet the preset condition, so that the target user is guaranteed to have better experience.

In a specific embodiment, when the application scenario is in a Virtual world related scenario such as VR (Virtual Reality) and AR (Augmented Reality), a person skilled in the art may add a head-wearing and hand-wearing device for VR and AR to obtain the specific positions of the head and hands of the target user, and train the target user by deep learning in combination with real skeleton point data to obtain the posture data of the head and hands, because the prior art has a problem of coordinate axes when VR and AR are integrated, because the perceived coordinate axes are different from those used by VR and AR systems, the coordinate axes need to be corrected each time when a whole body motion capture system is used, the process is cumbersome and has long-term errors, in order to avoid the problem, the application may automatically calibrate, by sensing the real-time position of the head of the human body, therefore, when the user uses the system, the displacement data of the head of the user and the displacement data worn by the VR/AR are simultaneously obtained, and the coordinate axes of the displacement data and the displacement data are calibrated, for example, when the VR is shifted from (0, 0, 0) to (1, 1, 1) relative to the previous frame, the head of the target user is automatically detected to be shifted from (-2, -2, -2) to (2, 2, 2) relative to the previous frame, so that the size ratio of the coordinate axes of the system to the coordinate axes of the VR system is 1:4, and the position error is (-2, -2, -2), so that the system can be applied to the whole body motion capture data to achieve the effect of automatically calibrating the coordinate axes.

Specifically, the above-mentioned determination method of the whole body posture of the human body can be applied to Unity (three-dimensional interactive content creation and operation platform), Unreal Engine (Unreal Engine) or other 3D (3-Dimension, three-dimensional) rendering engines, and is imported into the 3D game in the fbx (film box) format.

The embodiment of the present application further provides a device for determining a whole body posture of a human body, and it should be noted that the device for determining a whole body posture of a human body according to the embodiment of the present application can be used for executing the method for determining a whole body posture of a human body according to the embodiment of the present application. The following describes a device for determining a posture of a whole body of a human body according to an embodiment of the present application.

Fig. 2 is a schematic diagram of a human body posture determination apparatus according to an embodiment of the present application. As shown in fig. 2, the apparatus includes an acquisition unit 10 and a determination unit 20, wherein the acquisition unit 10 is configured to acquire foot pressure data and foot position data of a target object at each time, the foot position data including position data corresponding to a contact of a foot of the target object with a ground surface and position data corresponding to a non-contact of the foot with the ground surface; the specifying unit 20 is configured to specify the whole body posture and the whole body skeleton point of the target object based on at least one of the foot pressure data and the foot position data at each time.

The apparatus for determining a posture of the whole body of the human body may be configured to acquire foot pressure data and foot position data of a target object at each time point by the acquisition unit, the foot position data including position data corresponding to a contact of a foot of the target object with a ground surface and position data corresponding to a non-contact of the foot with the ground surface; the whole body posture and the whole body skeleton point of the target object are determined by the determination means based on at least one of the foot pressure data and the foot position data at each time. Compared with the problem that the whole body posture of the human body cannot be simply and accurately predicted in the prior art, the device for determining the whole body posture of the human body of the application acquires the foot pressure data and the foot position data, wherein the foot position data comprises position data corresponding to the contact between the foot of the target object and the ground and position data corresponding to the non-contact between the foot of the target object and the ground, and determines the whole body posture and the whole body skeleton point of the target object according to the foot pressure data and the foot position data, so that the whole body posture and the whole body skeleton point corresponding to the contact between the foot of the target object and the ground can be accurately obtained, the whole body posture and the whole body skeleton point corresponding to the non-contact between the foot of the target object and the ground can be accurately obtained, and the high-accuracy determination can be realized only according to the foot pressure data and the foot position data Above-mentioned whole body gesture and above-mentioned whole body skeleton point have avoided needing to set up a plurality of sensors in other positions of above-mentioned target object among the prior art, have guaranteed that the process of confirming above-mentioned whole body gesture of human is comparatively simple and the cost is lower, have guaranteed that above-mentioned target user's experience feels better, have solved the problem that can not simple and accurate prediction whole body gesture of human among the prior art, have guaranteed that the accuracy of the whole body gesture of above-mentioned human of confirming is higher.

According to a specific embodiment of the present application, the obtaining unit includes a first obtaining module, a correcting module, and a first determining module, where the first obtaining module is configured to obtain the foot pressure data and the corresponding foot posture data at each of the moments in real time; the correction module is used for correcting the corresponding foot posture data at least according to the foot pressure data to obtain target foot posture data; the first determining module is configured to determine the foot position data at least according to the target foot posture data. The foot pressure data and the corresponding foot posture data at each moment are acquired, and the corresponding foot posture data are corrected according to the foot pressure data, so that the accuracy of the obtained target foot posture data is high, the foot position data is determined according to the target foot posture data, the accuracy of the foot position data determined according to the target foot posture data is high, the determined whole body posture and the determined whole body skeleton point of the target object are high, and the determined whole body posture of the human body is high.

According to another specific embodiment of the present application, the determination unit includes a second determination module and a third determination module, wherein the second determination module is configured to determine the whole body posture and the whole body skeleton point from the foot pressure data at each of the time points when each of the feet of the target object is in contact with the ground; the third determining module is configured to determine the whole body posture and the whole body skeleton point based on the foot position data and the corresponding foot pressure data at each of the time points when the foot does not contact the ground. Determining the whole body posture and the whole body skeleton point based on the foot pressure data at each of the times by distinguishing whether each of the feet of the target object is in contact with the ground surface, and when each of the feet of the target object is in contact with the ground surface, determining the whole body posture and the whole body skeleton point based on the foot pressure data at each of the times, ensuring high accuracy of the whole body posture and the whole body skeleton point determined based on the foot pressure data, and when the foot is not in contact with the ground surface, since the foot away from the ground surface does not generate the foot pressure data, determining the whole body posture and the whole body skeleton point based on the foot position data and the corresponding foot pressure data at each of the times, further ensuring high accuracy of the whole body posture and the whole body skeleton point determined when the foot is not in contact with the ground surface, further ensuring the higher accuracy of the determined posture of the whole body of the human body.

In order to further ensure that the accuracy of the human body whole-body posture is high, according to another specific embodiment of the present application, the third determining module includes a first determining submodule and a processing submodule, wherein the first determining submodule is configured to determine an initial whole-body posture and an initial whole-body bone point of the target object according to the foot pressure data and the foot position data; the processing submodule is configured to perform predetermined processing on the initial whole-body posture and the initial whole-body skeleton point to obtain the whole-body posture and the whole-body skeleton point, where the predetermined processing includes at least one of: kalman filtering processing, multi-layer perceptron processing and extended Kalman filter processing. By performing the predetermined processing on the initial whole body skeleton points determined by the foot pressure data and the foot position data, the obtained whole body posture and the accuracy of the whole body skeleton points are ensured to be higher, and the accuracy of the determined whole body posture of the human body is further ensured to be higher.

Specifically, the kalman filter is an algorithm for performing optimal estimation on the system state by inputting and outputting observation data through a system by using a linear system state equation, and the observation data includes the influence of noise and interference in the system, so the optimal estimation can also be regarded as a filtering process, and the data filtering is a data processing technology for removing noise and restoring real data.

According to a specific embodiment of the present application, the correction module includes a fitting submodule and a first calculation submodule, where the fitting submodule is configured to fit first foot acceleration data by using a logistic regression algorithm or a linear regression algorithm to determine acceleration error data, the first foot acceleration data is foot acceleration data corresponding to each foot when the foot is in contact with the ground, and the acceleration error data is error data representing a target device that acquires the foot posture data; the first calculation submodule is configured to subtract, when the foot is not in contact with the ground surface from the state in which the foot is in contact with the ground surface, the acceleration error data from second foot acceleration data, which is the foot acceleration data corresponding to the case in which the foot is not in contact with the ground surface, to obtain second target acceleration data. The method comprises the steps of fitting foot acceleration data corresponding to the feet in contact with the ground by using a logistic regression algorithm or a linear regression algorithm to determine acceleration error data, and subtracting the acceleration error data from the foot acceleration data corresponding to the feet not in contact with the ground when the feet are in contact with the ground and are not in contact with the ground, so that the accuracy of second target acceleration data obtained by removing the acceleration error data is high, the accuracy of the target foot posture data is high, the accuracy of the whole body posture and the whole body bone points determined according to the foot posture data is high, and the accuracy of the determined whole body posture of the human body is further high.

Specifically, since the physical characteristics of the target device that collects the foot posture data make it impossible to maintain accurate position sensing for a long period of time, drift is likely to occur, and the first foot acceleration data corresponding to each foot in contact with the ground is small in the practical application, the acceleration generated at this time is mainly determined by the accumulated error of the target device that collects the foot posture data, the acceleration error data is determined by determining the foot acceleration data corresponding to each foot in contact with the ground, and the acceleration error data is subtracted from the second foot acceleration data generated when the subsequent foot is not in contact with the ground, and the correction is performed each time the target user makes contact with the ground, such high-frequency correction enables the target device to maintain real-time and accurate tracking of the foot, the obtained second target acceleration data does not include error data caused by the target equipment, the accuracy of the second target acceleration data is high, the accuracy of the target foot posture data is high, the accuracy of the whole body posture and the whole body skeleton point determined according to the foot posture data is high, and the accuracy of the determined whole body posture of the human body is further high.

According to another specific embodiment of the present application, the first determining module includes a first obtaining sub-module, a second determining sub-module, a second obtaining sub-module, a second calculating sub-module, and a third determining sub-module, where the first obtaining sub-module is configured to obtain the whole body posture and the whole body bone point corresponding to a first predetermined time, where the first predetermined time is a time when each of the feet contacts with the ground; the second determining submodule is configured to determine reference position data of the foot corresponding to the first predetermined time, based on the whole body posture and the whole body skeleton point corresponding to the first predetermined time; the second obtaining sub-module is configured to obtain first target foot posture data and second target foot posture data, the first target foot posture data being the target foot posture data corresponding to the first predetermined time, the second target foot posture data being the target foot posture data corresponding to a second predetermined time, the second predetermined time being a time when the foot is switched from a state of being in contact with the ground surface corresponding to the first predetermined time to a state of not being in contact with the ground surface; the second calculation submodule is configured to calculate a difference between the first target foot position data and the reference position data, and correct the second target foot position data according to the difference; the third determining submodule is configured to determine the foot position data corresponding to the second predetermined time, based on the corrected second target foot posture data. The accuracy of the whole body posture and the whole body skeleton point at the time when each of the feet is in contact with the ground is high by obtaining the whole body posture and the whole body skeleton point at the time when each of the feet is in contact with the ground, determining the reference position data of the feet based on the whole body posture and the whole body skeleton point at the time when each of the feet is in contact with the ground, ensuring the accuracy of the reference position data to be high, subtracting the reference position data from the first target foot posture data at the time when each of the feet is in contact with the ground, correcting the second target foot posture data at the time when the state where the feet are in contact with the ground is converted into the state where the feet are not in contact with the ground based on the difference, ensuring the accuracy of the foot position data determined based on the corrected second target foot posture data to be high, the accuracy of the foot position data when the foot of the target object is not in contact with the ground is ensured to be higher, and the accuracy of the determined posture of the whole body of the human body is further ensured to be higher.

According to another specific embodiment of the present application, the second determining module includes a third obtaining sub-module and a first input sub-module, wherein the third obtaining sub-module is configured to obtain a second deep learning model, the second deep learning model is trained through machine learning by using multiple sets of first data, each of the multiple sets of first data includes first historical foot pressure data and corresponding first historical whole-body bone points and first historical whole-body postures when the foot of the target object is in contact with the ground; the first input submodule is configured to input the foot pressure data of the target object when the feet of the target object are all in contact with the ground into the second deep learning model, to obtain the whole body posture and the whole body skeleton point when the feet of the target object are all in contact with the ground, the third determining module further includes a fourth obtaining submodule and a second input submodule, wherein the fourth obtaining submodule is configured to obtain a third deep learning model, the third deep learning model is trained through machine learning by using multiple sets of second data, each set of data in the plurality of sets of second data includes historical foot position data, second historical foot pressure data, and corresponding second historical whole body bone points and second historical whole body postures when the foot of the target object is not in contact with the ground; the second input submodule is configured to input the foot position data and the foot pressure data of the target object when the foot of the target object is not in contact with the ground surface into the third deep learning model, and obtain the whole body posture and the whole body skeleton point of the target object when the foot of the target object is not in contact with the ground surface. The method includes obtaining the whole body posture and the whole body skeleton point by discriminating whether each of the feet of the target object is in contact with the ground surface and processing the same according to a different learning model, and specifically, when each of the feet of the target object is in contact with the ground surface, inputting foot pressure data when each of the feet of the target object is in contact with the ground surface into the second deep learning model, wherein the second deep learning model is trained through machine learning by a plurality of sets of first data, the first data being generated when each of the feet of the target object is in contact with the ground surface and specifically including the first historical foot pressure data, the corresponding first historical whole body skeleton point, and the first historical whole body posture, thereby ensuring high accuracy of the obtained whole body posture and the whole body skeleton point when each of the feet of the target object is in contact with the ground surface, inputting, when the foot does not contact the ground surface, the foot position data and the foot pressure data obtained when the foot of the target object does not contact the ground surface into a third deep learning model, the third deep learning model being trained through machine learning using a plurality of sets of second data generated when the foot of the target object does not contact the ground surface, the third deep learning model including the historical foot position data, the second historical foot pressure data, the corresponding second historical whole body skeleton point, and the second historical whole body posture, the accuracy of the whole body posture and the whole body skeleton point being ensured when the foot of the target object processed by the third deep learning model does not contact the ground surface, further ensuring the higher accuracy of the determined posture of the whole body of the human body.

Specifically, the determined range of the foot pressure data is 0-10 newtons, the accuracy is +/-10%, the density is 1 sensing point/square centimeter, more than 250 pieces of sensing point data are provided for a single foot, the interface pressure change caused by different postures can be sufficiently reflected, the error of the position of the human joint point is not more than 3cm through 20 ten thousand pieces of data training of a prediction algorithm And comparison with an optical dynamic capture or LIDAR (Light Detection And Ranging) result.

According to an embodiment of the present application, the apparatus further includes an issuing unit, where the issuing unit is configured to issue a thermal sensing signal and a vibration signal when the foot pressure data satisfies a predetermined condition after acquiring the foot pressure data and the foot posture data of the target object, the thermal sensing signal is a signal for controlling the electrothermal layer to issue a predetermined amount of heat, and the vibration signal is a signal for controlling the vibration module to generate vibration of a predetermined magnitude. The thermal sensing signal and the vibration signal are sent out when the foot pressure data of the target user meet the preset condition, so that the target user is guaranteed to have better experience.

In a specific embodiment, when the application scenario is in a Virtual world related scenario such as VR (Virtual Reality) and AR (Augmented Reality), a person skilled in the art may add a head-wearing and hand-wearing device for VR and AR to obtain the specific positions of the head and hands of the target user, and train the target user by deep learning in combination with real skeleton point data to obtain the posture data of the head and hands, because the prior art has a problem of coordinate axes when VR and AR are integrated, because the perceived coordinate axes are different from those used by VR and AR systems, the coordinate axes need to be corrected each time when a whole body motion capture system is used, the process is cumbersome and has long-term errors, in order to avoid the problem, the application may automatically calibrate, by sensing the real-time position of the head of the human body, therefore, when the user uses the system, the displacement data of the head of the user and the displacement data worn by the VR/AR are simultaneously obtained, and the coordinate axes of the displacement data are calibrated, for example, when the VR is shifted from (0, 0, 0) to (1, 1, 1) relative to the previous frame, the head of the target user is automatically detected to be shifted from (-2, -2, -2) to (2, 2, 2) relative to the previous frame, so that the size ratio of the coordinate axes of the system to the coordinate axes of the VR system is 1:4, and the position error is (-2, -2, -2), so that the system can be applied to the whole body motion capture data to achieve the effect of automatically calibrating the coordinate axes.

The device for determining the posture of the whole body of the human body comprises a processor and a memory, wherein the acquisition unit, the determination unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to realize corresponding functions.

The processor comprises a kernel, and the kernel calls the corresponding program unit from the memory. The kernel can be set to be one or more than one, and the problem that the posture of the whole body of the human body cannot be simply and accurately predicted in the prior art is solved by adjusting the kernel parameters.

The memory may include volatile memory in a computer readable medium, Random Access Memory (RAM) and/or nonvolatile memory such as Read Only Memory (ROM) or flash memory (flash RAM), and the memory includes at least one memory chip.

An embodiment of the present invention provides a computer-readable storage medium, on which a program is stored, which, when executed by a processor, implements the above-described method for determining a whole-body posture of a human body.

The embodiment of the invention provides a processor, which is used for running a program, wherein the program executes the method for determining the posture of the whole body of the human body when running.

The embodiment of the invention provides equipment, which comprises a processor, a memory and a program which is stored on the memory and can run on the processor, wherein when the processor executes the program, at least the following steps are realized:

The device herein may be a server, a PC, a PAD, a mobile phone, etc.

The present application further provides a computer program product adapted to perform a program of initializing at least the following method steps when executed on a data processing device:

According to another exemplary embodiment of the application, a smart shoe is further provided, which includes an inner layer and an outer layer, wherein the inner layer includes a pressure sensor layer, and the pressure sensor is used for acquiring foot pressure data; the outer layer comprises a processing circuit for performing any of the above methods and an IMU for obtaining foot position data comprising position data corresponding to a target subject's foot in contact with a ground surface and position data corresponding to the foot not in contact with the ground surface.

The intelligent shoe comprises an inner layer and an outer layer, wherein the inner layer comprises a pressure sensor layer, and the pressure sensor layer is used for acquiring foot pressure data; the outer layer comprises a processing circuit for performing any of the above methods and an IMU for obtaining foot position data comprising position data corresponding to a target subject's foot in contact with a ground surface and position data corresponding to the foot not in contact with the ground surface. Compared with the problem that the whole body posture of the human body cannot be simply and accurately predicted in the prior art, the intelligent shoe of the application determines the whole body posture and the whole body skeleton points of the target object according to the foot pressure data and the foot position data by acquiring the foot pressure data and the foot position data, wherein the foot position data comprises position data corresponding to the contact between the foot of the target object and the ground and position data corresponding to the non-contact between the foot of the target object and the ground, so that the whole body posture and the whole body skeleton points corresponding to the contact between the foot of the target object and the ground can be accurately obtained, the whole body posture and the whole body skeleton points corresponding to the non-contact between the foot of the target object and the ground can be accurately obtained, and the whole body posture and the whole body skeleton points with high accuracy can be determined according to the foot pressure data and the foot position data only Above-mentioned whole body skeleton point has avoided needing to set up a plurality of sensors in other positions of above-mentioned target object among the prior art, has guaranteed that the process of confirming above-mentioned whole body gesture of human is comparatively simple and the cost is lower, has guaranteed that above-mentioned target user's experience feels better, has solved the problem that can not simple and accurate prediction whole body gesture of human among the prior art, has guaranteed that the accuracy of the whole body gesture of above-mentioned human of confirmation is higher.

In a specific embodiment, the whole body posture and the whole body skeleton point with higher accuracy of the target object can be determined through the intelligent shoe, so that the high practicability of the intelligent shoe is ensured.

Specifically, the sensors in the pressure sensor layers in the intelligent shoes are flexible fabric pressure sensors, wherein 250 pressure sensing points are arranged in each intelligent shoe, the IMU adopts BNO085 and is equipped with three-axis acceleration, three-axis angular velocity and three-axis magnetic force detection, the processing circuit adopts a teensy4.1 chip, the processing speed reaches 600MHz, 250 sensing points can be subjected to data collection in real time, a human body motion model is generated through an algorithm, and the delay is not more than 10 ms.

According to a specific embodiment of the present application, the inner layer further includes an electric heating layer and a vibration module, wherein the electric heating layer is located on one side of the pressure sensor layer, and the electric heating layer is used for emitting a predetermined amount of heat; the vibration module is positioned on one side of the electric heating layer far away from the pressure sensor layer and is used for generating vibration with a preset amplitude. Through setting up above-mentioned electric heat layer and above-mentioned vibrations module, above-mentioned electric heat layer sends predetermined heat, and above-mentioned vibrations module produces the vibrations of above-mentioned predetermined range, has guaranteed that above-mentioned target user's experience feels better.

In the above embodiments of the present invention, the descriptions of the respective embodiments have respective emphasis, and for parts that are not described in detail in a certain embodiment, reference may be made to related descriptions of other embodiments.

In the embodiments provided in the present application, it should be understood that the disclosed technology can be implemented in other ways. The above-described embodiments of the apparatus are merely illustrative, and for example, the above-described division of the units may be a logical division, and in actual implementation, there may be another division, for example, multiple units or components may be combined or may be integrated into another system, or some features may be omitted, or not executed. In addition, the shown or discussed mutual coupling or direct coupling or communication connection may be an indirect coupling or communication connection through some interfaces, units or modules, and may be in an electrical or other form.

The units described as separate parts may or may not be physically separate, and parts displayed as units may or may not be physical units, may be located in one place, or may be distributed on a plurality of units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of the embodiment.

In addition, functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may exist alone physically, or two or more units are integrated into one unit. The integrated unit can be realized in a form of hardware, and can also be realized in a form of a software functional unit.

The integrated unit may be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as a separate product. Based on such understanding, the technical solution of the present invention may be embodied in the form of a software product, which is stored in a storage medium and includes instructions for causing a computer device (which may be a personal computer, a server, or a network device) to execute all or part of the steps of the above methods according to the embodiments of the present invention. And the aforementioned storage medium includes: a U-disk, a Read-Only Memory (ROM), a Random Access Memory (RAM), a removable hard disk, a magnetic disk, or an optical disk, and various media capable of storing program codes.

From the above description, it can be seen that the above-described embodiments of the present application achieve the following technical effects:

1) first, foot pressure data and foot position data of a target object at each time are acquired, the foot position data including position data corresponding to a contact of a foot of the target object with a ground surface and position data corresponding to a non-contact of the foot with the ground surface; then, the whole body posture and the whole body skeleton point of the target object are determined based on at least one of the foot pressure data and the foot position data at each time. Compared with the problem that the whole body posture of the human body cannot be simply and accurately predicted in the prior art, the method for determining the whole body posture of the human body according to the application obtains the foot pressure data and the foot position data, wherein the foot position data comprise position data corresponding to the contact between the foot of the target object and the ground and position data corresponding to the non-contact between the foot of the target object and the ground, and then determines the whole body posture and the whole body skeleton point of the target object according to the foot pressure data and the foot position data, so that the whole body posture and the whole body skeleton point corresponding to the contact between the foot of the target object and the ground can be relatively accurately obtained, the whole body posture and the whole body skeleton point corresponding to the non-contact between the foot of the target object and the ground can be relatively accurately obtained, and the high-accuracy whole body posture and the whole body skeleton point can be determined only according to the foot pressure data and the foot position data Above-mentioned whole body gesture and above-mentioned whole body skeleton point have avoided needing to set up a plurality of sensors in other positions of above-mentioned target object among the prior art, have guaranteed that the process of confirming above-mentioned whole body gesture of human is comparatively simple and the cost is lower, have guaranteed that above-mentioned target user's experience feels better, have solved the problem that can not simple and accurate prediction whole body gesture of human among the prior art, have guaranteed that the accuracy of the whole body gesture of above-mentioned human of confirming is higher.

2) The apparatus for determining a posture of a whole body of a human body according to the present invention is configured to acquire foot pressure data and foot position data of a target object at each time point by the acquisition means, the foot position data including position data corresponding to a contact of a foot of the target object with a ground surface and position data corresponding to a non-contact of the foot with the ground surface; the whole body posture and the whole body skeleton point of the target object are determined by the determination means based on at least one of the foot pressure data and the foot position data at each time. Compared with the problem that the whole body posture of the human body cannot be simply and accurately predicted in the prior art, the device for determining the whole body posture of the human body of the application acquires the foot pressure data and the foot position data, wherein the foot position data comprises position data corresponding to the contact between the foot of the target object and the ground and position data corresponding to the non-contact between the foot of the target object and the ground, and determines the whole body posture and the whole body skeleton point of the target object according to the foot pressure data and the foot position data, so that the whole body posture and the whole body skeleton point corresponding to the contact between the foot of the target object and the ground can be accurately obtained, the whole body posture and the whole body skeleton point corresponding to the non-contact between the foot of the target object and the ground can be accurately obtained, and the high-accuracy determination can be realized only according to the foot pressure data and the foot position data Above-mentioned whole body gesture and above-mentioned whole body skeleton point have avoided needing to set up a plurality of sensors in other positions of above-mentioned target object among the prior art, have guaranteed that the process of confirming above-mentioned whole body gesture of human is comparatively simple and the cost is lower, have guaranteed that above-mentioned target user's experience feels better, have solved the problem that can not simple and accurate prediction whole body gesture of human among the prior art, have guaranteed that the accuracy of the whole body gesture of above-mentioned human of confirming is higher.

3) The intelligent shoe comprises an inner layer and an outer layer, wherein the inner layer comprises a pressure sensor layer, and the pressure sensor layer is used for acquiring foot pressure data; the outer layer comprises a processing circuit for performing any of the above methods and an IMU for obtaining foot position data comprising position data corresponding to a target subject's foot in contact with a ground surface and position data corresponding to the foot not in contact with the ground surface. Compared with the problem that the whole body posture of the human body cannot be simply and accurately predicted in the prior art, the intelligent shoe of the application determines the whole body posture and the whole body skeleton points of the target object according to the foot pressure data and the foot position data by acquiring the foot pressure data and the foot position data, wherein the foot position data comprises position data corresponding to the contact between the foot of the target object and the ground and position data corresponding to the non-contact between the foot of the target object and the ground, so that the whole body posture and the whole body skeleton points corresponding to the contact between the foot of the target object and the ground can be accurately obtained, the whole body posture and the whole body skeleton points corresponding to the non-contact between the foot of the target object and the ground can be accurately obtained, and the whole body posture and the whole body skeleton points with high accuracy can be determined according to the foot pressure data and the foot position data only Above-mentioned whole body skeleton point has avoided needing to set up a plurality of sensors in other positions of above-mentioned target object among the prior art, has guaranteed that the process of confirming above-mentioned whole body gesture of human is comparatively simple and the cost is lower, has guaranteed that above-mentioned target user's experience feels better, has solved the problem that can not simple and accurate prediction whole body gesture of human among the prior art, has guaranteed that the accuracy of the whole body gesture of above-mentioned human of confirmation is higher.

The above description is only a preferred embodiment of the present application and is not intended to limit the present application, and various modifications and changes may be made by those skilled in the art. Any modification, equivalent replacement, improvement and the like made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for determining a posture of a whole body of a human body, the method comprising:

acquiring foot pressure data and foot position data of a target object at each moment, wherein the foot position data comprises position data corresponding to the contact between the foot of the target object and the ground and position data corresponding to the non-contact between the foot and the ground;

and determining the whole body posture and the whole body bone points of the target object according to at least one of the foot pressure data and the foot position data at each moment.

2. The method of claim 1, wherein obtaining foot pressure data and foot position data for the target subject at each time comprises:

acquiring the foot pressure data and the corresponding foot posture data at each moment in real time;

correcting the corresponding foot posture data at least according to the foot pressure data to obtain target foot posture data;

determining the foot position data based at least on the target foot pose data.

3. The method of claim 2, wherein determining a full body pose and full body skeletal points of the target subject from at least one of the foot pressure data and the foot position data at the respective time instants comprises:

determining the whole body posture and the whole body bone points from the foot pressure data at each of the time instants, in a case where each of the feet of the target subject is in contact with the ground;

and under the condition that the feet are not in contact with the ground, determining the whole body posture and the whole body skeleton point according to the foot position data and the corresponding foot pressure data at each time.

4. The method of claim 3, wherein determining the full body pose and the full body skeletal points from the foot position data and the corresponding foot pressure data for each of the time instants comprises:

determining an initial whole body posture and an initial whole body bone point of the target object according to the foot pressure data and the foot position data;

performing predetermined processing on the initial whole-body posture and the initial whole-body bone points to obtain the whole-body posture and the whole-body bone points, wherein the predetermined processing at least comprises one of the following steps: kalman filtering processing, multi-layer perceptron processing and extended Kalman filter processing.

5. The method of claim 3, wherein modifying the corresponding foot pose data based at least on the foot pressure data to obtain target foot pose data comprises:

fitting first foot acceleration data by using a logistic regression algorithm or a linear regression algorithm to determine acceleration error data, wherein the first foot acceleration data is corresponding foot acceleration data when each foot is in contact with the ground, and the acceleration error data is error data representing target equipment for acquiring foot posture data;

and when the foot is in contact with the ground and is not in contact with the ground, subtracting second foot acceleration data from the acceleration error data to obtain second target acceleration data, wherein the second foot acceleration data is the corresponding foot acceleration data when the foot is not in contact with the ground.

6. The method of claim 4, wherein determining the foot position data based at least on the target foot pose data comprises:

acquiring the posture of the whole body and skeleton points of the whole body corresponding to a first preset moment, wherein the first preset moment is the moment when each foot part is in contact with the ground;

determining reference position data of the foot corresponding to the first preset time according to the whole body posture and the whole body skeleton points corresponding to the first preset time;

acquiring first target foot posture data and second target foot posture data, wherein the first target foot posture data is the target foot posture data corresponding to the first preset moment, the second target foot posture data is the target foot posture data corresponding to the second preset moment, and the second preset moment is the moment corresponding to the moment when the foot is converted from the state corresponding to the first preset moment and contacting with the ground into the state not contacting with the ground;

calculating a difference value between the first target foot attitude data and the reference position data, and correcting the second target foot attitude data according to the difference value;

and determining the foot position data corresponding to the second preset time according to the corrected second target foot posture data.

7. The method of claim 5,

determining the full body pose and the full body skeletal points from the foot pressure data at each of the time instants with each of the feet of the target subject in contact with the ground, comprising:

obtaining a second deep learning model, wherein the second deep learning model is trained through machine learning by using multiple groups of first data, and each group of data in the multiple groups of first data comprises first historical foot pressure data when the foot of the target object is in contact with the ground, and corresponding first historical whole body bone points and first historical whole body postures;

inputting the foot pressure data of the target object when the feet of the target object are all in contact with the ground into the second deep learning model to obtain the whole body posture and the whole body skeleton point when the feet of the target object are all in contact with the ground,

determining the whole body posture and the whole body skeletal points according to the foot position data and the corresponding foot pressure data at each time in the case that the foot is not in contact with the ground, comprising:

obtaining a third deep learning model, wherein the third deep learning model is trained through machine learning by using multiple sets of second data, and each set of data in the multiple sets of second data comprises historical foot position data, second historical foot pressure data, corresponding second historical whole body bone points and second historical whole body postures when the foot of the target object is not in contact with the ground;

inputting the foot position data and the foot pressure data of the target object when the foot is not in contact with the ground into the third deep learning model, and obtaining the whole body posture and the whole body bone point of the target object when the foot is not in contact with the ground.

8. The method of claim 1, wherein after obtaining foot pressure data and foot pose data for the target subject, the method further comprises:

and sending out a thermal sensing signal and a vibration signal under the condition that the foot pressure data meets a preset condition, wherein the thermal sensing signal is a signal for controlling the electric heating layer to send out preset heat, and the vibration signal is a signal for controlling the vibration module to generate vibration with a preset amplitude.

9. An apparatus for determining the posture of the whole body of a human body, the apparatus comprising:

an acquisition unit configured to acquire foot pressure data and foot position data of a target object at each time, the foot position data including position data corresponding to a foot of the target object being in contact with the ground and position data corresponding to the foot not being in contact with the ground;

a determination unit configured to determine a whole body posture and a whole body skeleton point of the target object based on at least one of the foot pressure data and the foot position data at each time.

10. A computer-readable storage medium, characterized in that the computer-readable storage medium comprises a stored program, wherein the program performs the method of any one of claims 1 to 8.

11. A processor, characterized in that the processor is configured to run a program, wherein the program when running performs the method of any of claims 1 to 8.

12. A smart shoe, comprising:

the inner layer comprises a pressure sensor layer, and the pressure sensor layer is used for acquiring foot pressure data;

an outer layer comprising processing circuitry for performing the method of any of claims 1 to 8 and an IMU for obtaining foot position data comprising position data corresponding to a target subject's foot in contact with a ground surface and position data corresponding to the foot not in contact with the ground surface.

13. The smart shoe of claim 12, wherein the inner layer further comprises:

the electrothermal layer is positioned on one side of the pressure sensor layer and is used for emitting preset heat;

the vibration module is positioned on one side, far away from the pressure sensor layer, of the electric heating layer and used for generating vibration with a preset amplitude.