CN111539300A - Human motion capture method, device, medium and equipment based on IK algorithm - Google Patents
Human motion capture method, device, medium and equipment based on IK algorithm Download PDFInfo
- Publication number
- CN111539300A CN111539300A CN202010312083.3A CN202010312083A CN111539300A CN 111539300 A CN111539300 A CN 111539300A CN 202010312083 A CN202010312083 A CN 202010312083A CN 111539300 A CN111539300 A CN 111539300A
- Authority
- CN
- China
- Prior art keywords
- target mark
- mark point
- frame image
- lost
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/20—Movements or behaviour, e.g. gesture recognition
- G06V40/23—Recognition of whole body movements, e.g. for sport training
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
- G06T7/251—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments involving models
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30196—Human being; Person
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30204—Marker
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Multimedia (AREA)
- Health & Medical Sciences (AREA)
- General Health & Medical Sciences (AREA)
- Psychiatry (AREA)
- Social Psychology (AREA)
- Human Computer Interaction (AREA)
- Image Analysis (AREA)
Abstract
The embodiment of the invention discloses a human motion capture method, a human motion capture device, a human motion capture medium and human motion capture equipment based on an IK algorithm. The method comprises the following steps: acquiring a target mark point of a user in an initial posture; tracking the image information captured in real time, and predicting the position of a target mark point in the current frame image according to the position of the target mark point in the previous frame image; if the target mark point of the current frame image is lost, correcting the predicted position of the lost target mark point in the current frame image by adopting an IK algorithm according to the tracked target mark point closest to the lost target mark point; and determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point. By adopting the technical scheme provided by the application, all parts of the human body can be tracked and predicted based on the IK algorithm according to the tracking result of the main parts of the body of the user, and the purpose of showing the whole body of the user in real time can be realized.
Description
Technical Field
The embodiment of the invention relates to the technical field of virtual reality, in particular to a human motion capturing method, a human motion capturing device, a human motion capturing medium and human motion capturing equipment based on an IK algorithm.
Background
With the rapid development of the technology level, the virtual reality technology has been gradually pushed to people's leisure and entertainment lives.
Virtual reality technology, taking VR as an example, most of VR devices that users know are mainly head-mounted devices such as HTC VIVE, OculusRift, PSVR, and the like. These head-mounted devices share a common feature of being able to track only a portion of the user's body organ, such as the user's head, hands, etc.
At present, in the interactive large-space VR scheme, only information of two hands and heads of players can be tracked, so that in a mutual co-located virtual space, the players can only see the accurate positions of the heads and the hands of the players, but the body postures are not completely synchronized. This greatly reduced VR's sense of immersion and interactive reality degree, restricted user experience, also restricted industry development.
Disclosure of Invention
The embodiment of the invention provides a human body motion capturing method, a human body motion capturing device, a human body motion capturing medium and human body motion capturing equipment based on an IK algorithm, which can track and predict all parts of a human body based on the IK algorithm according to a tracking result of a main part of the body of a user, so that the aim of displaying the whole body of the user in real time can be fulfilled in a virtual reality interaction process.
In a first aspect, an embodiment of the present invention provides a human motion capture method based on an IK algorithm, where the method includes:
acquiring target mark points of a user in an initial posture, determining the serial number and the joint type of the target mark points, and storing an initial posture model;
tracking the image information captured in real time, and predicting the position of a target mark point in the current frame image according to the position of the target mark point in the previous frame image;
if the target mark point of the current frame image is lost, correcting the predicted position of the lost target mark point in the current frame image by adopting an IK algorithm according to the tracked target mark point closest to the lost target mark point;
and determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point.
Optionally, the modifying, by using an IK algorithm, the predicted position of the missing target marker in the current frame image according to the tracked target marker closest to the missing target marker includes:
determining a bone length constraint and an angle constraint between joints;
and correcting the position of the lost target mark point in the predicted current frame image by adopting an IK algorithm and combining the skeleton length constraint and the angle constraint between joints according to the tracked target mark point closest to the lost target mark point.
Optionally, after determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point, the method further includes:
and outputting the current posture of the user through the VRPN.
Optionally, before acquiring the target mark point of the user in the initial posture, the method further includes:
acquiring a calibration image of the T-shaped rod calibration tool through the tracking camera;
and calibrating the tracking camera according to the calibration image of the T-shaped rod calibration tool.
Optionally, after calibrating the tracking camera, the method further includes:
the room coordinate system is established by means of the calibration image of the L-bar calibration tool.
Optionally, obtaining a target mark point of the user in the initial posture includes:
acquiring a reflective mark point worn by a user; the marking points are distributed on each joint point of the human body and are used for acquiring the posture of the joint; wherein, each joint point has at least 3 marking points.
And determining the human body initial posture bone model when the user is in the initial posture.
Optionally, predicting the position of the target mark point in the current frame image according to the position of the target mark point in the previous frame image, including:
predicting the position of a target mark point in the current frame image by adopting a Kalman filtering algorithm according to the position of the target mark point in the previous frame image;
and traversing all the target mark points.
In a second aspect, an embodiment of the present invention further provides an online human motion capture device based on an IK algorithm, including:
the target mark point determining module is used for acquiring a target mark point of a user in an initial posture, determining the serial number and the joint type of the target mark point and storing an initial posture model;
the position prediction module is used for tracking the image information captured in real time and predicting the position of the target mark point in the current frame image according to the position of the target mark point in the previous frame image;
the lost mark point correcting module is used for correcting the predicted position of the lost target mark point in the current frame image by adopting an IK algorithm according to the tracked target mark point closest to the lost target mark point if the target mark point is lost in the current frame image;
and the current posture determining module is used for determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point.
In a third aspect, the present application provides a computer-readable storage medium, on which a computer program is stored, which when executed by a processor, implements the human motion capture method based on the IK algorithm according to the present application.
In a fourth aspect, the present application provides a mobile device, including a memory, a processor, and a computer program stored on the memory and executable on the processor, wherein the processor executes the computer program to implement the human motion capture method based on the IK algorithm according to the embodiment of the present application.
According to the technical scheme provided by the embodiment of the application, the target mark points of a user in an initial posture are obtained, the serial number and the joint type of the target mark points are determined, and an initial posture model is stored; tracking the image information captured in real time, and predicting the position of a target mark point in the current frame image according to the position of the target mark point in the previous frame image; if the target mark point of the current frame image is lost, correcting the predicted position of the lost target mark point in the current frame image by adopting an IK algorithm according to the tracked target mark point closest to the lost target mark point; and determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point. By adopting the technical scheme provided by the application, all parts of the human body can be tracked and predicted based on the IK algorithm according to the tracking result of the main parts of the body of the user, so that the aim of displaying the whole body of the user in real time can be fulfilled in the interaction process of virtual reality.
Drawings
Fig. 1 is a flowchart of a human motion capture method based on an IK algorithm according to an embodiment of the present invention;
FIG. 2 is a schematic diagram of a human motion capture process based on an IK algorithm according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a human motion capture device based on IK algorithm according to a second embodiment of the present invention;
fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present application.
Detailed Description
The present invention will be described in further detail with reference to the accompanying drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be further noted that, for the convenience of description, only some of the structures related to the present invention are shown in the drawings, not all of the structures.
Before discussing exemplary embodiments in more detail, it should be noted that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although a flowchart may describe the steps as a sequential process, many of the steps can be performed in parallel, concurrently or simultaneously. In addition, the order of the steps may be rearranged. The process may be terminated when its operations are completed, but may have additional steps not included in the figure. The processes may correspond to methods, functions, procedures, subroutines, and the like.
Example one
Fig. 1 is a flowchart of an IK algorithm based human motion capture method according to an embodiment of the present invention, which is applicable to an IK algorithm based human motion capture situation, and can be executed by an IK algorithm based human motion capture apparatus provided by an embodiment of the present invention, where the apparatus can be implemented by software and/or hardware, and can be integrated into an electronic device such as a human tracking system for virtual reality.
As shown in fig. 1, the human motion capture method based on the IK algorithm includes:
s110, acquiring target mark points of a user in an initial posture, determining the serial number and the joint type of the target mark points, and storing an initial posture model.
The initial posture of the user can be a T-shaped posture with two arms lifted flat, and in the posture, the mark points worn on each joint position by the user are determined. The target mark points are arranged at key joints of a user, such as the head, the waist, the shoulders, the elbows, the hands, the crotch, the knees and the feet, no less than 3 mark points are distributed on each joint, and the structure distribution of the mark points on the same joint cannot change along with the movement of a human body and is relatively fixed.
After the marker points are successfully identified, the marker points can be numbered, and the joint types of the target marker points can be determined. The serial number and the joint type of the target mark point can be input by a worker or automatically bound according to the skeleton structure of the human body.
In this embodiment, optionally, the obtaining of the target mark point of the user in the initial posture includes:
acquiring a reflective mark point worn by a user; the marking points are distributed on each joint point of the human body and are used for acquiring the posture of the joint; wherein, each joint point has at least 3 marking points. And determining the human body initial posture bone model when the user is in the initial posture.
The mark point features may be preset, for example, features of mark points at different positions of a user are preset, so that the target mark point can be determined as long as the corresponding features can be obtained. The specific acquisition mode may be acquired by at least one camera. The marker may be characterized by color, shape, or a combination thereof.
In this embodiment, optionally, before determining the marker to be tracked according to the preset marker feature, the method further includes: acquiring a calibration image of the T-shaped rod calibration tool through the tracking camera; and calibrating the tracking camera according to the calibration image of the T-shaped rod calibration tool. Wherein the tracking cameras are mounted equidistantly on top of a large screen or on the ceiling.
Wherein the number of tracking cameras may be 8 or more.
The tracking cameras are used for determining the positions of the human body mark points, and the human body mark points are distributed on the whole body of a user, so that the number of the tracking cameras needs to be 8 or more than 8, the tracking cameras are distributed annularly, the whole body of the human body can be tracked, and no dead angle exists. The tracking camera is calibrated by the T-shaped rod calibration tool, or images are simultaneously acquired by the tracking camera after the T-shaped rod calibration tool is fixed, and calibration is performed according to the characteristics of the T-shaped rod calibration tool in the images and is calibrated to the same coordinate system. The method is beneficial to subsequent determination of the spatial position of the display data, and can be used within a period of time after calibration, so that the use and operation of a user are simple and convenient.
In this embodiment, optionally, after calibrating the tracking camera, the method further includes: the room coordinate system is established by means of the calibration image of the L-bar calibration tool.
The L-shaped rod calibration tool and the T-shaped rod calibration tool are respectively convenient for users to distinguish and use, and can determine the origin of coordinates of a room coordinate system according to the vertex position of the L-shaped rod calibration tool and complete the establishment of the room coordinate system.
And S120, tracking the image information captured in real time, and predicting the position of the target mark point in the current frame image according to the position of the target mark point in the previous frame image.
Specifically, a camera may be used to capture the image and identify the mark points in the captured image. And determining the identified marker points as the tracked marker bodies. And further, the postures and the positions of all joints of the human body can be determined according to the space positions of the light reflecting balls with the tracked mark points.
In this embodiment, optionally, predicting the position of the target mark point in the current frame image according to the position of the target mark point in the previous frame image includes: predicting the position of a target mark point in the current frame image by adopting a Kalman filtering algorithm according to the position of the target mark point in the previous frame image; and traversing all the target mark points.
The method comprises the steps of tracking real-time image information captured by a camera, adopting Kalman filtering to estimate the position P of a marker in a current frame according to the position of the marker in the previous frame, predicting the back projection of the position P to all cameras according to a camera calibration result to find corresponding image points P1, P2 and … … pm, and when m is larger than or equal to 2, carrying out three-dimensional reconstruction to obtain the three-dimensional coordinates of the marker in the current frame.
And after all the markers are traversed circularly, after the residual image points obtain a corresponding matching point set based on limit matching, performing three-dimensional reconstruction to obtain a new marker and automatically generating a new marker ID.
And S130, if the target mark point of the current frame image is lost, correcting the position of the predicted lost target mark point in the current frame image by adopting an IK algorithm according to the tracked target mark point closest to the lost target mark point.
For example, if the target mark points at the left elbow and left wrist positions in the current frame image are lost, the IK algorithm may be adopted to correct the predicted positions of the target mark points at the left elbow and left wrist in the current frame image according to the position of the tracked target mark point closest to the lost target mark point, that is, the target mark point at the left shoulder.
In this embodiment, the modifying the predicted position of the target marker point lost in the current frame image by using an IK algorithm according to the tracked target marker point closest to the target marker point lost includes: determining a bone length constraint and an angle constraint between joints; and correcting the position of the lost target mark point in the predicted current frame image by adopting an IK algorithm and combining the skeleton length constraint and the angle constraint between joints according to the tracked target mark point closest to the lost target mark point.
The length constraint of the skeleton and the angle constraint between the joints can be used as auxiliary factors, and the positions of the lost target mark points can be corrected more accurately. For example, the constraint on the length of the bone from the left shoulder to the left elbow may be a specific value, such as 35 cm, and the distance from the left elbow to the left wrist may be 25-30 cm, such as 27 cm, with the constraint on the angle at the elbow being 30-180 degrees. By such constraint conditions, more reasonable joint positions of the user can be obtained.
And S140, determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point.
The initial posture model can determine the position and the space posture of each joint of the user under the condition of the initial posture, and can determine the positions and the postures of all the mark points of the user by combining the recognized target mark points and the correction result of the predicted positions of the target mark points, so as to obtain the current posture of the user.
In this embodiment, optionally, after determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point, the method further includes: and outputting the current posture of the user through the VRPN.
After the current posture of the user is obtained, the current posture can be output, for example, the current posture of the user is displayed in real time in the AR interaction process, and the use experience of the user is improved.
According to the technical scheme provided by the embodiment of the application, the target mark points of a user in an initial posture are obtained, the serial number and the joint type of the target mark points are determined, and an initial posture model is stored; tracking the image information captured in real time, and predicting the position of a target mark point in the current frame image according to the position of the target mark point in the previous frame image; if the target mark point of the current frame image is lost, correcting the predicted position of the lost target mark point in the current frame image by adopting an IK algorithm according to the tracked target mark point closest to the lost target mark point; and determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point. By adopting the technical scheme provided by the application, all parts of the human body can be tracked and predicted based on the IK algorithm according to the tracking result of the main parts of the body of the user, so that the aim of displaying the whole body of the user in real time can be fulfilled in the interaction process of virtual reality.
In the above technical solution, estimating the position of the target mark point that is not successfully matched according to the serial number, the joint type, and the position of the target mark point that is successfully matched, includes: and estimating the posture and the position of the target mark point which is not successfully matched by adopting human joint length constraint limitation and a Kalman filtering algorithm according to the serial number, the joint type and the position of the target mark point which is successfully matched. Wherein, the positions of the target mark points which are not successfully matched are estimated by using the joint type, such as the hand, the joint length, namely the distance from the elbow to the hand, which is generally 18 cm to 30 cm, and by using a Kalman filtering algorithm. Kalman filtering is an algorithm for performing optimal estimation on the system state by using a linear system state equation and inputting and outputting observation data through a system. The optimal estimation can also be seen as a filtering process, since the observed data includes the effects of noise and interference in the system. Data filtering is a data processing technique for removing noise and restoring true data, and Kalman filtering can estimate the state of a dynamic system from a series of data with measurement noise under the condition that measurement variance is known. The system is convenient for the realization of computer programming and can update and process the data acquired on site in real time.
By adopting the scheme, the position of the target mark point which is successfully matched can be estimated more accurately, and the position and the posture of the joint point are calculated, so that the current posture of the user can be displayed accurately.
In order to make the solution more obvious to those skilled in the art, the present application also provides the following specific preferred embodiments:
fig. 2 is a schematic diagram of a human motion capture process based on an IK algorithm according to an embodiment of the present invention. As shown in fig. 2, the processing procedure of the present invention mainly includes the following steps:
acquiring user images through n tracking cameras;
acquiring a synchronous image dataset;
determining whether the camera is calibrated successfully, if so, performing the following steps, and if not, performing camera calibration of n tracking cameras;
predicting the position P of the current frame by Kalman filtering of a marker in the previous frame image;
the estimated position P is back-projected to all cameras, and corresponding image points P1, P2, … and pn are found;
traversing all markers;
p1, p2, …, pn carry on the three-dimensional reconstruction to get the three-dimensional coordinate of the markers of the current frame;
if the residual image points exist, matching all residual image point polar lines with a new marker for three-dimensional reconstruction and automatically generating a new marker ID;
if no Human exists, a new Human model is established, and if yes, whether the marker loss condition exists in the Human is determined;
if the loss exists, the lost marker estimates a possible position according to the combination of the predicted position P and an IK algorithm;
calculating the position and the posture of each joint according to the marker;
and outputting the position and the posture of the Human joint.
Firstly, system building.
A circle of tracking cameras (8-16) are arranged on the outer circle of the range of the space to be tracked to form a tracking space with overlapped view fields; all the tracking cameras are connected with the motion capture analysis unit through the network cable; camera calibration of the tracking software.
After the tracking software is started, firstly, under the condition that the camera is not calibrated, the T-shaped rod is used for calibrating the camera to obtain the pose relation between the camera and the camera, and the position tracking of the reflective ball in the space is realized.
Firstly, under the condition of no calibration, a T-shaped rod is used for camera calibration to obtain the pose relation between the camera and the camera, and the position tracking of the reflective ball in the space is realized.
Then, room coordinate system setting is performed with an L-shaped bar.
Secondly, a new Human is built.
The method comprises the steps that a person wears a moving capture suit with a reflective ball, walks into the center of a common view field of a camera, opens two hands, stands upright and forms a T shape, each identified target point belonging to the person is selected by a mouse frame on a tracking software interface, and the information of a human body three-dimensional structure template is stored as an initial posture.
And thirdly, three-dimensional reconstruction.
Tracking the real-time image information captured by a camera, adopting kalman filtering to estimate the position P of a Marker in a current frame according to the position of the Marker in the previous frame, predicting that the position P is back-projected to all cameras to find corresponding image points P1, P2 and … … pm according to the calibration result of the cameras, and when m is more than or equal to 2, performing three-dimensional reconstruction to obtain the three-dimensional coordinate of the Marker in the current frame;
and after all the markers are traversed circularly, after the residual image points obtain a corresponding matching point set based on limit matching, performing three-dimensional reconstruction to obtain a new marker and automatically generating a new marker ID.
And fourthly, correcting the position of the human body marker by an IK algorithm.
When a marker loss condition exists in Human, according to a predicted position P of the marker, a mainstream IK algorithm is utilized, length constraint of a Human skeleton and angle constraint between joints are combined, the predicted position P is corrected according to a tracked marker which is closest to the lost marker, and when the tracked marker conflicts with a newly generated marker, the newly generated marker is corrected to be correct marker ID;
with waist node as root node, human skeleton chain divide into 7 chains:
1. the right-hand node extends along the arm to the waist;
2. the left-hand node extends along the arm to the waist;
3. an upper triangle formed from two shoulders to the waist;
4. a lower triangle formed from waist to two crotch;
5. taking the right crotch from the right foot to the right crotch as a root node;
6. from the left foot to the left crotch, taking the left crotch as a root node;
7. the head and body make up a chain.
And fifthly, calculating the position and the posture of each joint according to the Marker.
And calculating the position of each joint according to the original posture model stored when the Human is newly built, and calculating the posture of the joint point by three or more markers on the same joint.
And sixthly, outputting the VRPN.
And outputting the position and the posture of the Human joint through VRPN.
The invention adopts the technical scheme that a human body motion capture method based on an IK algorithm corrects a marker predicted position and corrects a marker ID in real time by utilizing the IK algorithm and a kalman filtering algorithm. First, n (n >: 8) cameras are needed, which are installed in a ring shape, and the common field of view of the cameras covers the whole tracking area. The person to be tracked needs to first wear a kinetic fishing garment with reflective balls. After the system is built, firstly calibrating the pose relationship between the camera calibration camera and the camera to obtain a projection matrix between every two cameras. Before three-dimensional reconstruction, firstly, a kalman filtering algorithm is adopted to predict a current frame marker predicted position P, the P is back projected to a camera to find a corresponding image point pair, and a real position is calculated according to an image point pair trigonometry. After the matching points of the residual image points are found out according to epipolar line matching, a new marker is calculated through three-dimensional reconstruction, and a new marker ID is automatically generated. According to a Human body marker structure selected by a new Human frame, according to a predicted position P of a marker, by using a mainstream IK algorithm, in combination with length constraint of a Human skeleton and angle constraint between joints, and according to a tracked marker with a nearest lost marker distance, correcting the predicted position P to be a correct marker ID when the tracked marker conflicts with a newly generated marker. The position and pose of each joint is calculated from the Human associated Marker. And outputting the result through VRPN. Finally, the method provides finished and accurate human body actions for application scenes such as virtual reality and the like.
Example two
Fig. 3 is a schematic structural diagram of a human motion capture device based on an IK algorithm according to a second embodiment of the present invention. As shown in fig. 3, the human motion capture device based on IK algorithm includes:
the target mark point determining module 310 is configured to obtain a target mark point of a user in an initial posture, determine a serial number and a joint type of the target mark point, and store an initial posture model;
the position prediction module 320 is used for tracking the image information captured in real time and predicting the position of the target mark point in the current frame image according to the position of the target mark point in the previous frame image;
a lost mark point correcting module 330, configured to, if the target mark point is lost in the current frame image, correct the predicted position of the lost target mark point in the current frame image by using an IK algorithm according to a tracked target mark point closest to the lost target mark point;
and the current posture determining module 340 is configured to determine the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point.
According to the technical scheme provided by the embodiment of the application, the target mark points of a user in an initial posture are obtained, the serial number joint types of the target mark points are determined, and an initial posture model is stored; tracking the image information captured in real time, and predicting the position of a target mark point in the current frame image according to the position of the target mark point in the previous frame image; if the target mark point of the current frame image is lost, correcting the predicted position of the lost target mark point in the current frame image by adopting an IK algorithm according to the tracked target mark point closest to the lost target mark point; and determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point. By adopting the technical scheme provided by the application, all parts of the human body can be tracked and predicted based on the IK algorithm according to the tracking result of the main parts of the body of the user, so that the aim of displaying the whole body of the user in real time can be fulfilled in the interaction process of virtual reality.
The product can execute the method provided by any embodiment of the invention, and has corresponding functional modules and beneficial effects of the execution method.
EXAMPLE III
Embodiments of the present application also provide a storage medium containing computer-executable instructions, which when executed by a computer processor, are configured to perform a human motion capture method based on an IK algorithm, the method comprising:
acquiring target mark points of a user in an initial posture, determining the serial number and the joint type of the target mark points, and storing an initial posture model;
tracking the image information captured in real time, and predicting the position of a target mark point in the current frame image according to the position of the target mark point in the previous frame image;
if the target mark point of the current frame image is lost, correcting the predicted position of the lost target mark point in the current frame image by adopting an IK algorithm according to the tracked target mark point closest to the lost target mark point;
and determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point.
Storage medium-any of various types of memory devices or storage devices. The term "storage medium" is intended to include: mounting media such as CD-ROM, floppy disk, or tape devices; computer system memory or random access memory such as DRAM, DDR RAM, SRAM, EDO RAM, Lanbas (Rambus) RAM, etc.; non-volatile memory such as flash memory, magnetic media (e.g., hard disk or optical storage); registers or other similar types of memory elements, etc. The storage medium may also include other types of memory or combinations thereof. In addition, the storage medium may be located in the computer system in which the program is executed, or may be located in a different second computer system connected to the computer system through a network (such as the internet). The second computer system may provide the program instructions to the computer for execution. The term "storage medium" may include two or more storage media that may reside in different locations, such as in different computer systems that are connected by a network. The storage medium may store program instructions (e.g., embodied as a computer program) that are executable by one or more processors.
Of course, the storage medium provided by the embodiments of the present application contains computer executable instructions, and the computer executable instructions are not limited to the online human motion capture operation based on the IK algorithm as described above, and may also perform related operations in the human motion capture method based on the IK algorithm provided by any embodiments of the present application.
Example four
The embodiment of the application provides electronic equipment, and the typesetting device of the image provided by the embodiment of the application can be integrated into the electronic equipment. Fig. 4 is a schematic structural diagram of an electronic device according to a fourth embodiment of the present application. As shown in fig. 4, the present embodiment provides an electronic device 400, which includes: one or more processors 420; the storage device 410 is used for storing one or more programs, when the one or more programs are executed by the one or more processors 420, so that the one or more processors 420 implement the method for composing images, which is provided by the embodiment of the application, and the method comprises the following steps:
acquiring target mark points of a user in an initial posture, determining the serial number and the joint type of the target mark points, and storing an initial posture model;
tracking the image information captured in real time, and predicting the position of a target mark point in the current frame image according to the position of the target mark point in the previous frame image;
if the target mark point of the current frame image is lost, correcting the predicted position of the lost target mark point in the current frame image by adopting an IK algorithm according to the tracked target mark point closest to the lost target mark point;
and determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point.
And if the output grading result of the grading network model meets the preset standard, determining the current image state as the typesetting result of the image.
Of course, those skilled in the art will understand that the processor 420 may also implement the technical solution of the image layout method provided in any embodiment of the present application.
The electronic device 400 shown in fig. 4 is only an example, and should not bring any limitation to the functions and the scope of use of the embodiments of the present application.
As shown in fig. 4, the electronic device 400 includes a processor 420, a storage device 410, an input device 430, and an output device 440; the number of the processors 420 in the electronic device may be one or more, and one processor 420 is taken as an example in fig. 4; the processor 420, the storage device 410, the input device 430, and the output device 440 in the electronic apparatus may be connected by a bus or other means, and are exemplified by a bus 450 in fig. 4.
The storage device 410 is a computer-readable storage medium for storing a software program, a computer-executable program, and module units, such as program instructions corresponding to the image layout method in the embodiment of the present application.
The storage device 410 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal, and the like. Further, the storage 410 may include high speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid state storage device. In some examples, storage 410 may further include memory located remotely from processor 420, which may be connected via a network. Examples of such networks include, but are not limited to, the internet, intranets, local area networks, mobile communication networks, and combinations thereof.
The input means 430 may be used to receive input numbers, character information, or voice information, and to generate key signal inputs related to user settings and function control of the electronic device. The output device 440 may include a display screen, speakers, etc.
The electronic equipment provided by the embodiment of the application can track and predict all parts of a human body based on the IK algorithm according to the tracking result of the main parts of the body of the user, so that the aim of displaying the whole body of the user in real time can be fulfilled in the interaction process of virtual reality.
The image layout device, the medium and the electronic device provided in the above embodiments may operate the image layout method provided in any embodiment of the present application, and have corresponding functional modules and beneficial effects for operating the method. For technical details which are not described in detail in the above embodiments, reference may be made to a layout method of images provided in any of the embodiments of the present application.
It is to be noted that the foregoing is only illustrative of the preferred embodiments of the present invention and the technical principles employed. It will be understood by those skilled in the art that the present invention is not limited to the particular embodiments described herein, but is capable of various obvious changes, rearrangements and substitutions as will now become apparent to those skilled in the art without departing from the scope of the invention. Therefore, although the present invention has been described in greater detail by the above embodiments, the present invention is not limited to the above embodiments, and may include other equivalent embodiments without departing from the spirit of the present invention, and the scope of the present invention is determined by the scope of the appended claims.
Claims (10)
1. A human motion capture method based on an IK algorithm is characterized by comprising the following steps:
acquiring target mark points of a user in an initial posture, determining the serial number and the joint type of the target mark points, and storing an initial posture model;
tracking the image information captured in real time, and predicting the position of a target mark point in the current frame image according to the position of the target mark point in the previous frame image;
if the target mark point of the current frame image is lost, correcting the predicted position of the lost target mark point in the current frame image by adopting an IK algorithm according to the tracked target mark point closest to the lost target mark point;
and determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point.
2. The method of claim 1, wherein the correcting the position of the missing target marker in the predicted current frame image according to the nearest tracked target marker of the missing target marker by using an IK algorithm comprises:
determining a bone length constraint and an angle constraint between joints;
and correcting the position of the lost target mark point in the predicted current frame image by adopting an IK algorithm and combining the skeleton length constraint and the angle constraint between joints according to the tracked target mark point closest to the lost target mark point.
3. The method of claim 1, wherein after determining the current pose of the user based on the initial pose model and the correction of the position of the missing target marker point, the method further comprises:
and outputting the current posture of the user through the VRPN.
4. The method of claim 1, wherein prior to acquiring the target marker points of the user in the initial pose, the method further comprises:
acquiring a calibration image of the T-shaped rod calibration tool through the tracking camera;
and calibrating the tracking camera according to the calibration image of the T-shaped rod calibration tool.
5. The method of claim 4, wherein after calibrating the tracking camera, the method further comprises:
the room coordinate system is established by means of the calibration image of the L-bar calibration tool.
6. The method of claim 1, wherein obtaining target marker points of a user in an initial pose comprises:
acquiring a reflective mark point worn by a user; the marking points are distributed on each joint point of the human body and are used for acquiring the posture of the joint; wherein, each joint point has at least 3 marking points. And determining the human body initial posture bone model when the user is in the initial posture.
7. The method of claim 1, wherein predicting the position of the target mark point in the current frame image according to the position of the target mark point in the previous frame image comprises:
predicting the position of a target mark point in the current frame image by adopting a Kalman filtering algorithm according to the position of the target mark point in the previous frame image;
and traversing all the target mark points.
8. An IK algorithm-based human motion capture device, comprising:
the target mark point determining module is used for acquiring a target mark point of a user in an initial posture, determining the serial number and the joint type of the target mark point and storing an initial posture model;
the position prediction module is used for tracking the image information captured in real time and predicting the position of the target mark point in the current frame image according to the position of the target mark point in the previous frame image;
the lost mark point correcting module is used for correcting the predicted position of the lost target mark point in the current frame image by adopting an IK algorithm according to the tracked target mark point closest to the lost target mark point if the target mark point is lost in the current frame image;
and the current posture determining module is used for determining the current posture of the user according to the initial posture model and the correction result of the position of the lost target mark point.
9. A computer-readable storage medium on which a computer program is stored, which when executed by a processor implements the IK algorithm based human motion capture method according to any one of claims 1-7.
10. A mobile device comprising a memory, a processor and a computer program stored on the memory and executable on the processor, wherein the computer program when executed by the processor implements the IK algorithm based human motion capture method according to any of claims 1-7.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010312083.3A CN111539300A (en) | 2020-04-20 | 2020-04-20 | Human motion capture method, device, medium and equipment based on IK algorithm |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202010312083.3A CN111539300A (en) | 2020-04-20 | 2020-04-20 | Human motion capture method, device, medium and equipment based on IK algorithm |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN111539300A true CN111539300A (en) | 2020-08-14 |
Family
ID=71979121
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202010312083.3A Withdrawn CN111539300A (en) | 2020-04-20 | 2020-04-20 | Human motion capture method, device, medium and equipment based on IK algorithm |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN111539300A (en) |
Cited By (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112381925A (en) * | 2020-11-13 | 2021-02-19 | 广州引力波科技创新发展有限公司 | Whole body tracking and positioning method and system based on laser coding |
| CN112416133A (en) * | 2020-11-30 | 2021-02-26 | 魔珐(上海)信息科技有限公司 | Hand motion capture method and device, electronic equipment and storage medium |
| CN112465857A (en) * | 2020-12-11 | 2021-03-09 | 深圳市瑞立视多媒体科技有限公司 | Method for tracking rigid body position information, device, equipment and storage medium thereof |
| CN112508992A (en) * | 2020-12-11 | 2021-03-16 | 深圳市瑞立视多媒体科技有限公司 | Method for tracking rigid body position information based on filtering of 1 { (integral } blocks and device and equipment thereof |
| CN113925497A (en) * | 2021-10-22 | 2022-01-14 | 吉林大学 | Automobile passenger riding posture extraction method based on binocular vision measurement system |
| CN114939216A (en) * | 2022-05-30 | 2022-08-26 | 深圳英鸿骏智能科技有限公司 | Equipment and method for assisting rehabilitation exercise |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104424650A (en) * | 2013-08-21 | 2015-03-18 | 中国人民解放军第二炮兵工程大学 | Arm information compensation method in optical profile type human body motion capture |
| CN109671141A (en) * | 2018-11-21 | 2019-04-23 | 深圳市腾讯信息技术有限公司 | The rendering method and device of image, storage medium, electronic device |
| CN110531866A (en) * | 2019-10-29 | 2019-12-03 | 深圳市瑞立视多媒体科技有限公司 | The method and relevant device of attitude algorithm are carried out based on improved inverse kinematics |
-
2020
- 2020-04-20 CN CN202010312083.3A patent/CN111539300A/en not_active Withdrawn
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104424650A (en) * | 2013-08-21 | 2015-03-18 | 中国人民解放军第二炮兵工程大学 | Arm information compensation method in optical profile type human body motion capture |
| CN109671141A (en) * | 2018-11-21 | 2019-04-23 | 深圳市腾讯信息技术有限公司 | The rendering method and device of image, storage medium, electronic device |
| CN110531866A (en) * | 2019-10-29 | 2019-12-03 | 深圳市瑞立视多媒体科技有限公司 | The method and relevant device of attitude algorithm are carried out based on improved inverse kinematics |
Non-Patent Citations (2)
| Title |
|---|
| ANDREAS ARISTIDOU • JOAN LASENBY: "Real-time marker prediction and CoR estimation in optical motion capture", 《VIS COMPUT》 * |
| JOANNA SVANTESSON: "A RealTime Adaptation of Inverse Kinematics for Motion Capture", 《EXAMENSARBETE FÖR MASTEREXAMEN》 * |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112381925A (en) * | 2020-11-13 | 2021-02-19 | 广州引力波科技创新发展有限公司 | Whole body tracking and positioning method and system based on laser coding |
| CN112416133A (en) * | 2020-11-30 | 2021-02-26 | 魔珐(上海)信息科技有限公司 | Hand motion capture method and device, electronic equipment and storage medium |
| CN112465857A (en) * | 2020-12-11 | 2021-03-09 | 深圳市瑞立视多媒体科技有限公司 | Method for tracking rigid body position information, device, equipment and storage medium thereof |
| CN112508992A (en) * | 2020-12-11 | 2021-03-16 | 深圳市瑞立视多媒体科技有限公司 | Method for tracking rigid body position information based on filtering of 1 { (integral } blocks and device and equipment thereof |
| CN112508992B (en) * | 2020-12-11 | 2022-04-19 | 深圳市瑞立视多媒体科技有限公司 | Method, device and equipment for tracking rigid body position information |
| CN113925497A (en) * | 2021-10-22 | 2022-01-14 | 吉林大学 | Automobile passenger riding posture extraction method based on binocular vision measurement system |
| CN113925497B (en) * | 2021-10-22 | 2023-09-15 | 吉林大学 | Binocular vision measurement system-based automobile passenger riding posture extraction method |
| CN114939216A (en) * | 2022-05-30 | 2022-08-26 | 深圳英鸿骏智能科技有限公司 | Equipment and method for assisting rehabilitation exercise |
| CN114939216B (en) * | 2022-05-30 | 2023-11-10 | 深圳英鸿骏智能科技有限公司 | Device and method for assisting rehabilitation exercise |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN111539300A (en) | Human motion capture method, device, medium and equipment based on IK algorithm | |
| US11468612B2 (en) | Controlling display of a model based on captured images and determined information | |
| JP6171079B1 (en) | Inconsistency detection system, mixed reality system, program, and inconsistency detection method | |
| WO2020054442A1 (en) | Articulation position acquisition method and device, and motion acquisition method and device | |
| JP4148281B2 (en) | Motion capture device, motion capture method, and motion capture program | |
| JP3512992B2 (en) | Image processing apparatus and image processing method | |
| JPWO2016151691A1 (en) | Image processing apparatus, image processing system, image processing method, and program | |
| US8526734B2 (en) | Three-dimensional background removal for vision system | |
| CN109961523B (en) | Method, device, system, equipment and storage medium for updating virtual target | |
| CN111260793B (en) | Remote virtual-real high-precision matching positioning method for augmented and mixed reality | |
| Yang et al. | A virtual try-on system in augmented reality using RGB-D cameras for footwear personalization | |
| JP2005256232A (en) | Method, apparatus and program for displaying 3d data | |
| JP2015186531A (en) | Action information processing device and program | |
| CN110544302A (en) | Human body action reconstruction system and method based on multi-view vision and action training system | |
| JP2007066094A (en) | Posture estimation device and posture estimation method | |
| JP4695275B2 (en) | Video generation system | |
| CN109781104B (en) | Motion attitude determination and positioning method and device, computer equipment and medium | |
| JP2018106661A (en) | Inconsistency detection system, mixed reality system, program, and inconsistency detection method | |
| CN111539299B (en) | Human motion capturing method, device, medium and equipment based on rigid body | |
| US11461914B2 (en) | Measuring surface distances on human bodies | |
| US20210248802A1 (en) | Method for locating a center of rotation of an articulated joint | |
| Schönauer et al. | Wide area motion tracking using consumer hardware | |
| Phan et al. | Towards 3D human posture estimation using multiple kinects despite self-contacts | |
| JP2022516466A (en) | Information processing equipment, information processing methods, and programs | |
| WO2018173205A1 (en) | Information processing system, method for controlling same, and program |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| WW01 | Invention patent application withdrawn after publication |
Application publication date: 20200814 |
|
| WW01 | Invention patent application withdrawn after publication |