WO2024077837A1 - 虚拟人的驱动方法、装置、存储介质和电子设备 - Google Patents

虚拟人的驱动方法、装置、存储介质和电子设备 Download PDF

Info

Publication number
WO2024077837A1
WO2024077837A1 PCT/CN2023/077997 CN2023077997W WO2024077837A1 WO 2024077837 A1 WO2024077837 A1 WO 2024077837A1 CN 2023077997 W CN2023077997 W CN 2023077997W WO 2024077837 A1 WO2024077837 A1 WO 2024077837A1
Authority
WO
WIPO (PCT)
Prior art keywords
human
rotation matrix
joint
virtual
matrix
Prior art date
Application number
PCT/CN2023/077997
Other languages
English (en)
French (fr)
Inventor
汤志斌
贺天童
区志财
唐剑
Original Assignee
美的集团(上海)有限公司
美的集团股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 美的集团(上海)有限公司, 美的集团股份有限公司 filed Critical 美的集团(上海)有限公司
Publication of WO2024077837A1 publication Critical patent/WO2024077837A1/zh

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T13/00—Animation
    • G06T13/20—3D [Three Dimensional] animation
    • G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00—Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10—Complex mathematical operations
    • G06F17/16—Matrix or vector computation, e.g. matrix-matrix or matrix-vector multiplication, matrix factorization
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/40—Scenes; Scene-specific elements in video content
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00—Indexing scheme for image data processing or generation, in general
    • G06T2200/08—Indexing scheme for image data processing or generation, in general involving all processing steps from image acquisition to 3D model generation

Definitions

  • the present application belongs to the technical field of virtual humans, and in particular, relates to a method, device, storage medium and electronic device for driving a virtual human.
  • the present application aims to solve at least one of the technical problems existing in the related art.
  • a first aspect of the present application is to propose a method for driving a virtual human.
  • a second aspect of the present application is to provide a driving device for a virtual human.
  • the third aspect of the present application is to propose another driving device for a virtual human.
  • a fourth aspect of the present application is to provide a readable storage medium.
  • a fifth aspect of the present application is to provide an electronic device.
  • a sixth aspect of the present application is to provide a computer program product.
  • a driving method for a virtual human which includes: determining a first rotation matrix of human joints based on a captured first human image; adding a constraint matrix to the first rotation matrix to determine a second rotation matrix of the human joints, wherein the constraint matrix corresponds to the physical constraint conditions of the human joints; smoothing the second rotation matrix to determine a target rotation matrix of the human joints; and driving the virtual human action according to the target rotation matrix.
  • the execution subject of the driving method of the virtual person proposed in the present application may be a driving device of the virtual person.
  • the driving method of the virtual person is exemplified by taking the driving device of the virtual person as the executing subject.
  • the first human image represents a frame of image captured in a video
  • the constraint matrix represents physical constraints on human joints
  • the physical constraints represent conditions for limiting the movement of human joints to prevent their movements from exceeding certain limits
  • the rotation matrix represents a mathematical expression of human joint posture
  • the first rotation matrix represents the rotation matrix of human joints detected by a human posture detection network
  • the target rotation matrix represents the second rotation matrix after smoothing.
  • the driving device first determines the rotation matrix of each human joint of the virtual person according to the first human image collected from the video, that is, the first rotation matrix. Specifically, the driving device can determine the first rotation matrix by inputting the first human image into a human posture detection network, but is not limited thereto.
  • the detection device adds a constraint matrix to the first rotation matrix of the human joint to determine the rotation matrix after physical correction, that is, the above-mentioned second rotation matrix.
  • the first rotation matrix can be used to clarify the movement of each human joint between the three axes of the spatial coordinates XYZ.
  • the drive device can correct and limit the first rotation matrix by adding a constraint matrix to the first rotation matrix of the human joint, and obtain a corrected second rotation matrix.
  • the driving device smoothly processes the second rotation matrix obtained after adding the constraint matrix to determine a target rotation matrix for driving the above-mentioned virtual human limb movements.
  • the driving device needs to smooth the second rotation matrix to obtain the target rotation matrix after smoothing.
  • the driving device drives the virtual human according to the target rotation matrix so that the limbs of the virtual human present movements that are adapted to the first human body image.
  • the driving device can determine the first rotation matrix of each human joint of the virtual person based on the first human body image collected from the video, can add a constraint matrix to the first rotation matrix to determine the second rotation matrix, can smooth the parameters in the second rotation matrix to determine the target rotation matrix, and can drive the virtual person's limb movements according to the target rotation matrix.
  • the driving device adds a constraint matrix to the target matrix used to drive the virtual person's limb movements and performs smoothing. In this way, the movements made by the virtual person will not violate physical common sense, and will not shake during the movements, which solves the problem of directly operating the virtual person in the related art. Using the detection results of the human posture detection network to drive the virtual human to perform actions may violate physical common sense and have jitter problems, which ensures the driving effect of the virtual human.
  • the driving method of the virtual human proposed in the above technical solution of the present application may also have the following additional technical features:
  • the step of adding a constraint matrix to the first rotation matrix to determine the second rotation matrix of the human joint specifically includes: using the first rotation matrix of the sub-joint of the human joint as the constraint matrix; compensating the constraint matrix to the first rotation matrix of the human joint to determine the second rotation matrix.
  • the driving device can physically constrain the virtual human's limb movements by using the first rotation matrix corresponding to the sub-joint as a constraint matrix and compensating the constraint matrix to the first rotation matrix of the human joint, and determine the second rotation matrix of each human joint of the virtual human.
  • the rotation of the human joint on the XYZ axis and the positional relationship of the human joint relative to the human body are restricted, ensuring that there will be no violation of physical common sense when driving the virtual human's limb movements.
  • the sub-joints of different human joints are different.
  • the sub-joint of the thigh is the calf
  • the sub-joint of the elbow is the arm.
  • the first rotation matrices of different sub-joints are also different, so the constraint matrices added to different human joints are also different.
  • the second rotation matrix is smoothed, and the steps of determining the target rotation matrix of the human joint specifically include: determining the first acceleration of the human joint action and the first external force applied to the human joint according to the second rotation matrix; determining the first position of the human joint according to the first acceleration and the first external force; and determining the target rotation matrix according to the first position.
  • the driving device can solve the first acceleration and the first external force of the human joint according to the second rotation matrix, determine the first position of the human joint after smoothing by solving the optimal relationship between the first acceleration and the first external force, and infer the target rotation matrix according to the first position.
  • the target rotation matrix used to drive the virtual human limb movement is smoothed, ensuring that there is no shaking in the process of driving the virtual human limb movement.
  • the step of determining the first acceleration of the human joint movement and the first external force exerted on the human joint according to the second rotation matrix specifically includes: determining the second position of the human joint according to the second rotation matrix; acquiring the second human image and the third human image, as well as the acquisition time information of the second human image and the third human image; determining the first historical position of the human joint in the second human image and the second historical position of the human joint in the third human image; determining the first historical position, the second historical position, the second position and the acquisition time information of the human joint according to the first historical position, the second historical position, the second position and the acquisition time information.
  • the driving device solves the first external force exerted on each joint of the virtual person and the first acceleration of each joint of the human body according to the second rotation matrix obtained after adding the constraint matrix. This ensures the accuracy of the determined first external force and the second acceleration, and further ensures the accuracy of the target rotation matrix determined in the subsequent steps.
  • the steps of determining the first speed and the first acceleration of the human joint movement according to the first historical position, the second historical position and the second position specifically include: determining the first speed according to the second historical position, the second position and the collection time information; determining the second speed according to the first historical position, the second historical position and the collection time information; determining the first acceleration according to the difference between the first speed and the second speed.
  • the driving device calculates the speed and acceleration of each human joint in the first human image based on the position of the human joint in the human image collected before the first human image, that is, the driving device calculates the speed and acceleration in the most recent time. In this way, the accuracy of determining the first acceleration and the first speed is guaranteed.
  • the step of determining the first external force according to the second position, the first velocity and the first acceleration specifically includes: determining the first external force according to the second position, the first velocity and the first acceleration based on the friction cone constraint condition through the rigid body motion equation.
  • the driving device can solve the first external force that satisfies the friction cone constraint condition according to the first acceleration, the first speed and the second position through the rigid body motion equation, thus ensuring the accuracy of determining the first external force.
  • the step of determining the first position of the human joint according to the first acceleration and the first external force specifically includes: solving the first position according to the first acceleration and the first external force through an iterative algorithm.
  • the above-mentioned first position is the optimal position solved by an iterative algorithm. In this way, the accuracy of the determined first position is guaranteed, and further ensures that when the target rotation matrix inferred from the first position drives the virtual human's limb movements, there will be no jitter or violation of physical common sense.
  • a driving device for a virtual human comprises: a first processing module, used to determine a first rotation matrix of a human joint according to a captured first human image; a second processing module, used to add a constraint matrix to the first rotation matrix, and determine a second rotation matrix of the human joint, wherein the constraint matrix corresponds to a physical constraint condition of the human joint; a third processing module, used to smooth the second rotation matrix, and determine a target rotation matrix of the human joint; and the third processing module is further used to drive the virtual human according to the target rotation matrix. Move the virtual human.
  • the first human image represents a frame of image captured in a video
  • the constraint matrix represents physical constraints on human joints
  • the physical constraints represent conditions for limiting the movement of human joints to prevent their movements from exceeding certain limits
  • the rotation matrix represents a mathematical expression of human joint posture
  • the first rotation matrix represents the rotation matrix of human joints detected by a human posture detection network
  • the target rotation matrix represents the second rotation matrix after smoothing.
  • the first processing module first determines the rotation matrix of each human joint of the virtual person according to the first human image collected from the video, that is, the first rotation matrix. Specifically, the first processing module can determine the first rotation matrix by inputting the first human image into a human posture detection network, but is not limited thereto.
  • the second processing module adds a constraint matrix to the first rotation matrix of the human body joint to determine a physically corrected rotation matrix, ie, the second rotation matrix.
  • the first rotation matrix can be used to clarify the movement of each human joint between the three axes of the spatial coordinates XYZ, and by adding a constraint matrix, it can be limited on which axis of the XYZ three axes the above human joint moves, as well as the limit of the movement on each axis. Therefore, the second processing module can correct and limit the first rotation matrix by adding a constraint matrix to the first rotation matrix of the human joint, and obtain a corrected second rotation matrix.
  • the third processing module performs smoothing processing on the second rotation matrix obtained after adding the constraint matrix to determine a target rotation matrix for driving the above-mentioned virtual human limb movements.
  • the third processing module needs to smooth the second rotation matrix to obtain the target rotation matrix after smoothing.
  • the third processing module drives the virtual human according to the target rotation matrix, so that the limbs of the virtual human present movements that are adapted to the first human body image.
  • the first processing module can determine the first rotation matrix of each human joint of the virtual person according to the first human image collected from the video
  • the second processing module can add a constraint matrix to the first rotation matrix to determine the second rotation matrix
  • the third processing module can smooth the parameters in the second rotation matrix.
  • the target rotation matrix is determined and the limb movements of the virtual human can be driven according to the target rotation matrix.
  • the third processing module adds a constraint matrix to the target rotation matrix used to drive the limb movements of the virtual human and performs smoothing processing. In this way, the movements made by the virtual human will not violate physical common sense and will not shake during the movement. This solves the problem in the related technology that directly using the detection results of the human posture detection network to drive the virtual human to make movements may violate physical common sense and cause shaking, thereby ensuring the driving effect of the virtual human.
  • another driving device for a virtual human comprising: a memory, in which a program or instruction is stored; a processor, which executes the program or instruction stored in the memory to implement the steps of the driving method for a virtual human proposed in the above-mentioned technical scheme of the present application, and thus has all the beneficial technical effects of the driving method for a virtual human proposed in the above-mentioned technical scheme of the present application, and will not be elaborated on here.
  • a readable storage medium on which a program or instruction is stored.
  • the steps of the virtual human driving method proposed in the above technical scheme of the present application are implemented, and thus the method has all the beneficial technical effects of the virtual human driving method proposed in the above technical scheme of the present application, which will not be elaborated herein.
  • an electronic device comprising a driving device for a virtual person as proposed in the above technical solution of the present application, and/or a readable storage medium as proposed in the above technical solution of the present application. Therefore, the electronic device has all the beneficial effects of the driving device for a virtual person proposed in the above technical solution of the present application and/or the readable storage medium proposed in the above technical solution of the present application, which will not be repeated here.
  • a computer program product including a computer program.
  • the computer program When the computer program is executed by a processor, it implements the steps of the virtual human driving method proposed in the above technical scheme of the present application, and thus has all the beneficial technical effects of the virtual human driving method proposed in the above technical scheme of the present application, which will not be elaborated herein.
  • FIG1 shows a schematic diagram of a process flow of a method for driving a virtual human according to an embodiment of the present application
  • FIG2 shows a second flow chart of the method for driving a virtual human according to an embodiment of the present application
  • FIG3 shows a third flow chart of the method for driving a virtual human according to an embodiment of the present application
  • FIG4 shows one of the schematic block diagrams of the driving device of the virtual human according to an embodiment of the present application
  • FIG5 shows a second schematic block diagram of the driving device of a virtual human according to an embodiment of the present application
  • FIG. 6 shows an overall flow chart of a method for driving a virtual human according to an embodiment of the present application.
  • Embodiment 1 is a diagrammatic representation of Embodiment 1:
  • FIG1 is a schematic diagram showing a flow chart of a method for driving a virtual person according to an embodiment of the present application.
  • the method for driving a virtual person includes:
  • the first human image represents a frame image captured in a video
  • the constraint matrix represents physical constraints on human joints
  • the physical constraints represent conditions for limiting the movement of human joints to prevent their movements from exceeding certain limits
  • the rotation matrix represents a mathematical expression of human joint posture
  • the first rotation matrix represents the rotation matrix of human joints detected by a human posture detection network
  • the target rotation matrix represents the second rotation matrix after smoothing.
  • the driving device first determines the rotation matrix of each human joint of the virtual person according to the first human image collected from the video, that is, the first rotation matrix.
  • the driving device can determine the first rotation matrix by inputting the first human image into a human posture detection network, but is not limited thereto. Other human posture detection methods are also applicable to the present application.
  • the detection device adds a constraint matrix to the first rotation matrix of the human joint to determine the rotation matrix after physical correction, that is, the above-mentioned second rotation matrix.
  • the first rotation matrix can be used to clarify the movement of each human joint between the three axes of the spatial coordinates XYZ.
  • the drive device can correct and limit the first rotation matrix by adding a constraint matrix to the first rotation matrix of the human joint to obtain a corrected second rotation matrix.
  • the human joint As the calf, its first rotation matrix is decomposed in the XYZ three axes, and it can be found that the calf only rotates about the Y axis, but if the rotation of the calf on the XZ axis is directly deleted, it may cause the calf position to shift, so the rotation of the calf on the XZ axis can be physically constrained, and the relationship between the calf and the human body position can be restricted, that is, the constraint matrix is added to the first rotation matrix of the calf. In this way, the second rotation matrix of the calf can be obtained.
  • the driving device smoothly processes the second rotation matrix obtained after adding the constraint matrix to determine a target rotation matrix for driving the above-mentioned virtual human limb movements.
  • the driving device needs to smooth the second rotation matrix to obtain the target rotation matrix after smoothing.
  • the driving device drives the virtual human according to the target rotation matrix so that the limbs of the virtual human present movements that are adapted to the first human body image.
  • the driving device adds a constraint matrix to the target matrix used to drive the virtual human's limb movements and performs smoothing, so that the actions performed by the virtual human will not violate physical common sense and will not shake during the action, thereby ensuring the driving effect of the virtual human.
  • FIG2 is a schematic diagram showing a flow chart of a method for driving a virtual person according to an embodiment of the present application.
  • the method for driving a virtual person includes:
  • the process of determining the second rotation matrix is as follows: the driving device first determines the sub-joints of the human joints, and then determines the constraint matrix of the first rotation matrix for the human joints according to the sub-joints of the human joints.
  • the driving device uses the first rotation matrix corresponding to the sub-joints as a constraint matrix, and compensates the constraint matrix to the first rotation matrix of the human joints to determine the second rotation matrix of each human joint.
  • the resulting second rotation matrix limits the rotation of the human joints on the XYZ axis and the positional relationship of the human joints relative to the human body.
  • the human joint is the thigh
  • its sub-joint is the calf
  • the first rotation matrix of the thigh is C 0
  • the first rotation matrix of the calf is C 1.
  • the calf can only rotate about the Y axis
  • we can decompose the first rotation matrix of the calf along the XYZ axis in sequence, and obtain C 1 C Y1 C X1 C Z1 , where C Y1 is the matrix of C 1 in the Y axis direction, C X1 is the matrix of C 1 in the X axis direction, and C Z1 is the matrix of C 1 in the Z axis direction.
  • the driving device can physically constrain the virtual human's limb movements by using the first rotation matrix corresponding to the sub-joint as a constraint matrix and compensating the constraint matrix to the first rotation matrix of the human joint, thereby determining the second rotation matrix of each human joint of the virtual human.
  • the rotation of the human joint on the XYZ axis and the positional relationship of the human joint relative to the human body are restricted, ensuring that no violation of physical common sense occurs when driving the virtual human's limb movements.
  • the first rotation matrices of different sub-joints are different.
  • the sub-joints of different human joints are different, for example, the sub-joint of the thigh is the calf, and the sub-joint of the elbow is the arm. It can be understood that the first rotation matrices of different sub-joints are also different, so the constraint matrices added to different human joints are also different.
  • FIG3 is a schematic diagram showing a flow chart of a method for driving a virtual person according to an embodiment of the present application.
  • the method for driving a virtual person includes:
  • the first external force represents the external force on the human joint, such as the reaction force of the ground on the calf
  • the first position represents the position of the human joint determined after physical constraint and smoothing.
  • the driving device first solves the current acceleration of the human body joint, that is, the first acceleration, and the external force currently applied to the human body joint, that is, the first external force, according to the second rotation matrix.
  • the information of the current position of the human joint can be solved according to the parameters in the second rotation matrix.
  • the speed and acceleration of the human joint from the basic position to the current position can be determined.
  • the driving device can determine the first acceleration and the first external force according to the second rotation matrix.
  • the driving device calculates the position of the human joint after smoothing, that is, the first position, through the first acceleration and the first external force. Specifically, the driving device can calculate the optimal relationship between the acceleration of the human joint and the external force it receives based on the first acceleration and the first external force, and can calculate the first position of the human joint after smoothing based on the optimal relationship.
  • the driving device infers a rotation matrix for controlling the limb movements of the virtual human according to the determined first position, that is, the above-mentioned target rotation matrix.
  • the driving device can solve the first acceleration and the first external force of the human joint according to the second rotation matrix, determine the first position of the human joint after smoothing by solving the optimal relationship between the first acceleration and the first external force, and infer the target rotation matrix according to the first position.
  • the target rotation matrix used to drive the virtual human limb movement is smoothed, ensuring that there is no shaking in the process of driving the virtual human limb movement.
  • the step of determining the first acceleration of the human joint movement and the first external force exerted on the human joint according to the second rotation matrix specifically includes: determining the second position of the human joint according to the second rotation matrix; acquiring the second human image and the third human image, as well as the acquisition time information of the second human image and the third human image; determining the first historical position of the human joint in the second human image and the second historical position of the human joint in the third human image; determining the first speed and the first acceleration of the human joint movement according to the first historical position, the second historical position, the second position and the acquisition time information; determining the first External force; wherein, the second human body image and the third human body image are images captured before the first human body image.
  • the second human body image and the third human body image represent images captured before the first human body image, and the frame timing of the capture of the second human body image is earlier than that of the third human body image;
  • the first historical position represents the position of the human body joints in the second human body image
  • the second historical position represents the position of the human body joints in the third human body image
  • the second position represents the position of the human body joints in the first human body image after physical constraints.
  • human joints include each human joint of the virtual person.
  • human joint A among the human joints is used as an example below.
  • the process of determining the first external force and the first acceleration is as follows: the driving device first solves the second position of the human joint A after physical constraints according to the second rotation matrix.
  • the driving device obtains the second human body image and the third human body image, as well as the acquisition time information of the second human body image and the third human body image, and determines the first historical position and the second historical position of the human joint A.
  • the driving device may first input the second human body image and the third human body image into the human posture detection network, determine the rotation matrix of the human joint A in the second human body image and the rotation matrix of the human joint A in the third human body image, and then solve the first historical position and the second historical position according to the two rotation matrices.
  • the driving device solves the first acceleration and the first speed of the human joint A based on the second position, the first historical position, the second historical position and the acquisition time information. Specifically, the driving device can determine the distance based on any two of the three positions, and the driving device can calculate the first acceleration and the first speed based on the distance and the acquisition time information.
  • the driving device solves the first external force on the human joint A according to the first acceleration, the first velocity and the first external force. Specifically, there is a certain relationship between the first external force on the human joint A and the first acceleration, the first velocity and the first external force, such as the relationship restricted by the rigid body motion equation. Based on this relationship, the driving device can solve the first external force on the human joint A through the first acceleration, the first velocity and the first external force.
  • the driving device solves the first external force exerted on each joint of the virtual person and the first acceleration of each joint of the virtual person based on the second rotation matrix obtained after adding the constraint matrix. This ensures the accuracy of the determined first external force and second acceleration, and further ensures the accuracy of the target rotation matrix determined in subsequent steps.
  • the step of determining the first speed and the first acceleration of the human joint action according to the first historical position, the second historical position, the second position and the acquisition time information specifically includes: determining the first speed and the first acceleration of the human joint action according to the second historical position, the second historical position, the second position and the acquisition time information
  • the first speed is determined according to the first historical position, the second historical position and the collection time information
  • the second speed is determined according to the first historical position, the second historical position and the collection time information
  • the first acceleration is determined according to the difference between the first speed and the second speed.
  • the process of determining the first speed and the first acceleration is as follows: the driving device first calculates the first speed according to the second position, the second historical position and the collection time information. Specifically, the distance between the two positions can be clearly determined according to the second position and the second historical position, and the driving device can calculate the first speed according to the distance and the collection time information.
  • the driving device calculates the second speed according to the first historical position, the second historical position and the collection time information. Specifically, the distance between the two positions can be clearly determined according to the first historical position and the second historical position, and the driving device can calculate the second speed according to the distance and the collection time information.
  • the driving device calculates a difference between the first speed and the second speed to determine the first acceleration.
  • the driving device calculates the speed and acceleration of each human joint in the first human image based on the position of the human joint in the human image collected before the first human image, that is, the driving device calculates the speed and acceleration in the most recent time. In this way, the accuracy of determining the first acceleration and the first speed is guaranteed.
  • the step of determining the first external force according to the second position, the first velocity and the first acceleration specifically includes: determining the first external force according to the second position, the first velocity and the first acceleration based on the friction cone constraint condition through the rigid body motion equation.
  • the process of determining the first external force is as follows: the driving device adopts the rigid body motion equation to solve the first external force according to the first acceleration, the first speed and the second position while satisfying the friction cone constraint condition.
  • the formula for the above friction cone constraint condition is: Where F represents the friction cone constraint, ⁇ n represents the normal component, ⁇ t and ⁇ b represent the tangential components, ⁇ represents the friction coefficient, and R represents a three-dimensional vector.
  • the driving device can solve the first external force that satisfies the friction cone constraint condition according to the first acceleration, the first speed and the second position through the rigid body motion equation, thus ensuring the accuracy of determining the first external force.
  • the step of determining the first position of the human joint according to the first acceleration and the first external force specifically includes: solving the first position according to the first acceleration and the first external force through an iterative algorithm.
  • the process of determining the first position is: using an iterative algorithm to solve the optimal combination relationship between the first external force and the first acceleration to determine the optimal first position.
  • this application believes that when the limb is in contact with the ground, the speed perpendicular to the ground must be greater than 0, and the tangential speed should be constrained. Therefore, in the above iterative algorithm, the constraints of 0 ⁇ vn ,
  • the above-mentioned first position is the optimal position solved by an iterative algorithm, thus ensuring the accuracy of the determined first position, and further ensuring that when the target rotation matrix inferred from the first position drives the virtual human's limb movements, there will be no jitter or violation of physical common sense.
  • Embodiment 2 is a diagrammatic representation of Embodiment 1:
  • FIG4 shows a schematic block diagram of a driving device for a virtual human according to an embodiment of the present application, wherein the driving device 400 for a virtual human comprises: a first processing module 402 for determining a first rotation matrix of a human joint according to a first human image acquired; a second processing module 404 for adding a constraint matrix to the first rotation matrix to determine a second rotation matrix of a human joint; The rotation matrix and the constraint matrix correspond to the physical constraints of the human joints; the third processing module 406 is used to smooth the second rotation matrix and determine the target rotation matrix of the human joints; the third processing module 406 is also used to drive the virtual human action according to the target rotation matrix.
  • the driving device 400 for a virtual human comprises: a first processing module 402 for determining a first rotation matrix of a human joint according to a first human image acquired; a second processing module 404 for adding a constraint matrix to the first rotation matrix to determine a second rotation matrix of a human joint; The rotation matrix and the constraint matrix correspond to the physical constraints of
  • the first human image represents a frame image captured in a video
  • the constraint matrix represents physical constraints on human joints
  • the physical constraints represent conditions for limiting the movement of human joints to prevent their movements from exceeding certain limits
  • the rotation matrix represents a mathematical expression of human joint posture
  • the first rotation matrix represents the rotation matrix of human joints detected by a human posture detection network
  • the target rotation matrix represents the second rotation matrix after smoothing.
  • the first processing module 402 first determines the rotation matrix of each human joint of the virtual person according to the first human image collected from the video, that is, the first rotation matrix. Specifically, the first processing module 402 can determine the first rotation matrix by inputting the first human image into a human posture detection network, but is not limited thereto.
  • the second processing module 404 adds a constraint matrix to the first rotation matrix of the human body joint to determine a physically corrected rotation matrix, ie, the second rotation matrix.
  • the first rotation matrix can be used to clarify the movement of each human joint between the three axes of the spatial coordinates XYZ, and by adding the constraint matrix, it can be limited on which axis of the three axes of XYZ the above human joint moves, as well as the limit of the movement on each axis. Therefore, the second processing module 404 can correct and limit the first rotation matrix by adding the constraint matrix to the first rotation matrix of the human joint to obtain a corrected second rotation matrix.
  • the third processing module 406 performs smoothing processing on the second rotation matrix obtained after adding the constraint matrix to determine a target rotation matrix for driving the above-mentioned virtual human limb movements.
  • the third processing module 406 needs to smooth the second rotation matrix to obtain the target rotation matrix after smoothing.
  • the third processing module 406 drives the virtual human according to the target rotation matrix, so that the limbs of the virtual human present movements that are adapted to the first human body image.
  • the first processing module 402 can determine the first human body image collected from the video.
  • the first rotation matrix of each human joint of the virtual person is determined, and the second processing module 404 can add a constraint matrix to the first rotation matrix to determine the second rotation matrix.
  • the third processing module 406 can smooth the parameters in the second rotation matrix to determine the target rotation matrix, and can drive the virtual person's body movements according to the target rotation matrix.
  • the third processing module 406 adds a constraint matrix to the target rotation matrix used to drive the virtual person's body movements and performs smoothing. In this way, the actions performed by the virtual person will not violate physical common sense, and there will be no jitter during the action. This solves the problem in the related art that directly using the detection results of the human posture detection network to drive the virtual person to perform actions may violate physical common sense and there is a jitter problem, thereby ensuring the driving effect of the virtual person.
  • the second processing module 404 is specifically used to use the first rotation matrix of the sub-joint of the human joint as the constraint matrix; compensate the constraint matrix to the first rotation matrix of the human joint to determine the second rotation matrix.
  • the second processing module 404 can use the first rotation matrix corresponding to the sub-joint as a constraint matrix, and compensate the constraint matrix to the first rotation matrix of the human joint to physically constrain the virtual human's limb movements, and determine the second rotation matrix of each human joint of the virtual human. In this way, the rotation of the human joint on the XYZ axis and the positional relationship of the human joint relative to the human body are restricted, ensuring that no violation of physical common sense occurs when driving the virtual human's limb movements.
  • the first rotation matrices of different sub-joints are different.
  • the sub-joints of different human joints are different, for example, the sub-joint of the thigh is the calf, and the sub-joint of the elbow is the arm. It can be understood that the first rotation matrices of different sub-joints are also different, so the constraint matrices added to different human joints are also different.
  • the third processing module 406 is specifically used to determine the first acceleration of the human joint movement and the first external force exerted on the human joint according to the second rotation matrix; determine the first position of the human joint according to the first acceleration and the first external force; and determine the target rotation matrix according to the first position.
  • the third processing module 406 can solve the first acceleration and the first external force of the human joint according to the second rotation matrix, determine the first position of the human joint after smoothing by solving the optimal relationship between the first acceleration and the first external force, and infer the target rotation matrix according to the first position. In this way, the target rotation matrix used to drive the virtual human limb movement is smoothed, ensuring that there is no shaking in the process of driving the virtual human limb movement.
  • the driving device 400 of the virtual human further includes an acquisition module 408.
  • the third processing module 406 is specifically used to determine the second position of the human joint according to the second rotation matrix; the acquisition module 406 is used to acquire the second human joint. body image and the third body image, as well as the acquisition time information of the second body image and the third body image; the third processing module 406 is specifically used to determine the first historical position of the human joints in the second body image and the second historical position of the human joints in the third body image; determine the first speed and the first acceleration of the human joint movement according to the first historical position, the second historical position, the second position and the acquisition time information; determine the first external force according to the second position, the first speed and the first acceleration; wherein the second body image and the third body image are images acquired before the first body image.
  • the third processing module 406 solves the first external force exerted on each joint of the virtual person and the first acceleration of each joint of the virtual person based on the second rotation matrix obtained after adding the constraint matrix, which ensures the accuracy of the determined first external force and second acceleration, and further ensures the accuracy of the target rotation matrix determined in the subsequent steps.
  • the third processing module 406 is specifically used to determine the first speed according to the second historical position, the second position and the collection time information; determine the second speed according to the first historical position, the second historical position and the collection time information; determine the first acceleration according to the difference between the first speed and the second speed.
  • the third processing module 406 calculates the speed and acceleration of each human joint in the first human image according to the position of the human joint in the human image collected before the first human image, that is, the driving device calculates the speed and acceleration in the most recent time. In this way, the accuracy of determining the first acceleration and the first speed is guaranteed.
  • the third processing module 406 is specifically configured to determine the first external force according to the second position, the first velocity and the first acceleration based on the friction cone constraint condition by using the rigid body motion equation.
  • the third processing module 406 can solve the first external force that satisfies the friction cone constraint condition according to the first acceleration, the first velocity and the second position through the rigid body motion equation, thereby ensuring the accuracy of determining the first external force.
  • the third processing module 406 is specifically configured to solve the first position according to the first acceleration and the first external force through an iterative algorithm.
  • the above-mentioned first position is the optimal position solved by the third processing module 406 through an iterative algorithm. In this way, the accuracy of the determined first position is guaranteed, and further ensures that when the target rotation matrix inferred from the first position drives the virtual human's limb movements, there will be no jitter or violation of physical common sense.
  • FIG. 5 shows a schematic block diagram of another driving device 500 of a virtual human according to an embodiment of the present application.
  • the driving device 500 includes: a memory 502, in which a program or instruction is stored; a processor 504, which executes the program or instruction stored in the memory 502 to implement the steps of the driving method of the virtual person proposed in the above-mentioned embodiment of the present application, and thus has all the beneficial technical effects of the driving method of the virtual person proposed in the above-mentioned embodiment of the present application, which will not be elaborated herein.
  • Embodiment 4 is a diagrammatic representation of Embodiment 4:
  • a readable storage medium on which a program or instruction is stored.
  • the steps of the virtual human driving method proposed in the above embodiment of the present application are implemented, and thus the program or instruction has all the beneficial technical effects of the virtual human driving method proposed in the above embodiment of the present application, which will not be elaborated herein.
  • an electronic device comprising a driving device for a virtual person as proposed in the above embodiments of the present application, and/or a readable storage medium as proposed in the above embodiments of the present application. Therefore, the electronic device has all the beneficial effects of the driving device for a virtual person as proposed in the above embodiments of the present application and/or the readable storage medium as proposed in the above embodiments of the present application, which will not be repeated here.
  • a computer program product including a computer program.
  • the computer program When the computer program is executed by a processor, it implements the steps of the virtual human driving method proposed in the above embodiments of the present application, and thus has all the beneficial technical effects of the virtual human driving method proposed in the above embodiments of the present application, which will not be elaborated herein.
  • This embodiment exemplifies the driving method of the virtual human proposed in this application in conjunction with Figure 6. Specifically, as shown in Figure 6, the driving device first determines the predicted posture of the human body through the collected human body image using the human body posture detection network, that is, determines the first rotation matrix of the human body joint, and then adds a constraint matrix to the rotation matrix to determine the second rotation matrix after physical constraints.
  • the driving device inputs the determined second rotation matrix into the PD controller to solve the acceleration of the human body shutdown, and solves the first external force of the human body joints through the acceleration, and then solves the optimal combination of the acceleration and the first external force through an iterative algorithm, and smoothes the second rotation matrix according to the optimal combination to determine the target rotation matrix for driving the virtual human action.
  • connection can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium.
  • connection can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected or indirectly connected through an intermediate medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Multimedia (AREA)
  • Mathematical Physics (AREA)
  • Mathematical Analysis (AREA)
  • Human Computer Interaction (AREA)
  • Data Mining & Analysis (AREA)
  • Pure & Applied Mathematics (AREA)
  • Mathematical Optimization (AREA)
  • Computational Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Social Psychology (AREA)
  • Psychiatry (AREA)
  • General Health & Medical Sciences (AREA)
  • Algebra (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Databases & Information Systems (AREA)
  • Software Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Processing Or Creating Images (AREA)
  • User Interface Of Digital Computer (AREA)

Abstract

本申请提出了一种虚拟人的驱动方法、装置、存储介质和电子设备。该虚拟人的驱动方法包括：根据采集的第一人体图像，确定人体关节的第一旋转矩阵；对第一旋转矩阵添加约束矩阵，确定人体关节的第二旋转矩阵，约束矩阵对应于人体关节的物理约束条件；对第二旋转矩阵进行平滑处理，确定人体关节的目标旋转矩阵；根据目标旋转矩阵驱动虚拟人动作。

Description

虚拟人的驱动方法、装置、存储介质和电子设备
本申请要求于2022年10月13日提交到中国国家知识产权局、申请号为“202211254965.4”，申请名称为“虚拟人的驱动方法、装置、存储介质和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请属于虚拟人技术领域，具体而言，涉及一种虚拟人的驱动方法、装置、存储介质和电子设备。
背景技术
利用真实人体动作对虚拟人进行驱动，是当前虚拟人驱动的主要方法。如何使得所驱动的虚拟人动作更为自然流畅，是虚拟人驱动中所要解决的主要问题之一。
申请内容
本申请旨在至少解决相关技术中存在的技术问题之一。
为此，本申请的第一个方面在于提出一种虚拟人的驱动方法。
本申请的第二个方面在于提出一种虚拟人的驱动装置。
本申请的第三个方面在于提出另一种虚拟人的驱动装置。
本申请的第四个方面在于提出一种可读存储介质。
本申请的第五个方面在于提出一种电子设备。
本申请的第六个方面在于提出一种计算机程序产品。
有鉴于此，根据本申请的一个方面，提出了一种虚拟人的驱动方法，该驱动方法包括：根据采集的第一人体图像，确定人体关节的第一旋转矩阵；对第一旋转矩阵添加约束矩阵，确定人体关节的第二旋转矩阵，约束矩阵对应于人体关节的物理约束条件；对第二旋转矩阵进行平滑处理，确定人体关节的目标旋转矩阵；根据目标旋转矩阵驱动虚拟人动作。
需要说明的是，本申请所提出的虚拟人的驱动方法的执行主体可以是虚拟人的驱动装置，为了更加清楚的对本申请提出的虚拟人的驱动方法进行说明，下面技术方案 中以虚拟人的驱动方法的执行主体为虚拟人的驱动装置进行示例性说明。
在该技术方案中，上述第一人体图像表示在视频中采集的某一帧图像，上述约束矩阵表示对于人体关节的物理约束条件，物理约束条件表示限制人体关节的运动，避免其动作超出一定的限度的条件；上述旋转矩阵表示人体关节姿态的一种数学表达形式，上述第一旋转矩阵表示通过人体姿态检测网络检测出的人体关节的旋转矩阵，上述目标旋转矩阵表示通过平滑处理后的第二旋转矩阵。
具体地，驱动装置首先根据从视频中采集到的第一人体图像，确定出虚拟人每个人体关节的旋转矩阵，即上述第一旋转矩阵。具体而言，驱动装置可以通过将上述第一人体图像输入至人体姿态检测网络的方式，确定该第一旋转矩阵，但不限于此。
进一步地，检测装置对人体关节的第一旋转矩阵添加约束矩阵，以确定出经过物理修正后的旋转矩阵，即上述第二旋转矩阵。具体而言，通过第一旋转矩阵可以明确每个人体关节在空间坐标XYZ三轴之间的动作情况，通过添加约束矩阵，可以限制上述人体关节在XYZ三轴中哪个轴上动作，以及在每个轴上动作的极限。因此，驱动装置可以通过对人体关节的第一旋转矩阵添加约束矩阵的方式，对第一旋转矩阵进行修正和限制，得出修正后的第二旋转矩阵。
进一步地，驱动装置平滑处理添加约束矩阵后得出的第二旋转矩阵，以确定用于驱动上述虚拟人肢体动作的目标旋转矩阵。
具体而言，如果直接根据上述第二旋转矩阵驱动上述虚拟人肢体动作，可能会由于第二旋转矩阵中某些参数之间关系不合理，例如，上述人体关节动作的加速度和该人体关节所受的外力之间的关系不合理，使得驱动虚拟人肢体动作时会发生抖动，所以，驱动装置需要平滑处理上述第二旋转矩阵，以得出平滑处理后的目标旋转矩阵。
进一步地，驱动装置根据上述目标旋转矩阵对上述虚拟人进行驱动，以使上述虚拟人的肢体呈现出与第一人体图像相适应的动作。
在该技术方案中，驱动装置能够根据从视频中采集到的第一人体图像，确定出虚拟人每个人体关节的第一旋转矩阵，能够对该第一旋转矩阵添加约束矩阵确定第二旋转矩阵，能够对该第二旋转矩阵中的参数进行平滑处理确定出目标旋转矩阵，能够根据该目标旋转矩阵驱动虚拟人肢体动作。在本申请的技术方案中，驱动装置在用于驱动虚拟人肢体动作的目标矩阵添加了约束矩阵并进行了平滑处理，这样，使得虚拟人做出的动作不会违反物理常理，且动作过程中不会抖动，解决了相关技术中，直接运 用人体姿态检测网络的检测结果驱动虚拟人做动作可能会违反物理常理，并存在抖动的问题，保证了虚拟人的驱动效果。
此外，根据本申请的上述技术方案提出的虚拟人的驱动方法，还可以具有以下附加技术特征：
在上述技术方案中，对第一旋转矩阵添加约束矩阵，确定人体关节的第二旋转矩阵的步骤具体包括：以人体关节的子关节的第一旋转矩阵作为约束矩阵；将约束矩阵补偿给人体关节的第一旋转矩阵，确定第二旋转矩阵。
在该技术方案中，驱动装置能够通过将子关节对应的第一旋转矩阵作为约束矩阵，并将该约束矩阵补偿给人体关节的第一旋转矩阵对虚拟人肢体动作进行物理约束，确定出虚拟人每个人体关节的第二旋转矩阵。这样，限制了人体关节在XYZ轴上的转动以及人体关节相对于人体的位置关系，保证了驱动虚拟人肢体动作时不会出现违反物理常理的情况。
在上述技术方案中，不同的子关节的第一旋转矩阵不相同。
在该技术方案中，不同人体关节的子关节是不相同的，例如，大腿的子关节为小腿，手肘的子关节为手臂。可以理解的是，不同的子关节的第一旋转矩阵也是不相同的，所以对于不同人体关节添加的约束矩阵也是不相同的。在上述技术方案中，对第二旋转矩阵进行平滑处理，确定人体关节的目标旋转矩阵的步骤具体包括：根据第二旋转矩阵确定人体关节动作的第一加速度和人体关节受到的第一外力；根据第一加速度和第一外力确定人体关节的第一位置；根据第一位置确定目标旋转矩阵。
在该技术方案中，驱动装置可以将根据上述第二旋转矩阵求解出人体关节的第一加速度以及第一外力，可以通过求解第一加速度与第一外力的最优关系确定出平滑处理后的人体关节的第一位置，可以根据第一位置反推出目标旋转矩阵。这样，使得用于驱动虚拟人肢体动作的目标旋转矩阵经过了平滑处理，保证了驱动虚拟人肢体动作的过程中不会抖动。
在上述技术方案中，根据第二旋转矩阵确定人体关节动作的第一加速度和人体关节受到的第一外力的步骤具体包括：根据第二旋转矩阵确定人体关节的第二位置；获取第二人体图像和第三人体图像，以及第二人体图像和第三人体图像的采集时刻信息；确定第二人体图像中人体关节的第一历史位置和第三人体图像中人体关节的位置第二历史位置；根据第一历史位置、第二历史位置、第二位置和采集时刻信息，确定人体 关节动作的第一速度和第一加速度；根据第二位置、第一速度和第一加速度确定第一外力；其中，第二人体图像和第三人体图像为在第一人体图像之前采集的图像。
在该技术方案中，驱动装置是根据添加约束矩阵后得出的第二旋转矩阵求解虚拟人每个人体关节受到的第一外力和每个人体关节的第一加速度，这保证了确定出的第一外力和第二加速度的准确性，进而保证了后续步骤中确定出的目标旋转矩阵的准确性。
在上述技术方案中，根据第一历史位置、第二历史位置和第二位置确定人体关节动作的第一速度和第一加速度的步骤具体包括：根据第二历史位置、第二位置和采集时刻信息确定第一速度；根据第一历史位置、第二历史位置和采集时刻信息确定第二速度；根据第一速度和第二速度的差值确定第一加速度。
在该技术方案中，驱动装置是根据在第一人体图像之前采集的人体图像中的人体关节的位置计算第一人体图像中每个人体关节的速度和加速度，即驱动装置计算出的是最近时间内的速度和加速度。这样，保证了确定出第一加速度和第一速度的准确性。
在上述技术方案中，根据第二位置、第一速度和第一加速度确定第一外力的步骤具体包括：通过刚体运动方程，基于摩擦锥约束条件根据第二位置、第一速度和第一加速度确定第一外力。
在该技术方案中，驱动装置能够通过刚体运动方程，根据上述第一加速度、上述第一速度以及上述第二位置求解出满足摩擦锥约束条件第一外力，这样，保证了确定出第一外力的准确性。
在上述技术方案中，根据第一加速度和第一外力确定人体关节的第一位置的步骤具体包括：通过迭代算法，根据第一加速度和第一外力求解第一位置。
在该技术方案中，上述第一位置是通过迭代算法求解出的最优位置，这样，保证了确定出的第一位置的准确性，进而保证根据该第一位置反推出的目标旋转矩阵驱动虚拟人肢体动作时，不会出现抖动和违反物理常理的情况。
根据本申请的第二个方面，提出了一种虚拟人的驱动装置，该虚拟人的驱动装置包括：第一处理模块，用于根据采集的第一人体图像，确定人体关节的第一旋转矩阵；第二处理模块，用于对第一旋转矩阵添加约束矩阵，确定人体关节的第二旋转矩阵，约束矩阵对应于人体关节的物理约束条件；第三处理模块，用于对第二旋转矩阵进行平滑处理，确定人体关节的目标旋转矩阵；第三处理模块还用于根据目标旋转矩阵驱 动虚拟人动作。
在该技术方案中，上述第一人体图像表示在视频中采集的某一帧图像，上述约束矩阵表示对于人体关节的物理约束条件，物理约束条件表示限制人体关节的运动，避免其动作超出一定的限度的条件；上述旋转矩阵表示人体关节姿态的一种数学表达形式，上述第一旋转矩阵表示通过人体姿态检测网络检测出的人体关节后的旋转矩阵，上述目标旋转矩阵表示通过平滑处理后的第二旋转矩阵。
具体地，第一处理模块首先根据从视频中采集到的第一人体图像，确定出虚拟人每个人体关节的旋转矩阵，即上述第一旋转矩阵。具体而言，第一处理模块可以通过将上述第一人体图像输入至人体姿态检测网络的方式，确定该第一旋转矩阵，但不限于此。
进一步地，第二处理模块对人体关节的第一旋转矩阵添加约束矩阵，以确定出经过物理修正后的旋转矩阵，即上述第二旋转矩阵。
具体而言，通过第一旋转矩阵可以明确每个人体关节在空间坐标XYZ三轴之间的动作情况，通过添加约束矩阵，可以限制上述人体关节在XYZ三轴中哪个轴上动作，以及在每个轴上动作的极限。因此，第二处理模块可以通过对人体关节的第一旋转矩阵添加约束矩阵的方式，对第一旋转矩阵进行修正和限制，得出修正后的第二旋转矩阵。
进一步地，第三处理模块平滑处理添加约束矩阵后得出的第二旋转矩阵，以确定用于驱动上述虚拟人肢体动作的目标旋转矩阵。
具体而言，如果直接根据上述第二旋转矩阵驱动上述虚拟人肢体动作，可能会由于第二旋转矩阵中某些参数之间关系不合理，例如，上述人体关节动作的加速度和该人体关节所受的外力之间的关系不合理，使得驱动虚拟人肢体动作时会发生抖动，所以，第三处理模块需要平滑处理上述第二旋转矩阵，以得出平滑处理后的目标旋转矩阵。
进一步地，第三处理模块根据上述目标旋转矩阵对上述虚拟人进行驱动，以使上述虚拟人的肢体呈现出与第一人体图像相适应的动作。
在该技术方案中，第一处理模块能够根据从视频中采集到的第一人体图像，确定出虚拟人每个人体关节的第一旋转矩阵，第二处理模块能够对该第一旋转矩阵添加约束矩阵确定第二旋转矩阵，第三处理模块能够对该第二旋转矩阵中的参数进行平滑处 理确定出目标旋转矩阵，并能够根据该目标旋转矩阵驱动虚拟人肢体动作。在本申请的技术方案中，第三处理模块在用于驱动虚拟人肢体动作的目标旋转矩阵添加了约束矩阵并进行了平滑处理，这样，使得虚拟人做出的动作不会违反物理常理，且动作过程中不会抖动，解决了相关技术中，直接运用人体姿态检测网络的检测结果驱动虚拟人做动作可能会违反物理常理，并存在抖动的问题，保证了虚拟人的驱动效果。
根据本申请第三个方面，提出了另一种虚拟人的驱动装置，包括：存储器，存储器中存储有程序或指令；处理器，处理器执行存储在存储器中的程序或指令以实现如本申请上述技术方案提出的虚拟人的驱动方法的步骤，因而具有本申请上述技术方案提出的虚拟人的驱动方法的全部有益技术效果，在此不再做过多赘述。
根据本申请的第四个方面，提出了一种可读存储介质，其上存储有程序或指令，程序或指令被处理器执行时实现如本申请上述技术方案提出的虚拟人的驱动方法的步骤，因而具有本申请上述技术方案提出的虚拟人的驱动方法的全部有益技术效果，在此不再做过多赘述。
根据本申请的第五个方面，提出了一种电子设备，包括如本申请上述技术方案提出的虚拟人的驱动装置，和/或如本申请上述技术方案提出的可读存储介质，因此，该电子设备具备本申请上述技术方案提出的虚拟人的驱动装置和/或本申请上述技术方案提出的可读存储介质的全部有益效果，在此不再赘述。
根据本申请的第六个方面，提出了一种计算机程序产品，包括计算机程序，计算机程序被处理器执行时实现如本申请上述技术方案提出的虚拟人的驱动方法的步骤，因而具有本申请上述技术方案提出的虚拟人的驱动方法的全部有益技术效果，在此不再做过多赘述。
本申请的附加方面和优点将在下面的描述部分中变得明显，或通过本申请的实践了解到。
附图说明
本申请的上述和/或附加的方面和优点从结合下面附图对实施例的描述中将变得明显和容易理解，其中：
图1示出了本申请实施例的虚拟人的驱动方法的流程示意图之一；
图2示出了本申请实施例的虚拟人的驱动方法的流程示意图之二；
图3示出了本申请实施例的虚拟人的驱动方法的流程示意图之三；
图4示出了本申请实施例的虚拟人的驱动装置的示意框图之一；
图5示出了本申请实施例的虚拟人的驱动装置的示意框图之二；
图6示出了本申请实施例的虚拟人的驱动方法的整体流程框图。
具体实施方式
为了能够更清楚地理解本申请的上述目的、特征和优点，下面结合附图和具体实施方式对本申请进行进一步的详细描述。需要说明的是，在不冲突的情况下，本申请的实施例及实施例中的特征可以相互组合。
在下面的描述中阐述了很多具体细节以便于充分理解本申请，但是，本申请还可以采用其他不同于在此描述的其他方式来实施，因此，本申请的保护范围并不限于下面公开的具体实施例的限制。
下面结合图1至图6，通过具体的实施例及其应用场景对本申请实施例提供的一种虚拟人的驱动方法、装置、存储介质和电子设备进行详细地说明。
实施例一：
图1示出了本申请实施例的虚拟人的驱动方法的流程示意图，该驱动方法包括：
S102，根据采集的第一人体图像，确定人体关节的第一旋转矩阵；
S104，对第一旋转矩阵添加约束矩阵，确定人体关节的第二旋转矩阵，约束矩阵对应于人体关节的物理约束条件；
S106，对第二旋转矩阵进行平滑处理，确定人体关节的目标旋转矩阵；
S108，根据目标旋转矩阵驱动虚拟人动作。
在该实施例中，上述第一人体图像表示在视频中采集的某一帧图像，上述约束矩阵表示对于人体关节的物理约束条件，物理约束条件表示限制人体关节的运动，避免其动作超出一定的限度的条件；上述旋转矩阵表示人体关节姿态的一种数学表达形式，上述第一旋转矩阵表示通过人体姿态检测网络检测出的人体关节后的旋转矩阵，上述目标旋转矩阵表示通过平滑处理后的第二旋转矩阵。
具体地，驱动装置首先根据从视频中采集到的第一人体图像，确定出虚拟人每个人体关节的旋转矩阵，即上述第一旋转矩阵。具体而言，驱动装置可以通过将上述第一人体图像输入至人体姿态检测网络的方式，确定该第一旋转矩阵，但不限于此，其 他人体姿态检测方法同样适用于本申请。进一步地，检测装置对人体关节的第一旋转矩阵添加约束矩阵，以确定出经过物理修正后的旋转矩阵，即上述第二旋转矩阵。具体而言，通过第一旋转矩阵可以明确每个人体关节在空间坐标XYZ三轴之间的动作情况，通过添加约束矩阵，可以限制上述人体关节在XYZ三轴中哪个轴上动作，以及在每个轴上动作的极限。因此，驱动装置可以通过对人体关节的第一旋转矩阵添加约束矩阵的方式，对第一旋转矩阵进行修正和限制，以得出修正后的第二旋转矩阵。
示例性地，以人体关节为小腿为例，将其第一旋转矩阵在XYZ三轴进行分解，可以发现小腿仅存在对于Y轴的转动，但如果直接删除小腿在XZ轴上的旋转，可能会导致小腿位置的偏移，所以，可以对小腿在XZ轴上的旋转进行物理约束，并限制小腿相对于人体位置的关系，即对小腿的第一旋转矩阵添加约束矩阵。这样，可以得出小腿的第二旋转矩阵。
进一步地，驱动装置平滑处理添加约束矩阵后得出的第二旋转矩阵，以确定用于驱动上述虚拟人肢体动作的目标旋转矩阵。
具体而言，如果直接根据上述第二旋转矩阵驱动上述虚拟人肢体动作，可能会由于第二旋转矩阵中某些参数之间关系不合理，例如，上述人体关节动作的加速度和该人体关节所受的外力之间的关系不合理，使得驱动虚拟人肢体动作时会发生抖动，所以，驱动装置需要平滑处理上述第二旋转矩阵，以得出平滑处理后的目标旋转矩阵。
进一步地，驱动装置根据上述目标旋转矩阵对上述虚拟人进行驱动，以使上述虚拟人的肢体呈现出与第一人体图像相适应的动作。
根据申请人研究发现，直接运用人体姿态检测网络的检测结果驱动虚拟人做动作，虚拟人做出的动作可能会违反物理常理，且可能会出现严重的抖动。因此，在本申请的实施例中，驱动装置在用于驱动虚拟人肢体动作的目标矩阵添加了约束矩阵并进行了平滑处理，这样，使得虚拟人做出的动作不会违反物理常理，且动作过程中不会抖动，保证了虚拟人的驱动效果。
图2示出了本申请实施例的虚拟人的驱动方法的流程示意图，该驱动方法包括：
S202，根据采集的第一人体图像，确定人体关节的第一旋转矩阵；
S204，以人体关节的子关节的第一旋转矩阵作为约束矩阵，并将约束矩阵补偿给人体关节的第一旋转矩阵，确定第二旋转矩阵；
S206，对第二旋转矩阵进行平滑处理，确定人体关节的目标旋转矩阵；
S208，根据目标旋转矩阵驱动虚拟人动作。
在该实施例中，确定上述第二旋转矩阵的过程为：驱动装置首先确定人体关节的子关节，然后根据人体关节的子关节确定出对于人体关节的第一旋转矩阵的约束矩阵。
进一步地，在确定出人体关节的子关节后，驱动装置会将子关节对应的第一旋转矩阵作为约束矩阵，并将该约束矩阵补偿给人体关节的第一旋转矩阵，以确定出每个人体关节的第二旋转矩阵。具体地，将子关节对应的第一旋转矩阵补偿给人体关关节的第一旋转矩阵后，得出的第二旋转矩阵中限制了人体关节在XYZ轴上的转动以及该人体关节相对于人体的位置关系。
示例性地，以人体关节为大腿，其子关节为小腿，大腿的第一旋转矩阵为C0，小腿的第一旋转矩阵为C1进行示例。因为小腿只能存在Y轴的转动，因此我们可以对小腿的第一旋转矩阵依次沿着XYZ轴分解，得出C1＝CY1CX1CZ1，其中，CY1为C1在Y轴方向的矩阵，CX1为C1在X轴方向的矩阵，CZ1为C1在Z轴方向的矩阵，将C1补偿给C0，则可得出大腿和小腿的第二旋转矩阵为C′1＝CY1,C′0＝CZ1CZ1C0，这样，既保证了小腿可以绕Y轴转动，又保证了小腿相对于人体的位置不变。
在该实施例中，驱动装置能够通过将子关节对应的第一旋转矩阵作为约束矩阵，并将该约束矩阵补偿给人体关节的第一旋转矩阵对虚拟人肢体动作进行物理约束，确定出虚拟人每个人体关节的第二旋转矩阵。这样，限制了人体关节在XYZ轴上的转动以及人体关节相对于人体的位置关系，保证了驱动虚拟人肢体动作时不会出现违反物理常理的情况。
在上述实施例中，不同的子关节的第一旋转矩阵不相同。
在该实施例中，不同人体关节的子关节是不相同的，例如，大腿的子关节为小腿，手肘的子关节为手臂。可以理解的是，不同的子关节的第一旋转矩阵也是不相同的，所以对于不同人体关节添加的约束矩阵也是不相同的。
图3示出了本申请实施例的虚拟人的驱动方法的流程示意图，该驱动方法包括：
S302，根据采集的第一人体图像，确定人体关节的第一旋转矩阵；
S304，对第一旋转矩阵添加约束矩阵，确定人体关节的第二旋转矩阵，约束矩阵对应于人体关节的物理约束条件；
S306，根据第二旋转矩阵确定人体关节动作的第一加速度和人体关节受到的第一外力；
S308，根据第一加速度和第一外力确定人体关节的第一位置；
S310，根据第一位置确定目标旋转矩阵；
S312，根据目标旋转矩阵驱动虚拟人动作。
在该实施例中，上述第一外力表示人体关节受到的外力，例如地面对于小腿的反作用力，上述第一位置表示经过物理约束和平滑处理后确定出的人体关节的位置。
具体地，驱动装置首先根据上述第二旋转矩阵求解人体关节当前的加速度，即上述第一加速度，以及人体关节当前受到的外力，即上述第一外力。
具体而言，根据上述第二旋转矩阵中的参数可以求解出人体关节当前位置的信息，根据这个位置信息结合采集上述第一人体图像时间，以及采集上述第一人体图像之前，人体关节的位置以及时间，可以确定出人体关节从基础位置动作至上述当前位置的速度以及加速度，根据速度和加速度可以求解出人体关节所受到的外力。因此，驱动装置可以根据上述第二旋转矩阵确定出上述第一加速度以及上述第一外力。
进一步地，驱动装置通过上述第一加速度以及上述第一外力求解出平滑处理后人体关节的位置，即上述第一位置。具体而言，驱动装置根据上述第一加速度以及上述第一外力可以求解人体关节的加速度和所受外力的最优关系，根据这个最优关系可以求解出平滑后的人体关节的第一位置。
进一步地，驱动装置根据确定出的第一位置反推出用于控制虚拟人肢体动作的旋转矩阵，即上述目标旋转矩阵。
在该实施例中，驱动装置可以将根据上述第二旋转矩阵求解出人体关节的第一加速度以及第一外力，可以通过求解第一加速度与第一外力的最优关系确定出平滑处理后的人体关节的第一位置，可以根据第一位置反推出目标旋转矩阵。这样，使得用于驱动虚拟人肢体动作的目标旋转矩阵经过了平滑处理，保证了驱动虚拟人肢体动作的过程中不会抖动。
在上述实施例中，根据第二旋转矩阵确定人体关节动作的第一加速度和人体关节受到的第一外力的步骤具体包括：根据第二旋转矩阵确定人体关节的第二位置；获取第二人体图像和第三人体图像，以及第二人体图像和第三人体图像的采集时刻信息；确定第二人体图像中人体关节的第一历史位置和第三人体图像中人体关节的位置第二历史位置；根据第一历史位置、第二历史位置、第二位置和采集时刻信息，确定人体关节动作的第一速度和第一加速度；根据第二位置、第一速度和第一加速度确定第一 外力；其中，第二人体图像和第三人体图像为在第一人体图像之前采集的图像。
在该实施例中，上述第二人体图像和上述第三人体图像表示在第一人体图像之前采集的图，且第二人体图像的采集的帧时序要早于第三人体图像；上述第一历史位置表示第二人体图像中的人体关节的位置，上述第二历史位置表示第三人体图像中的人体关节的位置，上述第二位置表示通过物理约束后，第一人体图像中人体关节的位置。
需要说明的是，上述人体关节包括虚拟人的每个人体关节，为了更加清楚的描述本申请的方案，下面以人体关节中的人体关节A进行示例说明。
具体地，确定上述第一外力和上述第一加速度的过程为：驱动装置首先根据上述第二旋转矩阵求解人体关节A经过物理约束后的第二位置。
进一步地，驱动装置获取上述第二人体图像以及上述第三人体图像，以及第二人体图像和第三人体图像的采集时刻信息，并确定人体关节A的第一历史位置以及第二历史位置。具体而言，驱动装置可以先将上述第二人体图像以及上述第三人体图像输入至人体姿态检测网络中，确定人体关节A在第二人体图像中的旋转矩阵和人体关节A在第三人体图像中的旋转矩阵，然后在根据这两个旋转矩阵求解出上述第一历史位置和上述第二历史位置。
进一步地，驱动装置上述第二位置、上述第一历史位置、上述第二历史位置以及上述采集时刻信息求解出人体关节A的第一加速度和第一速度。具体而言，驱动装置根据上述三个位置中任两个位置可以确定出距离，驱动装置通过该距离和上述采集时刻信息，即可计算出上述第一加速度和第一速度。
进一步地，驱动装置根据上述第一加速度、上述第一速度以及上述第一外力求解人体关节A所受的第一外力。具体而言，人体关节A所受到的第一外力与上述第一加速度、上述第一速度以及上述第一外力存在一定的关系，如刚体运动方程限制的关系。驱动装置根据这个关系，通过上述第一加速度、上述第一速度以及上述第一外力即可求解出人体关节A所受的第一外力。
在该实施例中，驱动装置是根据添加约束矩阵后得出的第二旋转矩阵求解虚拟人每个人体关节受到的第一外力和每个人体关节的第一加速度，这保证了确定出的第一外力和第二加速度的准确性，进而保证了后续步骤中确定出的目标旋转矩阵的准确性。
在上述实施例中，根据第一历史位置、第二历史位置、第二位置和采集时刻信息确定人体关节动作的第一速度和第一加速度的步骤具体包括：根据第二历史位置、第 二位置和采集时刻信息确定第一速度；根据第一历史位置、第二历史位置和采集时刻信息确定第二速度；根据第一速度和第二速度的差值确定第一加速度。
在该实施例中，确定上述第一速度和上述第一加速度的过程为：驱动装置首先根据上述第二位置、上述第二历史位置和上述采集时刻信息计算上述第一速度。具体而言，根据上述第二位置和上述第二历史位置可以明确两个位置之间的距离，驱动装置根据该距离和采集时刻信息即可计算出上述第一速度。
进一步地，驱动装置根据上述第第一历史位置、上述第二历史位置和上述采集时刻信息计算第二速度。具体而言，根据上述第一历史位置和上述第二历史位置可以明确两个位置之间的距离，驱动装置根据该距离和采集时刻信息即可计算出上述第二速度。
进一步地，驱动装置计算上述第一速度和上述第二速度之间的差值，以确定出上述第一加速度。
示例性地，驱动装置还可以通过PD比例微分控制器计算，计算上述第一加速度，计算公式为：q″d＝q″+kd(q′d-q′)+kp(qd-q)，其中，q表示上述第二位置，q′表示上述第一速度，q″表示上述第一加速度，qd、q′d和q″d根据预先设计的规划路线确定，kp表示比例系数，kd表示微分系数。
在该实施例中，驱动装置是根据在第一人体图像之前采集的人体图像中的人体关节的位置计算第一人体图像中每个人体关节的速度和加速度，即驱动装置计算出的是最近时间内的速度和加速度。这样，保证了确定出第一加速度和第一速度的准确性。
在上述实施例中，根据第二位置、第一速度和第一加速度确定第一外力的步骤具体包括：通过刚体运动方程，基于摩擦锥约束条件根据第二位置、第一速度和第一加速度确定第一外力。
在该实施例中，确定上述第一外力的过程为：驱动装置采用刚体运动方程，在满足摩擦锥约束条件的情况下，根据上述第一加速度、上述第一速度以及上述第二位置求解上述第一外力。
具体地，上述刚体运动方程为：τ＝M(q)q″+c(q,q′)，将其进行转换可以得出方程f＝‖Mq″d+c(q,q′)-JGλ‖，其中，q表示上述第二位置，q′表示上述第一速度，q″表示上述第一加速度，q″d表示平滑处理后的第一加速度，M表示转动惯量矩阵，J表示描述虚拟人受力与每个人体关节关系的雅可比矩阵，G是将上述第一外力λ转换为线性力 和扭矩的转换矩阵，c表示系统所受的非外力，τ和f表示关于时间和刚体运动状态的已知函数。
具体地，上述摩擦锥约束条件的公式为：其中，F表示摩擦锥约束，λn表示法向分量，λt和λb表示切向分量，μ表示摩擦系数，R表示三维向量。
进一步地，通过限定min f s.t.F，解出上述第一外力，其中s.t的全称subject to，意思为使得“某某”满足“某某”，即上述公式的意思为使最小的f满足F。
在该实施例中，驱动装置能够通过刚体运动方程，根据上述第一加速度、上述第一速度以及上述第二位置求解出满足摩擦锥约束条件第一外力，这样，保证了确定出第一外力的准确性。
在上述实施例中，根据第一加速度和第一外力确定人体关节的第一位置的步骤具体包括：通过迭代算法，根据第一加速度和第一外力求解第一位置。
在该实施例中，确定上述第一位置的过程为：采用迭代算法，求解上述第一外力与上述第一加速度的最优组合关系，以确定最优的第一位置。
具体而言，上述迭代算法的条件为min‖q″-q″d‖+‖τ‖s.t.Mq″-τ＝JGλ-c(q,q′)，即使min‖q″-q″d‖+‖τ‖满足q″-τ＝JGλ-c(q,q′)，其中，该表达式中的字母含义与上述表达式相同，本申请在此不再赘述。
具体而言，虚拟人肢体动作的速度的表达式为：v＝Jq′，为了防止肢体动作穿透地面，本申请认为当肢体与地面接触时，垂直于地面的速度必须大于0，同时应约束切向速度，因此，在上述迭代算法中，还应该考虑0≤vn,|vt|≤σ,|vb|≤σ的约束条件，其中，v表示肢体动作的速度，vn表示垂直于地面的速度，vb和vt表示切向速度，σ预设的常数。
在该实施例中，上述第一位置是通过迭代算法求解出的最优位置，这样，保证了确定出的第一位置的准确性，进而保证根据该第一位置反推出的目标旋转矩阵驱动虚拟人肢体动作时，不会出现抖动和违反物理常理的情况。
实施例二：
图4示出了本申请实施例的虚拟人的驱动装置的示意框图，该虚拟人的驱动装置400包括：第一处理模块402，用于根据采集的第一人体图像，确定人体关节的第一旋转矩阵；第二处理模块404，用于对第一旋转矩阵添加约束矩阵，确定人体关节的第二 旋转矩阵，约束矩阵对应于人体关节的物理约束条件；第三处理模块406，用于对第二旋转矩阵进行平滑处理，确定人体关节的目标旋转矩阵；第三处理模块406还用于根据目标旋转矩阵驱动虚拟人动作。
在该实施例中，上述第一人体图像表示在视频中采集的某一帧图像，上述约束矩阵表示对于人体关节的物理约束条件，物理约束条件表示限制人体关节的运动，避免其动作超出一定的限度的条件；上述旋转矩阵表示人体关节姿态的一种数学表达形式，上述第一旋转矩阵表示通过人体姿态检测网络检测出的人体关节后的旋转矩阵上述目标旋转矩阵表示通过平滑处理后的第二旋转矩阵。
具体地，第一处理模块402首先根据从视频中采集到的第一人体图像，确定出虚拟人每个人体关节的旋转矩阵，即上述第一旋转矩阵。具体而言，第一处理模块402可以通过将上述第一人体图像输入至人体姿态检测网络的方式，确定该第一旋转矩阵，但不限于此。
进一步地，第二处理模块404对人体关节的第一旋转矩阵添加约束矩阵，以确定出经过物理修正后的旋转矩阵，即上述第二旋转矩阵。
具体而言，通过第一旋转矩阵可以明确每个人体关节在空间坐标XYZ三轴之间的动作情况，通过添加约束矩阵，可以限制上述人体关节在XYZ三轴中哪个轴上动作，以及在每个轴上动作的极限。因此，第二处理模块404可以通过对人体关节的第一旋转矩阵添加约束矩阵的方式，对第一旋转矩阵进行修正和限制，以得出修正后的第二旋转矩阵。
进一步地，第三处理模块406平滑处理添加约束矩阵后得出的第二旋转矩阵，以确定用于驱动上述虚拟人肢体动作的目标旋转矩阵。
具体而言，如果直接根据上述第二旋转矩阵驱动上述虚拟人肢体动作，可能会由于第二旋转矩阵中某些参数之间关系不合理，例如，上述人体关节动作的加速度和该人体关节所受的外力之间的关系不合理，使得驱动虚拟人肢体动作时会发生抖动，所以，第三处理模块406需要平滑处理上述第二旋转矩阵，以得出平滑处理后的目标旋转矩阵。
进一步地，第三处理模块406根据上述目标旋转矩阵对上述虚拟人进行驱动，以使上述虚拟人的肢体呈现出与第一人体图像相适应的动作。
在该实施例中，第一处理模块402能够根据从视频中采集到的第一人体图像，确 定出虚拟人每个人体关节的第一旋转矩阵，第二处理模块404能够对该第一旋转矩阵添加约束矩阵确定第二旋转矩阵，第三处理模块406能够对该第二旋转矩阵中的参数进行平滑处理确定出目标旋转矩阵，并能够根据该目标旋转矩阵驱动虚拟人肢体动作。在本申请的实施例中，第三处理模块406在用于驱动虚拟人肢体动作的目标旋转矩阵添加了约束矩阵并进行了平滑处理，这样，使得虚拟人做出的动作不会违反物理常理，且动作过程中不会抖动，解决了相关技术中，直接运用人体姿态检测网络的检测结果驱动虚拟人做动作可能会违反物理常理，并存在抖动的问题，保证了虚拟人的驱动效果。
在上述实施例中，第二处理模块404具体用于以人体关节的子关节的第一旋转矩阵作为约束矩阵；将约束矩阵补偿给人体关节的第一旋转矩阵，确定第二旋转矩阵。
在该实施例中，第二处理模块404能够通过将子关节对应的第一旋转矩阵作为约束矩阵，并将该约束矩阵补偿给人体关节的第一旋转矩阵对虚拟人肢体动作进行物理约束，确定出虚拟人每个人体关节的第二旋转矩阵。这样，限制了人体关节在XYZ轴上的转动以及人体关节相对于人体的位置关系，保证了驱动虚拟人肢体动作时不会出现违反物理常理的情况。
在上述实施例中，不同的子关节的第一旋转矩阵不相同。
在该实施例中，不同人体关节的子关节是不相同的，例如，大腿的子关节为小腿，手肘的子关节为手臂。可以理解的是，不同的子关节的第一旋转矩阵也是不相同的，所以对于不同人体关节添加的约束矩阵也是不相同的。
在上述实施例中，第三处理模块406具体用于根据第二旋转矩阵确定人体关节动作的第一加速度和人体关节受到的第一外力；根据第一加速度和第一外力确定人体关节的第一位置；根据第一位置确定目标旋转矩阵。
在该实施例中，第三处理模块406可以将根据上述第二旋转矩阵求解出人体关节的第一加速度以及第一外力，可以通过求解第一加速度与第一外力的最优关系确定出平滑处理后的人体关节的第一位置，可以根据第一位置反推出目标旋转矩阵。这样，使得用于驱动虚拟人肢体动作的目标旋转矩阵经过了平滑处理，保证了驱动虚拟人肢体动作的过程中不会抖动。
在上述实施例中，虚拟人的驱动装置400还包括获取模块408，第三处理模块406具体用于根据第二旋转矩阵确定人体关节的第二位置；获取模块408用于获取第二人 体图像和第三人体图像，以及第二人体图像和第三人体图像的采集时刻信息；第三处理模块406具体用于确定第二人体图像中人体关节的第一历史位置和第三人体图像中人体关节的位置第二历史位置；根据第一历史位置、第二历史位置、第二位置和采集时刻信息确定人体关节动作的第一速度和第一加速度；根据第二位置、第一速度和第一加速度确定第一外力；其中，第二人体图像和第三人体图像为在第一人体图像之前采集的图像。
在该实施例中，第三处理模块406是根据添加约束矩阵后得出的第二旋转矩阵求解虚拟人每个人体关节受到的第一外力和每个人体关节的第一加速度，这保证了确定出的第一外力和第二加速度的准确性，进而保证了后续步骤中确定出的目标旋转矩阵的准确性。
在上述实施例中，第三处理模块406具体用于根据第二历史位置、第二位置和采集时刻信息确定第一速度；根据第一历史位置、第二历史位置和采集时刻信息确定第二速度；根据第一速度和第二速度的差值确定第一加速度。
在该实施例中，第三处理模块406是根据在第一人体图像之前采集的人体图像中的人体关节的位置计算第一人体图像中每个人体关节的速度和加速度，即驱动装置计算出的是最近时间内的速度和加速度。这样，保证了确定出第一加速度和第一速度的准确性。
在上述实施例中，第三处理模块406具体用于通过刚体运动方程，基于摩擦锥约束条件根据第二位置、第一速度和第一加速度确定第一外力。
在该实施例中，第三处理模块406能够通过刚体运动方程，根据上述第一加速度、上述第一速度以及上述第二位置求解出满足摩擦锥约束条件第一外力，这样，保证了确定出第一外力的准确性。
在上述实施例中，第三处理模块406具体用于通过迭代算法，根据第一加速度和第一外力求解第一位置。
在该实施例中，上述第一位置是第三处理模块406通过迭代算法求解出的最优位置，这样，保证了确定出的第一位置的准确性，进而保证根据该第一位置反推出的目标旋转矩阵驱动虚拟人肢体动作时，不会出现抖动和违反物理常理的情况。
实施例三：
图5示出了本申请实施例的另一种虚拟人的驱动装置500的示意框图，该虚拟人 的驱动装置500包括：存储器502，存储器502中存储有程序或指令；处理器504，处理器504执行存储在存储器502中的程序或指令以实现如本申请上述实施例提出的虚拟人的驱动方法的步骤，因而具有本申请上述实施例提出的虚拟人的驱动方法的全部有益技术效果，在此不再做过多赘述。
实施例四：
根据本申请的第四个实施例，提出了一种可读存储介质，其上存储有程序或指令，程序或指令被处理器执行时实现如本申请上述实施例提出的虚拟人的驱动方法的步骤，因而具有本申请上述实施例提出的虚拟人的驱动方法的全部有益技术效果，在此不再做过多赘述。
实施例五：
根据本申请的第五个实施例，提出了一种电子设备，包括如本申请上述实施例提出的虚拟人的驱动装置，和/或如本申请上述实施例提出的可读存储介质，因此，该电子设备具备本申请上述实施例提出的虚拟人的驱动装置和/或本申请上述实施例提出的可读存储介质的全部有益效果，在此不再赘述。
实施例六：
根据本申请的第六个实施例，提出了一种计算机程序产品，包括计算机程序，计算机程序被处理器执行时实现如本申请上述实施例提出的虚拟人的驱动方法的步骤，因而具有本申请上述实施例提出的虚拟人的驱动方法的全部有益技术效果，在此不再做过多赘述。
实施例七：
本实施例结合图6对本申请提出的虚拟人的驱动方法进行示例性说明。具体地，如图6所示，驱动装置首先通过采集的人体图像，采用人体姿态检测网络确定出人体的预测姿态，即确定出人体关节的第一旋转矩阵，然后对该旋转矩阵添加约束矩阵，确定出经过物理约束的第二旋转矩阵。
进一步地，驱动装置将确定出的第二旋转矩阵输入至PD控制器中求解人体关机的加速度，并通过该加速度求解出人体关节的第一外力，然后通过迭代算法求解加速度和第一外力的最优组合，并根据最优组合对第二旋转矩阵进行平滑处理，确定出用于驱动虚拟人动作的目标旋转矩阵。
在本说明书的描述中，术语“第一”、“第二”仅用于描述的目的，而不能理解 为指示或暗示相对重要性，除非另有明确的规定和限定；术语“连接”、“安装”、“固定”等均应做广义理解，例如，“连接”可以是固定连接，也可以是可拆卸连接，或一体地连接；可以是直接相连，也可以通过中间媒介间接相连。对于本领域的普通技术人员而言，可以根据具体情况理解上述术语在本申请中的具体含义。
在本说明书的描述中，术语“一个实施例”、“一些实施例”、“具体实施例”等的描述意指结合该实施例或示例描述的具体特征、结构、材料或特点包含于本申请的至少一个实施例或示例中。在本说明书中，对上述术语的示意性表述不一定指的是相同的实施例或实例。而且，描述的具体特征、结构、材料或特点可以在任何的一个或多个实施例或示例中以合适的方式结合。
另外，本申请各个实施例之间的实施例可以相互结合，但是必须是以本领域普通技术人员能够实现为基础，当实施例的结合出现相互矛盾或无法实现时应当认为这种实施例的结合不存在，也不在本申请要求的保护范围之内。
以上仅为本申请的优选实施例而已，并不用于限制本申请，对于本领域的技术人员来说，本申请可以有各种更改和变化。凡在本申请的精神和原则之内，所作的任何修改、等同替换、改进等，均应包含在本申请的保护范围之内。

Claims (13)

  1. 一种虚拟人的驱动方法，其中，包括：
    根据采集的第一人体图像，确定人体关节的第一旋转矩阵；
    对所述第一旋转矩阵添加约束矩阵，确定所述人体关节的第二旋转矩阵，所述约束矩阵对应于所述人体关节的物理约束条件；
    对所述第二旋转矩阵进行平滑处理，确定人体关节的目标旋转矩阵；
    根据所述目标旋转矩阵驱动所述虚拟人动作。
  2. 根据权利要求1所述的虚拟人的驱动方法，其中，所述对所述第一旋转矩阵添加约束矩阵，确定所述人体关节的第二旋转矩阵，具体包括：
    以所述人体关节的子关节的所述第一旋转矩阵作为所述约束矩阵；
    将所述约束矩阵补偿给所述人体关节的第一旋转矩阵，确定所述第二旋转矩阵。
  3. 根据权利要求2所述的虚拟人的驱动方法，其中，不同的所述子关节的所述第一旋转矩阵不相同。
  4. 根据权利要求1至3中任一项所述的虚拟人的驱动方法，其中，所述对所述第二旋转矩阵进行平滑处理，确定人体关节的目标旋转矩阵，具体包括：
    根据所述第二旋转矩阵确定所述人体关节动作的第一加速度和所述人体关节受到的第一外力；
    根据所述第一加速度和所述第一外力确定所述人体关节的第一位置；
    根据所述第一位置确定所述目标旋转矩阵。
  5. 根据权利要求4所述的虚拟人的驱动方法，其中，所述根据所述第二旋转矩阵确定所述人体关节动作的第一加速度和所述人体关节受到的第一外力，具体包括：
    根据所述第二旋转矩阵确定所述人体关节的第二位置；
    获取第二人体图像和第三人体图像，以及所述第二人体图像和所述第三人体图像的采集时刻信息；
    确定所述第二人体图像中所述人体关节的第一历史位置和所述第三人体图像中所述人体关节的位置第二历史位置；
    根据所述第一历史位置、所述第二历史位置、所述第二位置和所述采集时刻信息，确定所述人体关节动作的第一速度和第一加速度；
    根据所述第二位置、所述第一速度和所述第一加速度确定所述第一外力；
    其中，所述第二人体图像和所述第三人体图像为在所述第一人体图像之前采集的图像。
  6. 根据权利要求5所述的虚拟人的驱动方法，其中，所述根据所述第一历史位置、所述第二历史位置、所述第二位置和所述采集时刻信息，确定所述人体关节动作的第一速度和第一加速度，具体包括：
    根据所述第二历史位置、所述第二位置和所述采集时刻信息确定所述第一速度；
    根据所述第一历史位置、所述第二历史位置和所述采集时刻信息确定第二速度；
    根据所述第一速度和所述第二速度的差值确定所述第一加速度。
  7. 根据权利要求5所述的虚拟人的驱动方法，其中，所述根据所述第二位置、所述第一速度和所述第一加速度确定所述第一外力，具体包括：
    通过刚体运动方程，基于摩擦锥约束条件根据所述第二位置、所述第一速度和所述第一加速度确定所述第一外力。
  8. 根据权利要求5至7中任一项所述的虚拟人的驱动方法，其中，所述根据所述第一加速度和所述第一外力确定所述人体关节的第一位置，具体包括：
    通过迭代算法，根据所述第一加速度和所述第一外力求解所述第一位置。
  9. 一种虚拟人的驱动装置，其中，包括：
    第一处理模块，用于根据采集的第一人体图像，确定人体关节的第一旋转矩阵；
    第二处理模块，用于对所述第一旋转矩阵添加约束矩阵，确定所述人体关节的第二旋转矩阵，所述约束矩阵对应于所述人体关节的物理约束条件；
    第三处理模块，用于对所述第二旋转矩阵进行平滑处理，确定人体关节的目标旋转矩阵；
    所述第三处理模块还用于根据所述目标旋转矩阵驱动所述虚拟人动作。
  10. 一种虚拟人的驱动装置，其中，包括：存储器和处理器，所述存储器存储有程序，所述处理器执行所述程序时实现如权利要求1至8中任一项所述的虚拟人的驱动方法的步骤。
  11. 一种可读存储介质，其中，所述可读存储介质上存储程序或指令，所述程序或指令被处理器执行时实现如权利要求1至8中任一项所述的虚拟人的驱动方法的步骤。
  12. 一种电子设备，其中，包括：
    如权利要求9或10所述的虚拟人的驱动装置；和/或
    如权利要求11所述的可读存储介质。
  13. 一种计算机程序产品，其中，包括计算机程序，所述计算机程序被处理器被执行时实现如权利要求1至8中任一项所述的虚拟人的驱动方法的步骤。
PCT/CN2023/077997 2022-10-13 2023-02-23 虚拟人的驱动方法、装置、存储介质和电子设备 WO2024077837A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202211254965.4A CN115564877A (zh) 2022-10-13 2022-10-13 虚拟人的驱动方法、装置、存储介质和电子设备
CN202211254965.4 2022-10-13

Publications (1)

Publication Number Publication Date
WO2024077837A1 true WO2024077837A1 (zh) 2024-04-18

Family

ID=84744981

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2023/077997 WO2024077837A1 (zh) 2022-10-13 2023-02-23 虚拟人的驱动方法、装置、存储介质和电子设备

Country Status (2)

Country Link
CN (1) CN115564877A (zh)
WO (1) WO2024077837A1 (zh)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115564877A (zh) * 2022-10-13 2023-01-03 美的集团(上海)有限公司 虚拟人的驱动方法、装置、存储介质和电子设备

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110267358A1 (en) * 2010-04-29 2011-11-03 Rennuit Antoine Felix Robert Animating a virtual object within a virtual world
CN111028346A (zh) * 2019-12-23 2020-04-17 北京奇艺世纪科技有限公司 一种视频对象的重建方法和装置
US20220028260A1 (en) * 2020-12-30 2022-01-27 Apollo Intelligent Connectivity (Beijing) Technology Co., Ltd. Method for acquiring three-dimensional perception information based on external parameters of roadside camera, and roadside device
CN115018959A (zh) * 2022-05-13 2022-09-06 阿里巴巴（中国）有限公司 三维虚拟模型的驱动处理方法、装置、设备和存储介质
CN115564877A (zh) * 2022-10-13 2023-01-03 美的集团(上海)有限公司 虚拟人的驱动方法、装置、存储介质和电子设备

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110267358A1 (en) * 2010-04-29 2011-11-03 Rennuit Antoine Felix Robert Animating a virtual object within a virtual world
CN111028346A (zh) * 2019-12-23 2020-04-17 北京奇艺世纪科技有限公司 一种视频对象的重建方法和装置
US20220028260A1 (en) * 2020-12-30 2022-01-27 Apollo Intelligent Connectivity (Beijing) Technology Co., Ltd. Method for acquiring three-dimensional perception information based on external parameters of roadside camera, and roadside device
CN115018959A (zh) * 2022-05-13 2022-09-06 阿里巴巴（中国）有限公司 三维虚拟模型的驱动处理方法、装置、设备和存储介质
CN115564877A (zh) * 2022-10-13 2023-01-03 美的集团(上海)有限公司 虚拟人的驱动方法、装置、存储介质和电子设备

Also Published As

Publication number Publication date
CN115564877A (zh) 2023-01-03

Similar Documents

Publication Publication Date Title
US11583348B2 (en) Medical robot arm apparatus, medical robot arm control system, medical robot arm control method, and program
US10668625B2 (en) Robot arm apparatus, robot arm apparatus control method, and program
US10505428B2 (en) Actuator and robot arm apparatus
US20210361381A1 (en) Medical supporting arm control apparatus, medical supporting arm apparatus control method, and medical system
EP3135444B1 (en) Robot arm apparatus, robot arm control method, and program
JP6555248B2 (ja) 医療用アーム装置、キャリブレーション方法及びプログラム
WO2024077837A1 (zh) 虚拟人的驱动方法、装置、存储介质和电子设备
JP5974668B2 (ja) マニピュレーションシステム
JP6858750B2 (ja) 医療用観察装置、駆動制御方法、医療用観察システム及び支持アーム装置
CN106945043A (zh) 一种主从式遥操作手术机器人多臂协同控制系统
US11950970B2 (en) Medical support arm system, medical support arm control method, and medical support arm control device
CN112891137A (zh) 上肢康复机器人系统、机器人控制方法及装置
CN110539315B (zh) 基于虚拟现实控制的建筑机器人
JP2017134775A (ja) 画像処理装置、画像処理方法及びプログラム
CN110333722A (zh) 一种机器人轨迹生成和控制方法、装置及系统
CN115841864B (zh) 一种康复运动质量评估方法及系统
US20220386981A1 (en) Information processing system and information processing method
CN117085301A (zh) 基于虚拟现实的康复机器人多反馈康复训练方法及系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23876051

Country of ref document: EP

Kind code of ref document: A1