CN116305994A

CN116305994A - Simulation data generation method and device of inertial measurement equipment, medium and equipment

Info

Publication number: CN116305994A
Application number: CN202310316498.1A
Authority: CN
Inventors: 王雄; 胡伟峰; 裴璇
Original assignee: Shanghai Jinsheng Communication Technology Co ltd
Current assignee: Shanghai Jinsheng Communication Technology Co ltd
Priority date: 2023-03-28
Filing date: 2023-03-28
Publication date: 2023-06-23

Abstract

The disclosure relates to the field of simulation technologies, and in particular relates to a simulation data generation method and device of an inertial measurement device, a computer readable storage medium and an electronic device. The method comprises the following steps: obtaining modeling parameters of an object motion model, wherein the modeling parameters comprise object motion information of a preset frame number; based on forward dynamics, determining motion information of grid patch vertexes of an object motion model in a motion process according to object motion information of a preset frame number; determining a target grid patch vertex corresponding to the virtual inertia measurement equipment according to the nearest grid patch vertex of the virtual inertia measurement equipment placed in the object motion model; and determining simulation motion data acquired by the virtual inertia measurement equipment based on the motion information of the vertex of the target grid patch in the motion process. The method and the device can improve the data acquisition efficiency of the inertial measurement equipment and the accuracy of the generated simulation data.

Description

Simulation data generation method and device of inertial measurement equipment, medium and equipment

Technical Field

The disclosure relates to the field of simulation technologies, and in particular relates to a simulation data generation method of an inertial measurement device, a simulation data generation device of the inertial measurement device, a computer readable storage medium and an electronic device.

Background

The object motion capture technology continuously acquires object motion information through an IMU (Inertial Measurement Unit ) device placed at a specific position of an object, thereby reconstructing an object motion pose.

In order to save the cost of data acquisition, a simulation modeling mode can be adopted, namely, IMU simulation data are acquired by placing a virtual IMU sensor in a virtual object model, so that object action gesture reconstruction is performed based on the IMU simulation data.

However, in the related art, IMU simulation motion data generated through simulation modeling is not accurate enough.

It should be noted that the information disclosed in the above background section is only for enhancing understanding of the background of the present disclosure and thus may include information that does not constitute prior art known to those of ordinary skill in the art.

Disclosure of Invention

An object of the present disclosure is to provide a simulation data generating method of an inertial measurement apparatus, a simulation data generating device of an inertial measurement apparatus, a computer-readable storage medium, and an electronic apparatus, thereby improving accuracy of generated simulation data of an inertial measurement apparatus at least to some extent.

According to a first aspect of the present disclosure, there is provided a simulation data generation method of an inertial measurement apparatus, including: obtaining modeling parameters of an object motion model, wherein the modeling parameters comprise object motion information of a preset frame number; based on forward dynamics, determining motion information of grid patch vertexes of the object motion model in a motion process according to the object motion information of the preset frame number; determining a target grid patch vertex corresponding to the virtual inertia measurement equipment according to the grid patch vertex closest to the virtual inertia measurement equipment placed in the object motion model; and determining simulation motion data acquired by the virtual inertia measurement equipment based on the motion information of the vertex of the target grid patch in the motion process.

According to a second aspect of the present disclosure, there is provided a simulation data generating apparatus of an inertial measurement device, comprising: a modeling parameter acquisition module configured to acquire modeling parameters of an object motion model, the modeling parameters including object motion information of a preset frame number; the vertex motion information determining module is configured to determine motion information of grid surface patch vertices of the object motion model in a motion process according to the object motion information of the preset frame number based on forward dynamics; the target grid patch vertex determining module is configured to determine a target grid patch vertex corresponding to the virtual inertia measurement equipment according to the grid patch vertex closest to the virtual inertia measurement equipment placed in the object motion model; and the simulation motion data generation module is configured to determine the simulation motion data acquired by the virtual inertia measurement equipment based on the motion information of the vertex of the target grid patch in the motion process.

According to a third aspect of the present disclosure, there is provided a computer-readable storage medium having stored thereon a computer program which, when executed by a processor, implements the method of generating simulation data of an inertial measurement device according to the first aspect described above.

According to a fourth aspect of the present disclosure, there is provided an electronic apparatus, comprising: one or more processors; and a memory for storing one or more programs which, when executed by the one or more processors, cause the one or more processors to implement the method for generating simulation data of an inertial measurement device according to the first aspect described above.

According to a fifth aspect of the present disclosure there is provided a computer program product comprising instructions which, when run on a computer, cause the computer to perform the steps of the method of generating simulation data of an inertial measurement device as described in the first aspect.

The technical scheme of the present disclosure has the following beneficial effects:

on one hand, the method simulates various motion postures and actions of the object through the object motion model, wherein the object motion model comprises joint moment and a mechanical system, and the real behavior of the object in a real environment can be more accurately simulated, so that the accuracy of simulation motion data of the generated inertial measurement equipment is improved; on the other hand, the data acquisition efficiency of the inertial measurement device can be improved by placing the virtual inertial measurement device in the object motion model without debugging hardware devices.

It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure.

Drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description, serve to explain the principles of the disclosure. It will be apparent to those of ordinary skill in the art that the drawings in the following description are merely examples of the disclosure and that other drawings may be derived from them without undue effort. In the drawings:

FIG. 1 shows a schematic diagram of one system architecture to which exemplary embodiments of the present disclosure may be applied;

FIG. 2 illustrates a flow diagram of a method of generating simulated data for an inertial measurement device in an exemplary embodiment of the present disclosure;

FIG. 3 illustrates a flow diagram of a method of determining motion information of mesh patch vertices during motion in an exemplary embodiment of the present disclosure;

FIG. 4 illustrates a simulated schematic diagram of simulating output data of a virtual inertial measurement device based on a human motion model in an exemplary embodiment of the disclosure;

FIG. 5 illustrates a flow diagram of a method of determining simulated motion data acquired by a virtual inertial measurement device in an exemplary embodiment of the present disclosure;

FIG. 6 illustrates a flow diagram of a method of determining simulated angular velocity data acquired by a virtual inertial measurement device in an exemplary embodiment of the present disclosure;

FIG. 7 illustrates a flow diagram of another method of determining simulated motion data acquired by a virtual inertial measurement device in an exemplary embodiment of the present disclosure;

FIG. 8 illustrates a schematic diagram of a system for generating simulated motion data of an inertial measurement device in an exemplary embodiment of the present disclosure;

FIG. 9 shows a schematic diagram of the composition of a simulation data generating apparatus of an inertial measurement device in an exemplary embodiment of the present disclosure;

fig. 10 shows a schematic diagram of an electronic device to which exemplary embodiments of the present disclosure may be applied.

Detailed Description

Example embodiments will now be described more fully with reference to the accompanying drawings. However, the exemplary embodiments may be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the concept of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

Furthermore, the drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and thus a repetitive description thereof will be omitted. Some of the block diagrams shown in the figures are functional entities and do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software or in one or more hardware modules or integrated circuits or in different networks and/or processor devices and/or microcontroller devices.

The object motion capture technology is widely used in the fields of movie production, virtual reality, augmented reality, and the like. Taking a human body as an example, the inertial motion capture system continuously collects human body motion information by wearing a plurality of IMU devices at specific positions of the human body, so as to reconstruct human body motion gestures based on the collected human body motion information.

An IMU includes 3 single-axis accelerometers and 3 single-axis gyroscopes, the accelerometers detect acceleration signals of the object in the carrier coordinate system on three independent axes, the gyroscopes detect angular velocity signals of the carrier relative to the navigation coordinate system, angular velocity and acceleration of the object in three-dimensional space are measured, and the attitude of the object is calculated according to the angular velocity and acceleration signals. Current human body pose reconstruction techniques employ data-driven based deep learning schemes that require model training on a large scale human body pose dataset.

In consideration of the cost of data acquisition, IMU data can be generated in a simulation modeling mode, namely, virtual IMU sensor equipment is placed in a virtual human body model, so that IMU simulation motion data which can be used for reconstructing human body motion gestures in a large scale are obtained.

In the related technology, based on a specific linear mixed skin human body motion skeleton model, acceleration information at a specific position is modeled through second-order differential simulation, and a rotation matrix of the specific position relative to a root node is calculated through forward dynamics, so that an output signal of a gyroscope is simulated.

However, this approach relies on a specific linear hybrid skinned human motion skeletal model, is limited by the fixed limits of model selection, and the data from simulation modeling is prone to homogeneous motion trends associated with the specific model selection, resulting in model overfitting based on the data training.

In another deep learning method based on video streaming, video is taken as input, and a trained neural network model is used for directly outputting output signals of an IMU sensor, so that simulation data of the IMU are obtained.

However, this approach fails to evaluate the accuracy of the output IMU signals due to the "black box" nature of the neural network model, resulting in insufficient accuracy of the generated IMU simulation data.

Meanwhile, when IMU simulation data are generated, the inherent measurement process noise of the IMU is not considered, the output simulation data have the problem of generalized idealization, and the robustness of the model trained based on the data is insufficient.

In view of one or more of the problems described above, exemplary embodiments of the present disclosure provide a simulation data generating method of an inertial measurement device.

Next, a system architecture of the operating environment of the present exemplary embodiment will be described with reference to fig. 1.

Fig. 1 shows a schematic diagram of a system architecture, which system architecture 100 may include a terminal 110 and a server 120. The terminal 110 may be a terminal device such as a smart phone, an intelligent wearable electronic device (e.g. a smart watch), a tablet computer, a desktop computer, a notebook computer, etc., and the server 120 generally refers to a background system that provides services related to the simulation data generating method of the inertial measurement device in the present exemplary embodiment, and may be a server or a cluster formed by multiple servers. The terminal 110 and the server 120 may form a connection through a wired or wireless communication link for data interaction.

In an exemplary embodiment, the above-described method may be performed by the server 120. For example, the server 120 acquires modeling parameters of the object motion model input by the user, and then determines motion information of the model vertices in the motion process according to the modeling parameters based on forward dynamics, and further determines simulation motion data acquired by a virtual inertial measurement device placed in the object motion model according to the motion information of the vertices in the motion process.

In another exemplary embodiment, the above-described method may also be performed by the terminal 110. For example, a user inputs modeling parameters of an object motion model in the terminal 110, and a processor in the terminal 110 determines motion information of grid patch vertices of the object motion model in a motion process according to object motion information in the modeling parameters based on forward dynamics, and further determines simulation motion data collected by a virtual inertial measurement device placed in the object motion model based on the motion information of target grid patch vertices in the motion process.

In yet another exemplary embodiment, the above-described method may also be performed by the terminal 110 and the server 120 together. For example, the user inputs modeling parameters of the object motion model in the terminal 110, the terminal 110 may send the modeling parameters of the object motion model input by the user to the server 120, and the server 120 determines motion information of grid patch vertices of the object motion model in a motion process based on forward dynamics according to object motion information in the motion modeling parameters, so as to determine simulation motion data collected by a virtual inertial measurement device placed in the object motion model based on the motion information of the target grid patch vertices in the motion process.

As can be seen from the above, the execution subject of the simulation data generation method of the inertial measurement apparatus in the exemplary embodiments of the present disclosure may be the terminal 110 and/or the server 120 described above, which is not limited by the present disclosure.

A flowchart of a simulation data generation method of an inertial measurement apparatus in the present exemplary embodiment will be described below with reference to fig. 2, and fig. 2 shows an exemplary flowchart of a simulation data generation method of an inertial measurement apparatus, which may include:

step S210, obtaining modeling parameters of an object motion model, wherein the modeling parameters comprise object motion information of a preset frame number;

step S220, determining the motion information of the grid patch vertexes of the object motion model in the motion process according to the object motion information of the preset frame number based on forward dynamics;

step S230, determining a target grid patch vertex corresponding to the virtual inertia measurement equipment according to the nearest grid patch vertex from the virtual inertia measurement equipment placed in the object motion model;

step S240, determining simulation motion data acquired by the virtual inertial measurement equipment based on the motion information of the vertex of the target grid patch in the motion process.

Based on the method, on one hand, the method simulates various motion postures and actions of the object through the object motion model, wherein the object motion model comprises joint moment and a mechanical system, and the real behavior of the object in a real environment can be more accurately simulated, so that the accuracy of simulation motion data of the generated inertial measurement equipment is improved; on the other hand, the data acquisition efficiency of the inertial measurement device can be improved by placing the virtual inertial measurement device in the object motion model without debugging hardware devices.

Each step in fig. 2 is specifically described below.

In step S210, modeling parameters of the object motion model are acquired.

In an exemplary embodiment, the object motion model may include one or more of a human motion model, an animal motion model. In other words, the method in the disclosure may be used for acquiring IMU simulation data of a human body based on a human body motion model, so as to perform human body posture reconstruction, or may be used for acquiring IMU simulation data of an animal body based on other animal body motion models, such as animal body motion models of cats, dogs, etc., so as to perform animal body posture reconstruction.

For example, in the present disclosure, an object motion model may be randomly determined from a plurality of candidate object motion models. For example, each object may correspond to a plurality of candidate object motion models, and each time the object is parametrically modeled, one candidate object motion model may be randomly selected from the plurality of candidate object motion models to be used as the object motion model for modeling, so as to avoid a homogeneous motion trend caused by model selection.

Taking the human motion model as an example, candidate human motion models may include an SMPL (a bare Multi-Person Linear Model, vertex-based three-dimensional model of the human body), a STAR (modified SMPL model), an SMPL-H (an SMPL model that adds hand joints), an SMPL-X (an SMPL model that adds hand joints and face joints), and the like.

Of course, the user may input the object motion model to be selected according to the user's requirement. The present exemplary embodiment is not particularly limited thereto.

In an exemplary embodiment, the modeling parameters of the object motion model include object motion information of a preset number of frames, wherein the object motion information of the preset number of frames may include one or more of global displacement information of the preset number of frames, global rotation angle information of the articulation point.

In another exemplary embodiment, the modeling parameters of the object motion model may further include body shape information and sex information of the object.

For example, the modeling parameters of the object motion model may mainly include position information (i.e., global displacement information) of each node of the object in each motion frame of a preset frame number, a relative rotation angle (i.e., global rotation angle information) of each node of the object model with respect to an axial angular representation of its parent node in each motion frame of the preset frame number, body shape information (e.g., height, width, head circumference, leg length, etc.) of the object, sex information of the object, and the like.

The displacement information includes 3 displacement components of the object in three coordinate axes of the world coordinate system, taking a preset frame number as an example, the tensor size of the displacement information is f×3, the relative rotation angle includes 3 rotation angle components of each node relative to the parent node in three coordinate axes of the world coordinate system, taking the number of the nodes of the object model as J, the preset frame number as an example, and the tensor size of the relative rotation angle is f×j×3.

For example, the source of the modeling parameter value may be a data set that the existing right-of-delivery party agrees to permit to use, a specific motion data set collected according to the service requirement, a data set derived by animation modeling, or data input by user customization, which is not particularly limited in this exemplary embodiment.

It should be noted that, for different object motion models, random numbers may be used to fill in when there are default items in the input data of the modeling parameters.

Next, in step S220, motion information of the mesh patch vertices of the object motion model in the motion process is determined according to the object motion information of the preset frame number based on forward dynamics.

For example, one specific embodiment of step S220 may include: and under the condition that the object motion information of the preset frame number comprises the global displacement information, determining the position information of the grid patch vertexes of the object motion model in the motion process based on forward dynamics according to the global displacement information.

By way of example, another embodiment of step S220 may include: and under the condition that the object motion information of the preset frame number comprises the global rotation angle information of the joint point, determining the rotation angle information of the grid patch vertex of the object motion model in the motion process based on forward dynamics according to the global rotation angle information of the joint point.

In the case where modeling parameters include body shape information and gender information, fig. 3 shows a flowchart of a method of determining motion information of mesh patch vertices during motion in an exemplary embodiment of the present disclosure. Referring to fig. 3, the method may include steps S310 to S320.

In step S310, a base template of the object motion model is generated according to the body shape information and the sex information.

Taking the above SMPL model as an example, a base template of the human motion model may be generated based on the body shape information and the sex information, and the base template may be understood as a posture of the human model when in the reference posture. The SMPL model takes the mechanical characteristics of different muscle groups and the kinematic relationship between different muscle tissues and the SMPL joint into consideration to construct a fine human body movement model.

In step S320, motion information of the mesh patch vertices of the object motion model in the motion process is determined according to the object motion information of the preset frame number based on the base template and the forward dynamics.

For example, after the base template is generated, forward dynamics may generate a series of motion matrices of the object during the motion according to the object motion information, and apply the motion matrices to the base template to change the position and posture of the human motion model, so that the human motion model moves.

Taking the example that the object motion information includes the global displacement information and the global rotation angle information of the joint point, forward dynamics can generate a position transformation matrix sequence according to the global displacement information, and apply the position transformation matrix sequence to the base template to change the position of the human motion model. The forward dynamics can also generate a rotation angle matrix sequence according to the global displacement information and the global rotation angle information of the joint points, and apply the rotation angle matrix sequence to the base template to change the rotation angle of the human body motion model, so that the human body motion model moves according to the changed position and rotation angle. The position information of each grid patch vertex forming the human body motion model in each motion frame can be derived according to the position change matrix, and the rotation angle information of each grid patch vertex forming the human body motion model in each motion frame can be derived according to the rotation angle matrix.

Through the steps S310 to S320, for the motion information of the same preset frame number, the motion information of the mesh surface patch vertices forming the human motion model generated finally in the motion process is different due to the difference of the base templates, so that the richness and accuracy of the generated simulation motion data can be improved.

Next, with continued reference to fig. 2, in step S230, a target mesh patch vertex corresponding to the virtual inertial measurement device is determined according to the mesh patch vertex closest to the virtual inertial measurement device placed in the object motion model.

For example, a plurality of virtual inertial measurement devices, i.e. simulated IMU devices, may be placed in the human motion model according to requirements to generate simulated motion data of the IMU, and as shown in fig. 4, 6 simulated IMU devices are placed at the knee, ankle, and wrist of the human motion model. For any virtual IMU device, a grid patch vertex near the virtual IMU device in grid patch vertices of the human body three-dimensional model can be used as a reference vertex, and simulation motion data acquired by the virtual IMU device can be determined according to the position information and the rotation angle information of the reference vertex.

In other words, for any virtual inertial measurement device simulated in the body motion model, the euclidean distance between each mesh patch vertex and the virtual inertial measurement device can be calculated, and the mesh patch vertex with the shortest euclidean distance is determined to be the target mesh patch vertex corresponding to the virtual inertial measurement device, namely the reference vertex corresponding to the virtual inertial measurement device.

In step S240, based on the motion information of the vertex of the target mesh patch in the motion process, the simulated motion data collected by the virtual inertial measurement device is determined.

For example, one specific embodiment of step S240 may include: and under the condition that the object motion information of the preset frame number comprises the global displacement information, determining simulation acceleration data acquired by the virtual inertia measurement equipment according to the position information of the vertex of the target grid surface patch in the motion process.

For example, another embodiment of step S240 may include: and under the condition that the object motion information of the preset frame number comprises the global rotation angle information of the joint point, determining the simulation angular velocity data acquired by the virtual inertia measurement equipment according to the rotation angle information of the vertex of the target grid surface patch in the motion process.

For example, the simulated acceleration data collected by the virtual inertial measurement device may be determined based on the position information of the vertex of the target mesh patch during the motion process, and the simulated angular velocity data collected by the virtual inertial measurement device may be determined based on the rotation angle information of the vertex of the target mesh patch during the motion process.

In an exemplary embodiment, the simulated motion velocity data collected by the virtual inertial measurement device may be determined according to a fourth-order center difference of a first derivative of motion information of the target mesh patch vertex in a motion process with respect to time, and the simulated motion acceleration data collected by the virtual inertial measurement chamber may be determined according to a fourth-order center difference of a second derivative of motion information of the target mesh patch vertex in a motion process with respect to time.

By way of example, FIG. 5 illustrates a flow chart of a method of determining simulated motion data collected by a virtual inertial measurement device in an exemplary embodiment of the present disclosure. Referring to fig. 5, the method may include steps S510 to S520.

In step S510, in the case that the object motion information of the preset frame number includes the global displacement information, the simulated acceleration data collected by the virtual inertial measurement device is determined based on the fourth-order center difference of the second derivative of the position information of the vertex of the target mesh patch with respect to time in the motion process.

As previously described, an accelerometer in the IMU device may detect acceleration data. In the present disclosure, the fourth order center difference may be used as an approximation of the second derivative of displacement with respect to time (i.e., acceleration in a physical sense), resulting in simulated acceleration data acquired by the IMU device.

Taking a human body motion model as an example, the acceleration of an ith IMU device placed in the human body motion model at the time t can be determined by the following formula (1):

equation (1) represents the fourth-order center difference of the second derivative of the position information of the ith IMU device at time t with respect to time. Wherein h represents step length, the value of h can be 1 or other values, a _i (t-2 h) represents position information of the vertex of the target mesh patch corresponding to the ith IMU equipment at the time t-2h, a _i (t-h) represents the position information of the vertex of the target grid patch corresponding to the ith IMU equipment at the time t-h, a _i (t) represents the position information of the vertex of the target mesh patch corresponding to the ith IMU equipment at the moment t, a _i And (t+2h) represents the position information of the vertex of the target mesh patch corresponding to the ith IMU equipment at the time t+2h.

In the formula (1), t may also represent the t frame, h represents the frame interval, for example, when h takes on a value of 1, t-2h represents 2 frames before the t frame. Meanwhile, when the position information of the vertex of the target mesh patch at the moment required by the formula (1) does not exist, the vertex is represented by an all-0 matrix, namely, the truncated term in the formula (1) is zero-filled, such as a when t is the 1 st frame _i (t-2 h) is absent, a _o (t-2 h) may be assigned a matrix of all 0's.

The position information may be determined by the position change matrix obtained based on forward dynamics in the step S220, where the position change matrix includes the position components of the vertex of the target mesh patch in three coordinate axes under the world coordinate system, so that the simulation acceleration data of the o-th IMU device in the three coordinate axes at the time t may be directly obtained based on the formula (1).

In step S520, in the case that the object motion information of the preset frame number includes global rotation angle information of the joint point, based on a fourth-order center difference of a first derivative of rotation angle information of the vertex of the target mesh patch with respect to time in the motion process, the simulated angular velocity data acquired by the virtual inertial measurement device is determined.

As previously described, gyroscopes in IMU devices may detect angular velocity data of an object during motion. Based on this, the simulated angular velocity data acquired by the virtual inertial measurement device can be determined based on the fourth-order center difference of the first derivative of the rotation angle information of the target mesh patch vertices with respect to time during motion.

In an exemplary embodiment, the rotation angle information of the mesh vertex of the object motion model during the motion determined in the step S220 may include a rotation angle matrix of the mesh vertex during the motion, where the rotation angle matrix includes rotation angle components of the mesh vertex in each coordinate axis of the world coordinate system.

For example, as described above, based on forward dynamics, a rotation angle matrix of the vertices of the target mesh during the motion process may be obtained, where the rotation angle matrix includes rotation angle components of the vertices of the target mesh in three coordinate axes of the world coordinate system.

For example, one specific embodiment of step S520 may include: and determining simulation angular velocity data acquired by the virtual inertial measurement equipment at the moment t based on a fourth-order center difference of a first derivative of a rotation angle matrix of the vertex of the target grid patch at the moment t with respect to time and a transposition of the rotation angle matrix of the vertex of the target grid patch at the moment t.

For example, the simulated angular velocity of the gyroscope acquisition of the IMU device may be determined by an antisymmetric matrix as follows. Wherein the antisymmetric matrix W may be expressed as:

wherein omega _x Representing the angular velocity component, ω, in the x-axis of the world coordinate system _y Representing the angular velocity component, ω, in the y-axis of the world coordinate system _z Representing the angular velocity component in the z-axis of the world coordinate system.

The above-described antisymmetric matrix W may be determined by the rotation angle matrix a based on the following formula (2):

that is, equation (2) indicates that the antisymmetric matrix W can be determined by the product of the derivative of the rotation angle matrix a and the transpose of the rotation angle matrix a. Therefore, an antisymmetric matrix can be determined by the product of the derivative of the rotation angle matrix A and the transpose of the rotation angle matrix A, and then the rotation angular velocity components of the rotation angles on three coordinate axes are determined by the antisymmetric matrix, so that the simulation angular velocity data acquired by the inertial measurement device are obtained based on the rotation angular velocity components.

For example, similar to the simulated acceleration signal, the fourth-order center difference may be used as an approximation of the first derivative of the rotation angle matrix with respect to time, so, based on the above formula (2), the simulated angular velocity matrix at time t for the ith IMU device placed in the human motion model may be determined by the following formula (3):

in the formula (3), A _i (t-2 h) represents a rotation angle matrix of the vertex of the target grid patch corresponding to the ith IMU equipment at the moment t-2h, A _i (t-h) represents the rotation angle of the vertex of the target grid patch corresponding to the ith IMU equipment at the time t-hMatrix A _i (t+h) represents a rotation angle matrix of the vertex of the target grid patch corresponding to the ith IMU equipment at the time t+h, A _i And (t+2h) represents a rotation angle matrix of the vertex of the target grid patch corresponding to the ith IMU equipment at the moment t+2h. h represents a step size, and may be set to 1 or another value.

In the formula (3), t may also represent the t frame, h represents the frame interval, and when h takes on a value of 1, t-2h represents 2 frames preceding the t frame. Meanwhile, when the rotation angle matrix of the target mesh patch vertex at the moment required by the formula (2) does not exist, the rotation angle matrix is expressed by an all-0 matrix, namely, the truncated term in the formula (1) is zero-filled, if when t is 1 st frame, no frame exists before 1 st frame, then A _i (t-2 h) is absent, then A _i (t-2 h) may be assigned a matrix of all 0's.

The antisymmetric matrix expressed by the angular velocity can be obtained through the formula (3), and the 3-axis simulation angular velocity of the IMU equipment can be obtained through the antisymmetric matrix. From the above representation of the anti-symmetric matrix W, it can be known that row 2 and column 1 of the anti-symmetric matrix W are ω _z Row 1 and column 3 of the antisymmetric matrix W are ω _y Row 3 and column 2 of the antisymmetric matrix W are ω _x 。

In the method, compared with the second-order center difference which only utilizes the front and back frame information, the method utilizes a longer time correlation through the fourth-order center difference, improves the accuracy of the determined simulation motion data, and has smaller calculation amount compared with the second-order center difference method with a smoothing term.

By way of example, FIG. 6 illustrates a flow chart of another method of determining simulated angular velocity data collected by a virtual inertial measurement device in an exemplary embodiment of the present disclosure. Referring to fig. 6, the method may include steps S610 to S650. Wherein:

in step S610, a simulated angular velocity matrix is determined based on a fourth-order center difference of a first derivative of the rotation angle matrix of the target mesh patch vertices with respect to time;

In step S620, it is determined whether the simulated angular velocity matrix is an antisymmetric matrix, if yes, the process goes to step S630, otherwise the process goes to step S640;

in step S630, according to the position of the angular velocity component in the antisymmetric matrix, determining a simulation value of the angular velocity component based on the element of the corresponding position in the simulated angular velocity matrix;

in step S640, according to the average value of the absolute values of the elements at the symmetrical positions of the indicated angular velocity components in the simulated angular velocity matrix, the simulated values of the angular velocity components are correspondingly determined;

in step S650, based on the simulation value of the angular velocity component, simulated angular velocity data acquired by the virtual inertial measurement apparatus is obtained.

For example, due to the numerical accuracy limitation of the matrix representation, the values in the matrix are rounded off if the numerical accuracy is met, resulting in an angular velocity matrix calculated according to equation (3)

May not be an antisymmetric matrix, it may be processed by step S640 described above to obtain a simulation value of the angular velocity component.

For example, the element and the angular velocity component ω at the symmetrical positions of the 1 st row, 2 nd column and 2 nd row, 1 st column in the antisymmetric matrix W can be known from the above-described expression form of the antisymmetric matrix W _z In relation, the elements and angular velocity component ω at the symmetrical positions of the 1 st row, 3 rd column and 3 rd row, 1 st column in the antisymmetric matrix W _y In relation, the elements and angular velocity component ω at the symmetrical positions of the 2 nd row, 3 rd column and 3 rd row, 2 nd column in the antisymmetric matrix W _x And (5) correlation.

In other words, the symmetrical positions of the 1 st row, 2 nd column, and 2 nd row, 1 st column in the antisymmetric matrix W indicate angular velocity components in the z-axis, the symmetrical positions of the 1 st row, 3 rd column, and 3 rd row, 1 st column in the antisymmetric matrix W indicate angular velocity components in the y-axis, and the symmetrical positions of the 2 nd row, 3 rd column, and 3 rd row, 2 nd column in the antisymmetric matrix W indicate angular velocity components in the x-axis.

Therefore, based onMatrix of angular velocity determined by formula (3) and simulated

In the case of a non-anti-symmetric matrix, it can be in +.>

On the basis of (2), the values of the various simulated angular velocity components are determined by the following formulas (4), (5) and (6), respectively:

the angular velocity component in the x-axis can be represented by the simulated angular velocity matrix

An average of the absolute values of the elements of row 2 and column 3 and the absolute values of the elements in row 3 and column 2; the angle component on the y-axis can be represented by the simulated angular velocity matrix +.>

An average of the absolute values of the elements of row 1, column 3 and the absolute values of the elements in row 3, column 1; the angular velocity component in the z-axis can be represented by the simulated angular velocity matrix +. >

An average of the absolute values of the elements in row 1, column 2 and the absolute values of the elements in row 2, column 1.

By way of example, FIG. 7 illustrates a flow chart of a method of determining simulated motion data collected by a virtual inertial measurement device in an exemplary embodiment of the present disclosure. Referring to fig. 7, the method may include steps S710 to S720. Wherein:

in step S710, candidate simulation motion data collected by the virtual inertial measurement device is determined based on the motion information of the vertices of the target mesh patch in the motion process.

For example, the above-described simulated motion data may be determined based on the fourth-order center difference as the candidate simulated motion data.

In step S720, noise adding processing is performed on the candidate simulated motion data, so as to determine simulated motion data acquired by the virtual inertial measurement device.

In an exemplary embodiment, the noise-adding process includes one or more of adding a constant to the candidate simulated motion data, adding a random variable that follows a gaussian distribution to the candidate simulated motion data, and multiplying the candidate simulated motion data by a random matrix having a determinant value less than a preset value.

For example, in the actual acquisition process, the IMU device has an acquisition error, and the data acquisition error of the IMU device mainly comes from 3 parts of noise, scale factors and axis deviation, and for the acceleration signal and the angular velocity signal, the data acquisition error can be represented by the following formulas (7) and (8):

a ^B ＝TAKA(a ^S +bA+3A)(7)

ω ^B ＝T ^G K ^G (ω ^S +b ^G +3 ^G )(8)

in the formula (7) and the formula (8), a represents an accelerometer, G represents a gyroscope, 6 represents a non-orthogonal measurement coordinate system, and B represents an orthogonal reference coordinate system; t represents a transformation matrix of the axis deviation, and K represents a scale factor; a, a ^B Representing the value of the acceleration signal, ω, measured by the accelerometer ^B Representing measured angular velocity of a gyroscope, a ^S 、ω ^s The true values of acceleration and angular velocity are indicated, b indicates zero bias, and 3 indicates white noise. The axis deviation T and the scale factor K are typically given by the manufacturer.

The candidate simulation motion data are simulation motion data of the IMU equipment obtained under ideal conditions, but in the actual acquisition process, the motion information of the vertex coordinates on the object motion model is directly used due to the existence of the intrinsic acquisition error of the IMU equipment, and the error of the motion data acquired by the actual IMU equipment is difficult to simulate, so that the simulation motion data determined by directly using the motion information of the target vertex on the object motion model are not accurate enough. And in consideration of the existence of the inherent acquisition error of the IMU equipment, the candidate simulation motion data can be subjected to noise processing, so that the finally determined simulation motion data is more similar to the real motion data acquired by the real IMU equipment.

For example, for both acceleration data and angular velocity data in the candidate simulated motion data, the following processing may be performed: a constant b may be added to the candidate simulated motion data to simulate the zero offset error, and a random variable 3 compliant with a gaussian distribution may be added to the candidate simulated motion data to simulate the white noise error; for the inherent scale factor K and axis deviation T, the simulation can be performed by multiplying left by a random matrix with small determinant values.

Through the steps from S710 to S720, the finally obtained simulation motion data is more similar to the motion data acquired by the real IMU equipment, and the accuracy of the generated IMU simulation motion data is improved.

Taking the object motion model as a human motion model as an example, fig. 8 illustrates a schematic structure diagram of a system for generating simulation motion data of an inertial measurement device in an exemplary embodiment of the present disclosure, and referring to fig. 8, the system may include a data input unit 810, a human motion model parameterization modeling unit 820, a forward dynamics simulation unit 830, and an IMU simulation data output unit 840.

Wherein the data input unit 810 is used for inputting modeling parameters required for the human motion model. The input modeling parameter data may include the global displacement of the preset number of frames, the relative rotation of the J joint points with respect to the axial angular representation of their parent nodes, sex information, body shape parameter information, and the like.

The human body motion model parametric modeling unit 820 is used for establishing a human body motion model according to the candidate human body motion model selected by the user or the candidate human body motion model randomly selected by the computer and the inputted modeling parameter data, so as to initialize the simulation process. Taking the SMPL model as an example, it extracts input parameters such as shape parameters and sex information of the body, and generates a corresponding base template according to these parameters.

After the base template is generated, the forward dynamics simulation unit 830 reads the input global displacement information and the relative rotation data, and generates position information and relative rotation of the grid patch vertex data of the human body motion model in the motion process. Taking the SMPL model as an example, after the base template is prepared, the forward dynamics simulation unit 830 reads the motion data of global displacement and relative rotation. These data are then passed to a position transformation matrix, which is then used to generate a series of position transformation matrices and apply them to the model, thereby changing the position of the model. In addition, the SMPL model uses an angle transformation matrix that generates a series of rotation matrices based on the global displacement and relative rotation motion data and applies them to the model to change the rotation angle of the model. Finally, the position information and the relative rotation information of the vertexes of each grid surface patch forming the human body motion model can be determined by using the position transformation matrix and the rotation matrix.

The IMU simulation data output unit 840 may simulate the placement positions of one or more IMU sensors in a real scene, and give the relative rotation and the position information sequence at the corresponding positions, thereby obtaining IMU simulation data. Taking the SMPL model as an example, the SMPL model consists of 24 human body nodes, 6890 vertices, 13776 patches. Suppose that N IMU sensor data placed in N places on a human body is desired. The N vertices closest to the N sensor Euclidean distances can be selected from 6890 vertices in the SMPL model as reference points, and the relative rotation and position information sequences of the N reference points can be extracted. And then obtaining the simulation acceleration data and the simulation angular velocity data output by the virtual IMU equipment based on the formula (1) and the formula (3).

Meanwhile, the simulated acceleration data and the simulated angular velocity data obtained based on the above formula (1) and the above formula (3) are output by the IMU device under ideal conditions, and the IMU simulated data output unit 840 may further perform noise processing on the simulated acceleration data and the simulated angular velocity data obtained based on the above formula (1) and the above formula (3) in consideration of an intrinsic acquisition error of the IMU device, so as to obtain final simulated data of the IMU device based on the data after the noise processing.

In one exemplary scenario, IMU simulation data output unit 840, when outputting a simulated acceleration signal of a simulated accelerometer and a simulated angular velocity signal of a simulated gyroscope, may upsample the signals using methods including, but not limited to, spline interpolation, bezier interpolation, etc.; the signal may also be downsampled using methods including, but not limited to, low frequency filtering, resampling, and the like.

In the method, the real behavior of the object in the real environment can be simulated more accurately through the object motion model, so that more accurate IMU simulation data can be obtained. And the IMU data can be collected faster without debugging a hardware sensor, so that the acquisition time and the acquisition cost can be greatly reduced in practice. Meanwhile, the user can conveniently generate customized and scene IMU simulation data for training the neural network model in the business scenes of digital people, virtual people and the like.

In addition, in the method, the homogenization and overcomplete problems in IMU simulation data generation are avoided by randomly selecting an object motion model and a fourth-order center difference technology. Furthermore, the method and the device are based on the intrinsic acquisition error theory of the IMU, and the accuracy of the generated simulation motion data of the IMU equipment is improved by carrying out noise adding processing on the obtained candidate simulation motion data, so that the simulation motion data is more similar to the simulation motion data acquired by the real IMU equipment.

In summary, in the present disclosure, on one hand, according to the present disclosure, various motion postures and actions of an object are simulated by an object motion model, where the object motion model includes a joint moment and a mechanical system, so that a real behavior of the object in a real environment can be more accurately simulated, thereby improving accuracy of simulation motion data of a generated inertial measurement device; on the other hand, the data acquisition efficiency of the inertial measurement device can be improved by placing the virtual inertial measurement device in the object motion model without debugging hardware devices.

It is noted that the above-described figures are merely schematic illustrations of processes involved in a method according to exemplary embodiments of the present disclosure, and are not intended to be limiting. It will be readily appreciated that the processes shown in the above figures do not indicate or limit the temporal order of these processes. In addition, it is also readily understood that these processes may be performed synchronously or asynchronously, for example, among a plurality of modules.

Further, referring to fig. 9, in this exemplary embodiment, a simulation data generating apparatus 900 of an inertial measurement device is further provided, where the simulation data generating apparatus 900 of an inertial measurement device may include a modeling parameter obtaining module 910, a vertex motion information determining module 920, a target mesh patch vertex determining module 930, and a simulation motion data generating module 940. Wherein:

A modeling parameter acquisition module 910 configured to acquire modeling parameters of an object motion model, the modeling parameters including object motion information of a preset frame number;

the vertex motion information determining module 920 is configured to determine motion information of the mesh patch vertices of the object motion model in a motion process according to the object motion information of the preset frame number based on forward dynamics;

a target mesh patch vertex determining module 930 configured to determine a target mesh patch vertex corresponding to a virtual inertial measurement device placed in the object motion model according to a mesh patch vertex nearest to the virtual inertial measurement device;

and the simulation motion data generation module 940 is configured to determine the simulation motion data acquired by the virtual inertial measurement device based on the motion information of the vertex of the target mesh patch in the motion process.

In an exemplary implementation, based on the foregoing examples, the modeling parameters further include body shape information and gender information; based on this, the vertex motion information determination module 920 may be specifically configured to: generating a base template of the object motion model according to the body shape information and the gender information; and determining the motion information of the grid surface patch vertexes of the object motion model in the motion process according to the object motion information of the preset frame number based on the base template and the forward dynamics.

In an exemplary implementation manner, based on the foregoing embodiment, the object motion information of the preset frame number includes one or more of global displacement information and global rotation angle information of the node; based on this, the vertex motion information determination module 920 may be specifically configured to: under the condition that the object motion information of the preset frame number comprises the global displacement information, determining the position information of grid patch vertexes of the object motion model in the motion process according to the global displacement information; under the condition that the object motion information of the preset frame number comprises global rotation angle information of the joint point, determining rotation angle information of grid patch vertexes of the object motion model in the motion process according to the global rotation angle information of the joint point; the simulated motion data generation module 940 may be specifically configured to: under the condition that the object motion information of the preset frame number comprises the global displacement information, determining simulation acceleration data acquired by the virtual inertia measurement equipment according to the position information of the vertex of the target grid patch in the motion process; and under the condition that the object motion information of the preset frame number comprises the global rotation angle information of the joint point, determining the simulation angular velocity data acquired by the virtual inertia measurement equipment according to the rotation angle information of the vertex of the target grid surface patch in the motion process.

In an exemplary implementation, based on the foregoing embodiment, the simulated motion data generation module 940 may be specifically configured to: determining simulation acceleration data acquired by the virtual inertia measurement equipment based on a fourth-order center difference of a second derivative of position information of the vertex of the target grid patch in a motion process with respect to time under the condition that the object motion information of the preset frame number comprises the global displacement information; and under the condition that the object motion information of the preset frame number comprises the global rotation angle information of the joint point, determining the simulation angular velocity data acquired by the virtual inertial measurement equipment based on the fourth-order center difference of the first derivative of the rotation angle information of the vertex of the target grid patch in the motion process with respect to time.

In an exemplary implementation manner, based on the foregoing embodiment, the rotation angle information of the grid patch vertex in the motion process of the object motion model includes a rotation angle matrix of the grid patch vertex in the motion process, where the rotation angle matrix includes rotation angle components of the grid patch vertex in coordinate axes of a world coordinate system; the determining the simulation angular velocity data collected by the virtual inertial measurement device based on the fourth-order center difference of the first derivative of the rotation angle information of the target mesh patch vertex in the motion process with respect to time comprises the following steps: and determining simulation angular velocity data acquired by the virtual inertial measurement equipment at the moment t based on a fourth-order center difference of a first derivative of a rotation angle matrix of the vertex of the target grid patch at the moment t with respect to time and a transposition of the rotation angle matrix of the vertex of the target grid patch at the moment t.

In an exemplary implementation manner, based on the foregoing embodiment, the rotation angle information of the grid patch vertex in the motion process of the object motion model includes a rotation angle matrix of the grid patch vertex in the motion process, where the rotation angle matrix includes rotation angle components of the grid patch vertex in coordinate axes of a world coordinate system; the determining the simulation angular velocity data collected by the virtual inertial measurement device based on the fourth-order center difference of the first derivative of the rotation angle information of the target mesh patch vertex in the motion process with respect to time comprises the following steps: determining a simulation angular velocity matrix based on a fourth-order center difference of a first derivative of the rotation angle matrix of the vertex of the target grid patch with respect to time; under the condition that the simulation angular velocity matrix is an asymmetric matrix, correspondingly determining a simulation value of the angular velocity component according to the average value of absolute values of elements in the simulation angular velocity matrix, which are positioned at symmetrical positions of the indication angular velocity component; and obtaining the simulation angular velocity data acquired by the virtual inertial measurement equipment based on the simulation value of the angular velocity component.

In an exemplary implementation, based on the foregoing embodiment, the simulated motion data generation module 940 may be specifically configured to: determining candidate simulation motion data acquired by the virtual inertia measurement equipment based on the motion information of the vertex of the target grid patch in the motion process; noise adding is carried out on the candidate simulation motion data so as to determine simulation motion data acquired by the virtual inertial measurement equipment; the noise adding process comprises one or more of adding a constant to the candidate simulation motion data, adding a random variable which obeys Gaussian distribution to the candidate simulation motion data, and adding a random matrix with a value of a left-hand determinant of the candidate simulation motion data smaller than a preset value to the candidate simulation motion data.

In an exemplary implementation, based on the foregoing examples, the object motion model includes one or more of a human motion model, an animal motion model.

In an exemplary implementation, based on the foregoing embodiment, the simulation data generating apparatus of the inertial measurement device may further include an object motion model determining module, which may be configured to: the object motion model is randomly determined from a plurality of candidate object motion models.

The specific details of each module in the above apparatus are already described in the method section, and the details that are not disclosed can be referred to the embodiment of the method section, so that they will not be described in detail.

The exemplary embodiments of the present disclosure also provide an electronic device for performing the simulation data generation method of the inertial measurement device described above, which may be the terminal 110 described above. In general, the electronic device may include a processor and a memory for storing executable instructions of the processor, the processor being configured to perform the above-described simulation data generating method of the inertial measurement device via execution of the executable instructions.

The configuration of the electronic device will be exemplarily described below using the mobile terminal 1000 in fig. 10 as an example. It will be appreciated by those skilled in the art that the configuration of fig. 10 can be applied to stationary type devices in addition to components specifically for mobile purposes.

As shown in fig. 10, the mobile terminal 1000 may specifically include: processor 1001, memory 1002, bus 1003, mobile communication module 1004, antenna 1, wireless communication module 1005, antenna 2, display module 1006, camera module 1007, audio module 1008, power module 1009, and sensor module 1010.

The processor 1001 may include one or more processing units, such as: the processor 1001 may include an AP (Application Processor ), modem processor, ISP (Image Signal Processor ), controller, encoder, decoder, DSP (Digital Signal Processor ), baseband processor and/or NPU (Neural-Network Processing Unit, neural network processor), and the like.

The processor 1001 may form a connection with the memory 1002 or other components through a bus 1003.

Memory 1002 may be used to store computer-executable program code that includes instructions. The processor 1001 performs various functional applications and data processing of the mobile terminal 1000 by executing instructions stored in the memory 1002. The memory 1002 may also pre-store a plurality of candidate object motion models.

The communication functions of the mobile terminal 1000 can be implemented by a mobile communication module 1004, an antenna 1, a wireless communication module 1005, an antenna 2, a modem processor, a baseband processor, and the like. The antennas 1 and 2 are used for transmitting and receiving electromagnetic wave signals. The mobile communication module 1004 may provide a 2G, 3G, 4G, 5G, etc. mobile communication solution applied on the mobile terminal 1000. The wireless communication module 1005 may provide wireless communication solutions for wireless local area networks, bluetooth, near field communications, etc. that are employed on the mobile terminal 1000.

The display module 1006 is used to implement display functions, such as displaying a user interface, etc. The camera module 1007 is used to implement a photographing function, such as photographing an image, video, and the like. The audio module 1008 is configured to implement audio functions, such as playing audio, capturing speech, and the like. The power module 1009 is configured to perform power management functions, such as charging a battery, powering a device, monitoring a battery status, and the like. The sensor module 1010 may include a depth sensor 10101, a pressure sensor 10102, a gyro sensor 10103, a barometric sensor 10104, etc. to implement a corresponding sensing function.

Those skilled in the art will appreciate that the various aspects of the present disclosure may be implemented as a system, method, or program product. Accordingly, various aspects of the disclosure may be embodied in the following forms, namely: an entirely hardware embodiment, an entirely software embodiment (including firmware, micro-code, etc.) or an embodiment combining hardware and software aspects may be referred to herein as a "circuit," module "or" system.

Exemplary embodiments of the present disclosure also provide a computer-readable storage medium having stored thereon a program product capable of implementing the method described above in the present specification. In some possible implementations, various aspects of the disclosure may also be implemented in the form of a program product comprising program code for causing a terminal device to carry out the steps according to the various exemplary embodiments of the disclosure described in the "exemplary methods" section of this specification, when the program product is run on the terminal device, e.g. any one or more of the steps of fig. 2, 3 and 5-7 may be carried out.

The computer readable medium shown in the present disclosure may be a computer readable signal medium or a computer readable storage medium or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

In the context of this disclosure, a computer-readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. In the present disclosure, however, the computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, with the computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, fiber optic cable, RF, etc., or any suitable combination of the foregoing.

Furthermore, the program code for carrying out operations of the present disclosure may be written in any combination of one or more programming languages, including an object oriented programming language such as Java, C++ or the like and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computing device, partly on the user's device, as a stand-alone software package, partly on the user's computing device, partly on a remote computing device, or entirely on the remote computing device or server. In the case of remote computing devices, the remote computing device may be connected to the user computing device through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computing device (e.g., connected via the Internet using an Internet service provider).

Other embodiments of the disclosure will be apparent to those skilled in the art from consideration of the specification and practice of the disclosure disclosed herein. This application is intended to cover any adaptations, uses, or adaptations of the disclosure following, in general, the principles of the disclosure and including such departures from the present disclosure as come within known or customary practice within the art to which the disclosure pertains. It is intended that the specification and examples be considered as exemplary only, with a true scope and spirit of the disclosure being indicated by the following claims.

It is to be understood that the present disclosure is not limited to the precise arrangements and instrumentalities shown in the drawings, and that various modifications and changes may be effected without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A simulation data generation method of an inertial measurement device, comprising:

obtaining modeling parameters of an object motion model, wherein the modeling parameters comprise object motion information of a preset frame number;

based on forward dynamics, determining motion information of grid patch vertexes of the object motion model in a motion process according to the object motion information of the preset frame number;

determining a target grid patch vertex corresponding to the virtual inertia measurement equipment according to the grid patch vertex closest to the virtual inertia measurement equipment placed in the object motion model;

and determining simulation motion data acquired by the virtual inertia measurement equipment based on the motion information of the vertex of the target grid patch in the motion process.

2. The simulation data generation method of an inertial measurement device according to claim 1, wherein the modeling parameters further include body shape information and sex information;

The determining, based on forward dynamics, motion information of the mesh patch vertices of the object motion model in the motion process according to the object motion information of the preset frame number, includes:

generating a base template of the object motion model according to the body shape information and the gender information;

and determining the motion information of the grid surface patch vertexes of the object motion model in the motion process according to the object motion information of the preset frame number based on the base template and the forward dynamics.

3. The simulation data generation method of an inertial measurement device according to claim 1 or 2, wherein the object motion information of the preset frame number includes one or more of global displacement information and global rotation angle information of an articulation point;

under the condition that the object motion information of the preset frame number comprises the global displacement information, determining the position information of grid patch vertexes of the object motion model in the motion process according to the global displacement information;

Under the condition that the object motion information of the preset frame number comprises global rotation angle information of the joint point, determining rotation angle information of grid patch vertexes of the object motion model in the motion process according to the global rotation angle information of the joint point;

the determining the simulation motion data collected by the virtual inertial measurement device based on the motion information of the vertex of the target grid patch in the motion process comprises the following steps:

under the condition that the object motion information of the preset frame number comprises the global displacement information, determining simulation acceleration data acquired by the virtual inertia measurement equipment according to the position information of the vertex of the target grid patch in the motion process;

and under the condition that the object motion information of the preset frame number comprises the global rotation angle information of the joint point, determining the simulation angular velocity data acquired by the virtual inertia measurement equipment according to the rotation angle information of the vertex of the target grid surface patch in the motion process.

4. The method for generating simulation data of an inertial measurement device according to claim 3, wherein determining the simulation motion data collected by the virtual inertial measurement device based on the motion information of the vertices of the target mesh patch in the motion process includes:

Determining simulation acceleration data acquired by the virtual inertia measurement equipment based on a fourth-order center difference of a second derivative of position information of the vertex of the target grid patch in a motion process with respect to time under the condition that the object motion information of the preset frame number comprises the global displacement information;

and under the condition that the object motion information of the preset frame number comprises the global rotation angle information of the joint point, determining the simulation angular velocity data acquired by the virtual inertial measurement equipment based on the fourth-order center difference of the first derivative of the rotation angle information of the vertex of the target grid patch in the motion process with respect to time.

5. The method for generating simulation data of an inertial measurement device according to claim 4, wherein the rotation angle information of the grid patch vertices of the object motion model in the motion process includes a rotation angle matrix of the grid patch vertices in the motion process, and the rotation angle matrix includes rotation angle components of the grid patch vertices in coordinate axes of a world coordinate system;

the determining the simulation angular velocity data collected by the virtual inertial measurement device based on the fourth-order center difference of the first derivative of the rotation angle information of the target mesh patch vertex in the motion process with respect to time comprises the following steps:

And determining simulation angular velocity data acquired by the virtual inertial measurement equipment at the moment t based on a fourth-order center difference of a first derivative of a rotation angle matrix of the vertex of the target grid patch at the moment t with respect to time and a transposition of the rotation angle matrix of the vertex of the target grid patch at the moment t.

6. The method for generating simulation data of an inertial measurement device according to claim 4, wherein the rotation angle information of the grid patch vertices of the object motion model in the motion process includes a rotation angle matrix of the grid patch vertices in the motion process, and the rotation angle matrix includes rotation angle components of the grid patch vertices in coordinate axes of a world coordinate system;

determining a simulation angular velocity matrix based on a fourth-order center difference of a first derivative of the rotation angle matrix of the vertex of the target grid patch with respect to time;

under the condition that the simulation angular velocity matrix is an asymmetric matrix, correspondingly determining a simulation value of the angular velocity component according to the average value of absolute values of elements in the simulation angular velocity matrix, which are positioned at symmetrical positions of the indication angular velocity component;

And obtaining the simulation angular velocity data acquired by the virtual inertial measurement equipment based on the simulation value of the angular velocity component.

7. The method for generating simulation data of an inertial measurement device according to claim 1, wherein determining the simulation motion data collected by the virtual inertial measurement device based on the motion information of the vertices of the target mesh patch in the motion process includes:

determining candidate simulation motion data acquired by the virtual inertia measurement equipment based on the motion information of the vertex of the target grid patch in the motion process;

noise adding is carried out on the candidate simulation motion data so as to determine simulation motion data acquired by the virtual inertial measurement equipment;

the noise adding process comprises one or more of adding a constant to the candidate simulation motion data, adding a random variable which obeys Gaussian distribution to the candidate simulation motion data, and adding a random matrix with a value of a left-hand determinant of the candidate simulation motion data smaller than a preset value to the candidate simulation motion data.

8. The method of generating simulation data of an inertial measurement device according to claim 1, wherein the object motion model comprises one or more of a human body motion model and an animal body motion model.

9. The method of generating simulation data of an inertial measurement device of claim 1, further comprising:

the object motion model is randomly determined from a plurality of candidate object motion models.

10. An simulation data generating apparatus of an inertial measurement device, comprising:

a modeling parameter acquisition module configured to acquire modeling parameters of an object motion model, the modeling parameters including object motion information of a preset frame number;

the vertex motion information determining module is configured to determine motion information of grid surface patch vertices of the object motion model in a motion process according to the object motion information of the preset frame number based on forward dynamics;

the target grid patch vertex determining module is configured to determine a target grid patch vertex corresponding to the virtual inertia measurement equipment according to the grid patch vertex closest to the virtual inertia measurement equipment placed in the object motion model;

and the simulation motion data generation module is configured to determine the simulation motion data acquired by the virtual inertia measurement equipment based on the motion information of the vertex of the target grid patch in the motion process.

11. A computer readable storage medium, on which a computer program is stored, which computer program, when being executed by a processor, implements the method according to any one of claims 1 to 9.

12. An electronic device, comprising:

one or more processors; and

a memory for storing one or more programs that, when executed by the one or more processors, cause the one or more processors to implement the method of any of claims 1-9.