CN115205340A - Data processing method, device, equipment, readable storage medium and program product - Google Patents
Data processing method, device, equipment, readable storage medium and program product Download PDFInfo
- Publication number
- CN115205340A CN115205340A CN202211001275.8A CN202211001275A CN115205340A CN 115205340 A CN115205340 A CN 115205340A CN 202211001275 A CN202211001275 A CN 202211001275A CN 115205340 A CN115205340 A CN 115205340A
- Authority
- CN
- China
- Prior art keywords
- motion data
- data
- rotational motion
- coding network
- processed
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000003672 processing method Methods 0.000 title claims abstract description 19
- 230000033001 locomotion Effects 0.000 claims abstract description 271
- 238000000034 method Methods 0.000 claims abstract description 20
- 230000006870 function Effects 0.000 claims description 77
- 238000004590 computer program Methods 0.000 claims description 22
- 238000006243 chemical reaction Methods 0.000 claims description 12
- 238000010606 normalization Methods 0.000 claims description 10
- 238000010586 diagram Methods 0.000 description 7
- 238000011176 pooling Methods 0.000 description 7
- 230000003238 somatosensory effect Effects 0.000 description 7
- 238000007781 pre-processing Methods 0.000 description 6
- 230000009286 beneficial effect Effects 0.000 description 5
- 238000004364 calculation method Methods 0.000 description 4
- 210000004394 hip joint Anatomy 0.000 description 4
- 230000004913 activation Effects 0.000 description 3
- 238000005516 engineering process Methods 0.000 description 3
- 230000003287 optical effect Effects 0.000 description 3
- 230000008569 process Effects 0.000 description 3
- 230000003993 interaction Effects 0.000 description 2
- 230000003068 static effect Effects 0.000 description 2
- 230000005540 biological transmission Effects 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000013481 data capture Methods 0.000 description 1
- 238000004880 explosion Methods 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 238000011022 operating instruction Methods 0.000 description 1
- 230000002093 peripheral effect Effects 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
- A63F13/50—Controlling the output signals based on the game progress
- A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
- G06N3/00—Computing arrangements based on biological models
- G06N3/02—Neural networks
- G06N3/08—Learning methods
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T13/00—Animation
- G06T13/20—3D [Three Dimensional] animation
- G06T13/40—3D [Three Dimensional] animation of characters, e.g. humans, animals or virtual beings
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T9/00—Image coding
- G06T9/002—Image coding using neural networks
-
- A—HUMAN NECESSITIES
- A63—SPORTS; GAMES; AMUSEMENTS
- A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
- A63F2300/00—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game
- A63F2300/80—Features of games using an electronically generated display having two or more dimensions, e.g. on a television screen, showing representations related to the game specially adapted for executing a specific type of game
- A63F2300/8082—Virtual reality
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Artificial Intelligence (AREA)
- Evolutionary Computation (AREA)
- General Engineering & Computer Science (AREA)
- Software Systems (AREA)
- Computer Hardware Design (AREA)
- Computational Linguistics (AREA)
- Computer Graphics (AREA)
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Biomedical Technology (AREA)
- Biophysics (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Data Mining & Analysis (AREA)
- General Health & Medical Sciences (AREA)
- Molecular Biology (AREA)
- Computing Systems (AREA)
- Mathematical Physics (AREA)
- Image Analysis (AREA)
Abstract
The application provides a data processing method, a device, equipment, a readable storage medium and a program product, wherein the method is executed by a body sensing device and comprises the following steps: acquiring first rotational motion data of a target object; inputting the first rotational motion data into a trained convolutional self-coding network, and performing data processing to obtain processed first rotational motion data; the data precision of the processed first rotational motion data is greater than that of the first rotational motion data; driving a role model corresponding to the target object through the processed first rotation motion data; so, the first rotary motion data that body sensing equipment gathered, low accuracy rotary motion data promptly, through the first rotary motion data of convolution self-encoding network generation after handling, high accuracy rotary motion data promptly, body sensing equipment is used for driving role model with high accuracy rotary motion data to the accuracy of role model motion has been promoted.
Description
Technical Field
The present application relates to the field of computer technologies, and in particular, to a data processing method, apparatus, device, readable storage medium, and program product.
Background
With the popularity of virtual reality environments, popular, low-cost motion sensing devices are widely used to drive character models in virtual worlds. The motion sensing device is a machine connected to the game host, and can receive actions or voice information of a player through the sensor, so that conversion of games can be completed; the player does not depend on a handle or a remote controller and other equipment, and the player interacts with a television game and the like through the motion sensing equipment. However, the rotational motion data acquired by the motion sensing equipment is low in precision and is not suitable for directly driving the character model. If the character model is directly driven by adopting low-precision rotational motion data, the motion of the character model is unnatural, namely the motion precision of the character model is not high.
Disclosure of Invention
The present application provides a data processing method, an apparatus, a device, a computer-readable storage medium, and a computer program product, which are used to solve the problem of how to improve the accuracy of the movement of a character model, aiming at the disadvantages of the existing methods.
In a first aspect, the present application provides a data processing method, executed by a motion sensing device, including:
acquiring first rotational motion data of a target object;
inputting the first rotational motion data into a trained convolutional self-coding network, and performing data processing to obtain processed first rotational motion data; the data precision of the processed first rotational motion data is greater than that of the first rotational motion data;
and driving the role model corresponding to the target object through the processed first rotation motion data.
In one embodiment, acquiring first rotational motion data of a target object comprises:
acquiring original rotation motion data of a target object;
and carrying out normalization and relative coordinate conversion on the original rotational motion data to obtain first rotational motion data of the target object.
In one embodiment, before acquiring the first rotational motion data of the target object, the method further comprises:
obtaining second rotational motion data of the sample object;
inputting the second rotational motion data of the sample object into a convolution self-coding network, and performing data processing to obtain processed second rotational motion data;
determining a value of a loss function of the convolutional self-coding network based on the processed second rotational motion data and preset tag data; the value of the loss function is used for representing the error between the processed second rotational motion data and the preset label data;
if the value of the loss function is larger than the preset loss threshold value, training the convolutional self-coding network, and updating the network parameters of the convolutional self-coding network;
repeatedly executing the steps of obtaining second rotational motion data of the sample object, inputting the second rotational motion data of the sample object into the convolutional self-coding network, performing data processing to obtain processed second rotational motion data, determining the value of a loss function of the convolutional self-coding network based on the processed second rotational motion data and preset label data, and training the convolutional self-coding network if the value of the loss function is larger than a preset loss threshold, updating network parameters of the convolutional self-coding network until the value of the loss function is equal to the preset loss threshold, and obtaining the trained convolutional self-coding network.
In one embodiment, determining a value of a loss function of the convolutional self-coding network based on the processed second rotational motion data and the preset tag data comprises:
determining a mean square error of a joint point in the sample object based on the processed second rotational motion data and preset label data; the data precision of the preset label data is larger than the data precision of the processed second rotation motion data;
and determining the value of the loss function of the convolutional self-coding network based on the mean square error of the joint points in the sample object, the number of the joint points of the sample object and the preset joint point weight.
In one embodiment, determining a value of a loss function of a convolutional self-coding network based on a mean square error of the nodes in the sample object, the number of nodes of the sample object, and a preset node weight comprises:
determining an initial value of a loss function of the convolutional self-coding network based on the mean square error of the joint points in the sample object, the number of the joint points of the sample object and a preset joint point weight;
and determining the value of the loss function of the convolutional self-coding network based on the initial value and a preset smooth L1 function.
In a second aspect, the present application provides a data processing apparatus, which is applied to a motion sensing device, and includes:
the first processing module is used for acquiring first rotation motion data of a target object;
the second processing module is used for inputting the first rotational motion data into the trained convolutional self-coding network and processing the data to obtain processed first rotational motion data; the data precision of the processed first rotational motion data is greater than that of the first rotational motion data;
and the third processing module is used for driving the role model corresponding to the target object through the processed first rotation motion data.
In an embodiment, the first processing module is specifically configured to:
acquiring original rotation motion data of a target object;
and carrying out normalization and relative coordinate conversion on the original rotational motion data to obtain first rotational motion data of the target object.
In one embodiment, the data processing apparatus further comprises a training module for:
obtaining second rotational motion data of the sample object;
inputting the second rotational motion data of the sample object into a convolution self-coding network, and performing data processing to obtain processed second rotational motion data;
determining a value of a loss function of the convolutional self-coding network based on the processed second rotational motion data and preset tag data; the value of the loss function is used for representing the error between the processed second rotational motion data and the preset label data;
if the value of the loss function is larger than the preset loss threshold value, training the convolutional self-coding network, and updating the network parameters of the convolutional self-coding network;
repeatedly executing the steps of obtaining second rotational motion data of the sample object, inputting the second rotational motion data of the sample object into the convolutional self-coding network, performing data processing to obtain processed second rotational motion data, determining the value of a loss function of the convolutional self-coding network based on the processed second rotational motion data and preset label data, training the convolutional self-coding network if the value of the loss function is larger than a preset loss threshold value, updating network parameters of the convolutional self-coding network until the value of the loss function is equal to the preset loss threshold value, and obtaining the trained convolutional self-coding network.
In one embodiment, the training module is specifically configured to:
determining the mean square error of the joint point in the sample object based on the processed second rotational motion data and the preset label data; the data precision of the preset label data is greater than the data precision of the processed second rotation motion data;
and determining the value of the loss function of the convolutional self-coding network based on the mean square error of the joint points in the sample object, the number of the joint points of the sample object and the preset joint point weight.
In one embodiment, the training module is specifically configured to:
determining an initial value of a loss function of the convolutional self-coding network based on the mean square error of the joint points in the sample object, the number of the joint points of the sample object and a preset joint point weight;
and determining the value of the loss function of the convolutional self-coding network based on the initial value and a preset smooth L1 function.
In a third aspect, the present application provides an electronic device, comprising: a processor, a memory, and a bus;
a bus for connecting the processor and the memory;
a memory for storing operating instructions;
and the processor is used for executing the data processing method of the first aspect of the application by calling the operation instruction.
In a fourth aspect, the present application provides a computer-readable storage medium storing a computer program for executing the data processing method of the first aspect of the present application.
In a fifth aspect, the present application provides a computer program product comprising a computer program which, when executed by a processor, performs the steps of the data processing method of the first aspect of the present application.
The technical scheme provided by the embodiment of the application at least has the following beneficial effects:
acquiring first rotational motion data of a target object; inputting the first rotational motion data into a trained convolutional self-coding network, and performing data processing to obtain processed first rotational motion data; the data precision of the processed first rotation motion data is greater than that of the first rotation motion data; driving a role model corresponding to the target object through the processed first rotational motion data; so, the first rotary motion data that body sensing equipment gathered, low accuracy rotary motion data promptly, through the first rotary motion data of convolution self-encoding network generation after handling, high accuracy rotary motion data promptly, body sensing equipment is used for driving role model with high accuracy rotary motion data to the accuracy of role model motion has been promoted.
Drawings
In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings used in the description of the embodiments of the present application will be briefly described below.
FIG. 1 is a block diagram of a data processing system according to an embodiment of the present application;
fig. 2 is a schematic flowchart of a data processing method according to an embodiment of the present application;
FIG. 3 is a schematic diagram of a convolutional self-coding network according to an embodiment of the present application;
fig. 4 is a schematic flowchart of another data processing method according to an embodiment of the present application;
fig. 5 is a schematic structural diagram of a data processing apparatus according to an embodiment of the present application;
fig. 6 is a schematic structural diagram of an electronic device according to an embodiment of the present application.
Detailed Description
Embodiments of the present application are described below in conjunction with the drawings in the present application. It should be understood that the embodiments set forth below in connection with the drawings are exemplary descriptions for explaining technical solutions of the embodiments of the present application, and do not limit the technical solutions of the embodiments of the present application.
As used herein, the singular forms "a", "an", "the" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should be further understood that the terms "comprises" and/or "comprising," when used in this specification in connection with embodiments of the present application, specify the presence of stated features, information, data, steps, operations, elements, and/or components, but do not preclude the presence or addition of other features, information, data, steps, operations, elements, components, and/or groups thereof, as embodied in the art. It will be understood that when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements may be present. Further, "connected" or "coupled" as used herein may include wirelessly connected or wirelessly coupled. The term "and/or" as used herein indicates at least one of the items defined by the term, e.g., "a and/or B" indicates either an implementation as "a", or an implementation as "B", or an implementation as "a and B".
It is understood that in the specific implementation of the present application, data related to data processing is referred to, when the above embodiments of the present application are applied to specific products or technologies, user permission or consent needs to be obtained, and the collection, use and processing of the related data need to comply with relevant laws and regulations and standards of relevant countries and regions.
To make the objects, technical solutions and advantages of the present application more clear, embodiments of the present application will be described in further detail below with reference to the accompanying drawings.
The embodiment of the application relates to a data processing method provided by a data processing system, and the data processing method relates to the fields related to character model driving, such as the fields of motion sensing games, movie production, medical rehabilitation and the like.
For better understanding and description of the embodiments of the present application, some technical terms used in the embodiments of the present application will be briefly described below.
Kinect body sensing equipment: the Kinect somatosensory device is a 3D somatosensory camera, and simultaneously has the functions of instant dynamic capture, image recognition, microphone input, voice recognition, community interaction and the like.
The technical scheme provided by the embodiment of the application relates to a data processing technology, and the technical scheme of the application is described in detail by specific embodiments. The following several specific embodiments may be combined with each other, and details of the same or similar concepts or processes may not be repeated in some embodiments. Embodiments of the present application will be described below with reference to the accompanying drawings.
In order to better understand the scheme provided by the embodiment of the present application, the scheme is described below with reference to a specific application scenario.
In an embodiment, fig. 1 shows an architecture diagram of a data processing system to which the embodiment of the present application is applied, and it can be understood that the data processing method provided in the embodiment of the present application may be applied to, but is not limited to, the application scenario shown in fig. 1.
In this example, as shown in fig. 1, the architecture of the data processing system may include, but is not limited to, a motion sensing device 10, a display device 20, a network 30, and a database 40; the motion sensing device 10, the display device 20, and the database 40 may interact with each other through the network 30. The motion sensing device 10 acquires first rotational motion data of a target object; the somatosensory device 10 inputs the first rotational motion data to the trained convolutional self-coding network, and performs data processing to obtain processed first rotational motion data; the data precision of the processed first rotational motion data is greater than that of the first rotational motion data; the motion sensing device 10 drives a role model corresponding to the target object through the processed first rotational motion data; the display device 20 displays the character model motion; the motion sensing device 10 may store the processed first rotational motion data in the database 40.
It is understood that the above is only an example, and the present embodiment is not limited thereto.
Referring to fig. 2, fig. 2 shows a flowchart of a data processing method provided in an embodiment of the present application, where the method may be executed by any electronic device, such as a motion sensing device, as an optional implementation, the method may be executed by the motion sensing device, and for convenience of description, in the following description of some optional embodiments, the motion sensing device will be taken as an example of a main body of the method. As shown in fig. 2, the data processing method provided in the embodiment of the present application includes the following steps:
s201, first rotation motion data of the target object is acquired.
Specifically, the target object may be a human, an animal, or the like. The somatosensory equipment can acquire original rotational motion data of a target object through a camera, a sensor and the like; the motion sensing device preprocesses the original rotational motion data through the preprocessing module to obtain first rotational motion data of the target object. The data accuracy of the raw rotational motion data may be the data accuracy of the rotational motion data collected by a low cost motion data capture device, such as a Kinect somatosensory device, so that both the raw rotational motion data and the first rotational motion data are low accuracy rotational motion data.
S202, inputting the first rotational motion data into the trained convolutional self-coding network, and performing data processing to obtain processed first rotational motion data; the data accuracy of the processed first rotational motion data is greater than the data accuracy of the first rotational motion data.
Specifically, the processed first rotational motion data is high-precision rotational motion data. The network structure of the convolutional self-coding network is shown in fig. 3, and the network structure of the convolutional self-coding network shown in fig. 3 includes a first layer convolutional layer (25 × 1conv, 256), an activation function layer Relu, a pooling layer pool, a second layer convolutional layer (25 × 1conv, 128), a third layer convolutional layer (25 × 1conv, 128), an anti-pooling layer pool, a fourth layer convolutional layer (25 × 1conv, 256), and a full connection layer FC. For example, 25 × 1 in the convolutional layer (25 × 1conv, 256) represents the size of the convolution kernel, i.e., the height × width of the convolution kernel; 256 denotes the dimension, i.e. the convolutional layer has 256 25 × 1 convolutional kernels; conv denotes convolution.
And S203, driving the role model corresponding to the target object through the processed first rotation motion data.
Specifically, the motion sensing device uses the high-precision rotational motion data (the processed first rotational motion data) to drive the character model corresponding to the target object, so that the motion precision of the character model is improved, and the motion of the character model is natural and smooth.
In the embodiment of the application, first rotation motion data of a target object is obtained; inputting the first rotational motion data into a trained convolutional self-coding network, and performing data processing to obtain processed first rotational motion data; the data precision of the processed first rotation motion data is greater than that of the first rotation motion data; driving a role model corresponding to the target object through the processed first rotational motion data; so, the first rotary motion data that body sensing equipment gathered, low accuracy rotary motion data promptly, through the first rotary motion data of convolution self-encoding network generation after handling, high accuracy rotary motion data promptly, body sensing equipment is used for driving role model with high accuracy rotary motion data to the accuracy of role model motion has been promoted.
In one embodiment, acquiring first rotational motion data of a target object comprises:
acquiring original rotation motion data of a target object;
and carrying out normalization and relative coordinate conversion on the original rotational motion data to obtain first rotational motion data of the target object.
Specifically, the motion sensing device preprocesses the original rotational motion data through a preprocessing module to obtain first rotational motion data of the target object. The preprocessing can be normalization and relative coordinate conversion processing, namely the preprocessing comprises normalization and relative coordinate conversion; the relative coordinate conversion is to convert the absolute coordinate into a relative coordinate, for example, calculate a difference between certain joint data (an absolute coordinate of a certain joint) and hip joint data (an absolute coordinate of a hip joint) of the target object, and obtain a relative rotation coordinate of the hip joint with respect to the joint.
In one embodiment, before acquiring the first rotational motion data of the target object, the method further comprises:
obtaining second rotational motion data of the sample object;
inputting the second rotational motion data of the sample object into a convolution self-coding network, and performing data processing to obtain processed second rotational motion data;
determining a value of a loss function of the convolutional self-coding network based on the processed second rotational motion data and preset tag data; the value of the loss function is used for representing the error between the processed second rotational motion data and the preset label data;
if the value of the loss function is larger than the preset loss threshold value, training the convolutional self-coding network, and updating the network parameters of the convolutional self-coding network;
repeatedly executing the steps of obtaining second rotational motion data of the sample object, inputting the second rotational motion data of the sample object into the convolutional self-coding network, performing data processing to obtain processed second rotational motion data, determining the value of a loss function of the convolutional self-coding network based on the processed second rotational motion data and preset label data, and training the convolutional self-coding network if the value of the loss function is larger than a preset loss threshold, updating network parameters of the convolutional self-coding network until the value of the loss function is equal to the preset loss threshold, and obtaining the trained convolutional self-coding network.
Specifically, raw rotational motion data of a sample object is acquired; and carrying out normalization and relative coordinate conversion on the original rotational motion data of the sample object to obtain second rotational motion data of the sample object.
Specifically, for example, as shown in fig. 3, the second rotational motion data of the sample object is taken as the input data θ in ，θ in May be f × d; wherein f is theta in D is the number of coordinates of the skeletal joint points of the sample object, d =3 × n, n is the number of joint points of the sample object. Will theta in Inputting the data into a first layer convolution layer (25 multiplied by 1conv, 256) of a convolution self-coding network, and obtaining a characteristic vector with dimension of f multiplied by 256 through convolution calculation; processing the feature vector with the dimension of f multiplied by 0256 through an activation function layer Relu and a pooling layer pool to obtain the feature vector with the dimension of (f/2) multiplied by 1256, wherein the pooling layer pool adopts maximum pooling; carrying out convolution calculation on the feature vector with the dimension of (f/2) multiplied by 2256 through a second convolution layer (25 multiplied by 31conv, 128) and a third convolution layer (25 multiplied by 41conv, 128) to obtain the feature vector with the dimension of (f/2) multiplied by 128; processing the feature vector with the dimension of (f/2) multiplied by 128 through a depool layer to obtain the feature vector with the dimension of f multiplied by 128; inputting the feature vector with the dimension of f multiplied by 128 into a fourth layer of convolution layer (25 multiplied by 1conv, 256), and carrying out convolution calculation to obtain the feature vector with the dimension of f multiplied by 256; inputting the characteristic vector with the dimension of f multiplied by 256 into the full connection layer FC to obtain output data theta with the dimension of f multiplied by d out I.e. the processed second rotational movement data.
It should be noted that the activation function layer Relu is to ensure that the convolution obtains a non-linear manifold from the coding network to better fit the motion; the pool layer pool has the functions of ensuring invariance of a time domain to a certain extent, accelerating calculation speed and preventing overfitting of convolutional self-coding network training; the role of the anti-pooling layer depool is to keep consistent with that before pooling in the time dimension; the role of the full connectivity layer FC is to change the data dimension.
In one embodiment, determining a value of a loss function of the convolutional self-coding network based on the processed second rotational motion data and the preset tag data comprises:
determining the mean square error of the joint point in the sample object based on the processed second rotational motion data and the preset label data; the data precision of the preset label data is greater than the data precision of the processed second rotation motion data;
and determining the value of the loss function of the convolutional self-coding network based on the mean square error of the joint points in the sample object, the number of the joint points of the sample object and the preset joint point weight.
In particular, a loss function theta of a convolutional self-coding network is calculated Loss As shown in equation (1):
wherein, ω is i A weight of an articulation point representing a sample object (preset articulation point weight),
i represents the serial number of the joint points of the sample object, the value range of i is 2-n +1, and n represents the number of the joint points of the sample object; f represents the number of acquisition frames of the input data of the convolutional self-coding network, theta c Denotes label data (preset label data), θ out Output data for the convolutional self-coding network; [ theta ] out -θ c || 2 Representing the mean square error of the rotational coordinates of the joint point in the sample object.
It should be noted that, the closer the joint point of the sample object is to the root node, the heavier the weight of the joint point is, and the weight of the end joint point of the sample object farthest from the root node is the smallest; wherein the root node may be a hip joint in the center of the sample object.
In one embodiment, determining a value of a loss function of a convolutional self-coding network based on a mean square error of the nodes in the sample object, the number of nodes of the sample object, and a preset node weight comprises:
determining an initial value of a loss function of the convolutional self-coding network based on the mean square error of the joint points in the sample object, the number of the joint points of the sample object and a preset joint point weight;
and determining the value of the loss function of the convolution self-coding network based on the initial value and a preset smooth L1 function.
In particular, to prevent training gradient explosions, the initial value θ of the loss function in the convolutional self-coding network loss Adding smooth L1 function to obtain loss function theta of the convolutional self-coding network' Loss (ii) a Therefore, the robustness and stability of the convolutional self-coding network training can be improved. Calculating a loss function theta 'of a convolutional self-coding network' Loss As shown in equation (2):
wherein, theta Loss An initial value representing a loss function of the convolutional self-coding network; 0.5 × theta Loss 2 、|θ Loss And 0.5 represents a preset smooth L1 function.
The application of the embodiment of the application has at least the following beneficial effects:
the motion sensing device comprises a body sensing device, wherein the body sensing device is used for acquiring first rotational motion data, namely low-precision rotational motion data, and the first rotational motion data after processing, namely high-precision rotational motion data, is generated through a convolution self-coding network, and is used for driving a role model, so that the motion accuracy of the role model is improved.
In order to better understand the method provided by the embodiment of the present application, the following further describes the scheme of the embodiment of the present application with reference to an example of a specific application scenario.
In a specific application scenario embodiment, for example, a motion sensing game scenario, referring to fig. 4, a processing flow of a data processing method is shown, and as shown in fig. 4, the processing flow of the data processing method provided in the embodiment of the present application includes the following steps:
s401, the motion sensing device obtains a training sample set, and preprocesses the training sample set to obtain a preprocessed training sample set.
Specifically, the training sample set may be raw rotational motion data of sample objects, which may be game characters; the preprocessing may be a process of normalization and relative coordinate conversion; the pre-processed training sample set may be rotational motion data of the sample object.
S402, the somatosensory device trains the convolutional self-coding network based on the preprocessed training sample set to obtain the trained convolutional self-coding network.
Specifically, inputting the preprocessed training sample set into a convolutional self-coding network, and performing data processing to obtain processed rotational motion data; determining a value of a loss function of the convolutional self-coding network based on the processed rotational motion data and preset tag data; the value of the loss function is used for representing the error between the processed rotational motion data and the preset label data; if the value of the loss function is larger than the preset loss threshold value, training the convolutional self-coding network, and updating the network parameters of the convolutional self-coding network; and if the value of the loss function is equal to the preset loss threshold value, obtaining the trained convolutional self-coding network.
And S403, the motion sensing device acquires original rotation motion data of the target object, and preprocesses the original rotation motion data to obtain the rotation motion data of the target object.
Specifically, the original rotational motion data of the target object and the rotational motion data of the target object are both low-precision rotational motion data, and the target object may be a target game character; the preprocessing may be a process of normalization and relative coordinate conversion.
And S404, the somatosensory device inputs the rotational motion data of the target object into the trained convolutional self-coding network, and performs data processing to obtain the processed rotational motion data of the target object.
Specifically, the data accuracy of the processed rotational motion data of the target object is greater than the data accuracy of the rotational motion data of the target object.
And S405, the motion sensing device drives the role model corresponding to the target object through the processed rotating motion data of the target object.
Specifically, the display device connected with the motion sensing device can display the motion of the character model; the character model may be a game character model.
The application of the embodiment of the application has at least the following beneficial effects:
the rotational motion data of the target game role, namely the low-precision rotational motion data, acquired by the motion sensing device is generated into the processed rotational motion data of the target game role, namely the high-precision rotational motion data, through the convolution self-coding network, and the high-precision rotational motion data is used for driving the game role model by the motion sensing device, so that the motion precision of the game role model is improved, and the motion of the game role model is natural and smooth.
The embodiment of the present application further provides a data processing apparatus, which is applied to a motion sensing device, and a schematic structural diagram of the data processing apparatus is shown in fig. 5, where the data processing apparatus 50 includes a training module 501, a first processing module 502, a second processing module 503, and a third processing module 504.
In an embodiment, the first processing module 502 is specifically configured to:
acquiring original rotation motion data of a target object;
and carrying out normalization and relative coordinate conversion on the original rotational motion data to obtain first rotational motion data of the target object.
In one embodiment, the data processing apparatus further comprises a training module 501, the training module 501 is configured to:
obtaining second rotational motion data of the sample object;
inputting the second rotational motion data of the sample object into a convolution self-coding network, and performing data processing to obtain processed second rotational motion data;
determining a value of a loss function of the convolutional self-coding network based on the processed second rotational motion data and preset tag data; the value of the loss function is used for representing the error between the processed second rotational motion data and the preset label data;
if the value of the loss function is larger than the preset loss threshold value, training the convolutional self-coding network, and updating the network parameters of the convolutional self-coding network;
repeatedly executing the steps of obtaining second rotational motion data of the sample object, inputting the second rotational motion data of the sample object into the convolutional self-coding network, performing data processing to obtain processed second rotational motion data, determining the value of a loss function of the convolutional self-coding network based on the processed second rotational motion data and preset label data, and training the convolutional self-coding network if the value of the loss function is larger than a preset loss threshold, updating network parameters of the convolutional self-coding network until the value of the loss function is equal to the preset loss threshold, and obtaining the trained convolutional self-coding network.
In one embodiment, the training module 501 is specifically configured to:
determining the mean square error of the joint point in the sample object based on the processed second rotational motion data and the preset label data; the data precision of the preset label data is larger than the data precision of the processed second rotation motion data;
and determining the value of the loss function of the convolutional self-coding network based on the mean square error of the joint points in the sample object, the number of the joint points of the sample object and the preset joint point weight.
In one embodiment, the training module 501 is specifically configured to:
determining an initial value of a loss function of the convolutional self-coding network based on the mean square error of the joint points in the sample object, the number of the joint points of the sample object and a preset joint point weight;
and determining the value of the loss function of the convolutional self-coding network based on the initial value and a preset smooth L1 function.
The application of the embodiment of the application has at least the following beneficial effects:
acquiring first rotational motion data of a target object; inputting the first rotational motion data into a trained convolutional self-coding network, and performing data processing to obtain processed first rotational motion data; the data precision of the processed first rotational motion data is greater than that of the first rotational motion data; driving a role model corresponding to the target object through the processed first rotational motion data; so, the first rotary motion data that body sensing equipment gathered, low accuracy rotary motion data promptly, through the first rotary motion data of convolution self-encoding network generation after handling, high accuracy rotary motion data promptly, body sensing equipment is used for driving role model with high accuracy rotary motion data to the accuracy of role model motion has been promoted.
An embodiment of the present application further provides an electronic device, a schematic structural diagram of the electronic device is shown in fig. 6, and an electronic device 4000 shown in fig. 6 includes: a processor 4001 and a memory 4003. Processor 4001 is coupled to memory 4003, such as via bus 4002. Optionally, the electronic device 4000 may further include a transceiver 4004, and the transceiver 4004 may be used for data interaction between the electronic device and other electronic devices, such as transmission of data and/or reception of data. It should be noted that the transceiver 4004 is not limited to one in practical applications, and the structure of the electronic device 4000 is not limited to the embodiment of the present application.
The Processor 4001 may be a CPU (Central Processing Unit), a general-purpose Processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array) or other Programmable logic device, a transistor logic device, a hardware component, or any combination thereof. Which may implement or perform the various illustrative logical blocks, modules, and circuits described in connection with the disclosure. The processor 4001 may also be a combination that performs a computational function, including, for example, a combination of one or more microprocessors, a combination of a DSP and a microprocessor, or the like.
The Memory 4003 may be a ROM (Read Only Memory) or other types of static storage devices that can store static information and instructions, a RAM (Random Access Memory) or other types of dynamic storage devices that can store information and instructions, an EEPROM (Electrically Erasable Programmable Read Only Memory), a CD-ROM (Compact Disc Read Only Memory) or other optical Disc storage, optical Disc storage (including Compact Disc, laser Disc, optical Disc, digital versatile Disc, blu-ray Disc, etc.), a magnetic Disc storage medium, other magnetic storage devices, or any other medium that can be used to carry or store a computer program and that can be Read by a computer, without limitation.
The memory 4003 is used for storing computer programs for executing the embodiments of the present application, and is controlled by the processor 4001 to execute. The processor 4001 is configured to execute a computer program stored in the memory 4003 to implement the steps shown in the foregoing method embodiments.
Wherein, the electronic device includes but is not limited to: a motion sensing device, etc.
The application of the embodiment of the application has at least the following beneficial effects:
acquiring first rotational motion data of a target object; inputting the first rotational motion data into a trained convolutional self-coding network, and performing data processing to obtain processed first rotational motion data; the data precision of the processed first rotational motion data is greater than that of the first rotational motion data; driving a role model corresponding to the target object through the processed first rotational motion data; so, the first rotary motion data that motion sensing equipment gathered, low accuracy rotary motion data promptly generates the first rotary motion data after handling, high accuracy rotary motion data promptly through the convolution self-coding network, and motion sensing equipment is used for driving role model with high accuracy rotary motion data to role model motion's accuracy has been promoted.
Embodiments of the present application provide a computer-readable storage medium, on which a computer program is stored, and when being executed by a processor, the computer program may implement the steps and corresponding contents of the foregoing method embodiments.
Embodiments of the present application further provide a computer program product, which includes a computer program, and when the computer program is executed by a processor, the steps and corresponding contents of the foregoing method embodiments can be implemented.
Based on the same principle as the method provided by the embodiment of the present application, the embodiment of the present application also provides a computer program product or a computer program, and the computer program product or the computer program comprises computer instructions, and the computer instructions are stored in a computer readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to cause the computer device to perform the method provided in any of the alternative embodiments of the present application.
It should be understood that, although each operation step is indicated by an arrow in the flowchart of the embodiment of the present application, the implementation order of the steps is not limited to the order indicated by the arrow. In some implementation scenarios of the embodiments of the present application, the implementation steps in the flowcharts may be performed in other sequences as desired, unless explicitly stated otherwise herein. In addition, some or all of the steps in each flowchart may include multiple sub-steps or multiple stages based on an actual implementation scenario. Some or all of these sub-steps or stages may be performed at the same time, or each of these sub-steps or stages may be performed at different times, respectively. In a scenario where execution times are different, an execution sequence of the sub-steps or the phases may be flexibly configured according to requirements, which is not limited in the embodiment of the present application.
The foregoing is only an optional implementation manner of a part of implementation scenarios in this application, and it should be noted that, for those skilled in the art, other similar implementation means based on the technical idea of this application are also within the protection scope of the embodiments of this application without departing from the technical idea of this application.
Claims (10)
1. A data processing device is applied to a body sensing device and is characterized by comprising:
the first processing module is used for acquiring first rotation motion data of a target object;
the second processing module is used for inputting the first rotational motion data into the trained convolutional self-coding network and carrying out data processing to obtain processed first rotational motion data; the data precision of the processed first rotational motion data is greater than that of the first rotational motion data;
and the third processing module is used for driving the role model corresponding to the target object through the processed first rotation motion data.
2. The apparatus of claim 1, wherein the first processing module is specifically configured to:
acquiring original rotation motion data of a target object;
and carrying out normalization and relative coordinate conversion on the original rotational motion data to obtain first rotational motion data of the target object.
3. The apparatus of claim 1, wherein the data processing apparatus further comprises a training module configured to:
obtaining second rotational motion data of the sample object;
inputting the second rotational motion data of the sample object into a convolution self-coding network, and performing data processing to obtain processed second rotational motion data;
determining a value of a loss function of the convolutional self-coding network based on the processed second rotational motion data and preset tag data; the value of the loss function is used for representing the error between the processed second rotational motion data and the preset label data;
if the value of the loss function is larger than a preset loss threshold value, training the convolutional self-coding network, and updating the network parameters of the convolutional self-coding network;
repeatedly executing the second rotation motion data of the sample object, inputting the second rotation motion data of the sample object into a convolution self-coding network, performing data processing to obtain processed second rotation motion data, determining the value of a loss function of the convolution self-coding network based on the processed second rotation motion data and preset label data, and training the convolution self-coding network and updating the network parameters of the convolution self-coding network if the value of the loss function is greater than a preset loss threshold value until the value of the loss function is equal to the preset loss threshold value, thereby obtaining the trained convolution self-coding network.
4. The apparatus of claim 3, wherein the training module is specifically configured to:
determining a mean square error of a joint point in the sample object based on the processed second rotational motion data and preset tag data; the data precision of the preset label data is larger than that of the processed second rotation motion data;
and determining the value of the loss function of the convolutional self-coding network based on the mean square error of the joint points in the sample object, the number of the joint points of the sample object and preset joint point weight.
5. The apparatus of claim 4, wherein the training module is specifically configured to:
determining an initial value of a loss function of the convolutional self-coding network based on the mean square error of the joint points in the sample object, the number of the joint points of the sample object and a preset joint point weight;
and determining the value of the loss function of the convolution self-coding network based on the initial value and a preset smooth L1 function.
6. A data processing method is executed by a body sensing device and is characterized by comprising the following steps:
acquiring first rotational motion data of a target object;
inputting the first rotational motion data into a trained convolutional self-coding network, and performing data processing to obtain processed first rotational motion data; the data precision of the processed first rotational motion data is greater than that of the first rotational motion data;
and driving the role model corresponding to the target object through the processed first rotation motion data.
7. The method of claim 6, further comprising, prior to the obtaining first rotational motion data of the target object:
obtaining second rotational motion data of the sample object;
inputting the second rotational motion data of the sample object into a convolution self-coding network, and performing data processing to obtain processed second rotational motion data;
determining a value of a loss function of the convolutional self-coding network based on the processed second rotational motion data and preset tag data; the value of the loss function is used for representing the error between the processed second rotational motion data and the preset label data;
if the value of the loss function is larger than a preset loss threshold value, training the convolutional self-coding network, and updating the network parameters of the convolutional self-coding network;
repeatedly executing the second rotation motion data of the sample object, inputting the second rotation motion data of the sample object into a convolution self-coding network, performing data processing to obtain processed second rotation motion data, determining the value of a loss function of the convolution self-coding network based on the processed second rotation motion data and preset label data, and training the convolution self-coding network and updating the network parameters of the convolution self-coding network if the value of the loss function is greater than a preset loss threshold value until the value of the loss function is equal to the preset loss threshold value, thereby obtaining the trained convolution self-coding network.
8. An electronic device comprising a memory, a processor and a computer program stored on the memory, characterized in that the processor executes the computer program to implement the steps of the method of any of claims 6-7.
9. A computer-readable storage medium, on which a computer program is stored which, when being executed by a processor, carries out the steps of the method according to any one of claims 6 to 7.
10. A computer program product comprising a computer program, characterized in that the computer program realizes the steps of the method of any one of claims 6-7 when executed by a processor.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211001275.8A CN115205340A (en) | 2022-08-19 | 2022-08-19 | Data processing method, device, equipment, readable storage medium and program product |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211001275.8A CN115205340A (en) | 2022-08-19 | 2022-08-19 | Data processing method, device, equipment, readable storage medium and program product |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115205340A true CN115205340A (en) | 2022-10-18 |
Family
ID=83573763
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211001275.8A Pending CN115205340A (en) | 2022-08-19 | 2022-08-19 | Data processing method, device, equipment, readable storage medium and program product |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115205340A (en) |
-
2022
- 2022-08-19 CN CN202211001275.8A patent/CN115205340A/en active Pending
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108694369B (en) | Predicting multiple poses based on a graphical image | |
| JP7453470B2 (en) | 3D reconstruction and related interactions, measurement methods and related devices and equipment | |
| CN111815754B (en) | Three-dimensional information determining method, three-dimensional information determining device and terminal equipment | |
| WO2018119889A1 (en) | Three-dimensional scene positioning method and device | |
| CN112446919B (en) | Object pose estimation method and device, electronic equipment and computer storage medium | |
| CN113706699B (en) | Data processing method and device, electronic equipment and computer readable storage medium | |
| CN110866977B (en) | Augmented reality processing method, device, system, storage medium and electronic equipment | |
| CN110310333A (en) | Localization method and electronic equipment, readable storage medium storing program for executing | |
| CN110335317B (en) | Image processing method, device, equipment and medium based on terminal equipment positioning | |
| CN115631305A (en) | Driving method of skeleton model of virtual character, plug-in and terminal equipment | |
| CN113043267A (en) | Robot control method, device, robot and computer readable storage medium | |
| CN109754464B (en) | Method and apparatus for generating information | |
| CN115546365A (en) | Virtual human driving method and system | |
| CN115205383A (en) | Camera pose determination method and device, electronic equipment and storage medium | |
| CN114241597A (en) | Posture recognition method and related equipment thereof | |
| CN113592015B (en) | Method and device for positioning and training feature matching network | |
| CN111681302A (en) | Method and device for generating 3D virtual image, electronic equipment and storage medium | |
| CN104573737B (en) | The method and device of positioning feature point | |
| CN111179408B (en) | Three-dimensional modeling method and equipment | |
| CN114972634A (en) | Multi-view three-dimensional deformable human face reconstruction method based on feature voxel fusion | |
| CN117876608A (en) | Three-dimensional image reconstruction method, three-dimensional image reconstruction device, computer equipment and storage medium | |
| CN112819963B (en) | Batch differential modeling method for tree branch model and related equipment | |
| CN116734834A (en) | Positioning and mapping method and device applied to dynamic scene and intelligent equipment | |
| CN115205340A (en) | Data processing method, device, equipment, readable storage medium and program product | |
| CN109816709B (en) | Monocular camera-based depth estimation method, device and equipment |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |