CN115964933A - Construction method of virtual and real training device based on digital twins - Google Patents
Construction method of virtual and real training device based on digital twins Download PDFInfo
- Publication number
- CN115964933A CN115964933A CN202211473502.7A CN202211473502A CN115964933A CN 115964933 A CN115964933 A CN 115964933A CN 202211473502 A CN202211473502 A CN 202211473502A CN 115964933 A CN115964933 A CN 115964933A
- Authority
- CN
- China
- Prior art keywords
- data
- real
- motion
- digital
- twin
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Classifications
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02A—TECHNOLOGIES FOR ADAPTATION TO CLIMATE CHANGE
- Y02A90/00—Technologies having an indirect contribution to adaptation to climate change
- Y02A90/10—Information and communication technologies [ICT] supporting adaptation to climate change, e.g. for weather forecasting or climate simulation
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The invention discloses a construction method of a virtual-real training device based on digital twins, belonging to the technical field of digital twins, and the method comprises the following steps: constructing a digital twinning system; respectively constructing a digital three-dimensional twin body of a moving object and a twin model of a moving scene; fusing the digital three-dimensional twin body of the moving object and the twin model of the moving scene to obtain a static training model, and importing the static training model into a digital twin system; acquiring real-time motion data of a moving object in a motion process, and transmitting the real-time motion data to a digital twin system; and according to the real-time motion data, the static training model is iteratively optimized to obtain a real-time training model, and virtual and real synchronous training is realized. The invention solves the problem that the training process of the sports object can not be analyzed digitally, provides digital support for the discovery of the problems in the training process of the sports and helps the sports object to effectively improve the training performance.
Description
Technical Field
The invention belongs to the technical field of digital twins, and particularly relates to a construction method of a virtual and real training device based on digital twins.
Background
Wearable equipment such as sports bracelet, sports watch accessible inertial sensor record step number, monitoring rhythm of the heart, record motion data etc. because the data of gathering are incomplete, many training plans, body-building plan are only fixed the generation according to the procedure mechanically, can not be accurate, individualized each sports object of being applicable to, also can not reflect sports object's motion state, physiological state and motion demand in real time. Sports injuries are a common disease occurring during sports, and are often found in gymnastics athletes, ball athletes, long distance runners, military and military trainees, and in various groups interested in physical exercise. Tennis elbow, running knee and various soft tissue injuries are more and more younger and popular, and are often not appreciated by people, so that the sports life of athletes is greatly shortened, and various diseases of the bodies of sports enthusiasts are caused.
With the development of new-generation information technologies (such as artificial intelligence, cloud computing, industrial internet of things, intelligent manufacturing, industrial 4.0 and smart cities), digital Twin (Digital Twin) technologies have also come into development and become new technologies which are widely concerned by various circles. With the intensive understanding of digital twins, research on human motion data digital twins is also a new research topic. The application of the digital twin technology in the field of sports training will also become a new development trend. The digital twin technology fuses real-time motion data, historical data, fusion derivative data, service data, simulation data and other data to construct a virtual digital twin motion body, fully utilizes physical models, sensors, motion historical data and the like, integrates multidisciplinary, multi-physical quantity, multi-scale and multi-probability simulation processes, and finishes mapping a real physical world in a virtual space, so that virtual-real mapping and virtual-real data interaction are realized, simulation analysis can be performed on physical objects, and different motion states of the physical objects can be detected and predicted.
Currently, research on human motion digital twins is also in the stage of launch. Generally, the research on the human motion digital twin theory method is shallow, and a human motion training digital twin system is not seen in the field of motion.
With the rapid development of the internet of things and information sensing technology, textile sensors are developed towards flexibility, miniaturization, intellectualization and multiple functions. The flexible wearable equipment is a core device for acquiring various physiological information of a human body, becomes a research hotspot in the fields of textile and clothing industry, biomedicine, national defense and military industry and the like, and at present, scholars at home and abroad are still in the stages of test and research on the flexible wearable equipment, and have less research on the theory and technology aspects of the flexible wearable equipment.
The flexible electronic technology deposits organic, inorganic or organic-inorganic composite materials on a flexible substrate to form a new cross scientific technology of electronic components and integrated systems represented by circuits, and the flexible electronics has the characteristics of lightness, flexibility, thinness, transparency and the like, can be conformal with other things and greatly expands the application range of electronic devices. By utilizing the flexible wearable technology, the sensor is integrated into the T-shirt without metal parts, and the textile sensor can monitor a plurality of important physical sign indexes.
The construction of the virtual moving object is the basis of constructing the moving life cycle of the moving digital twin body in the virtual environment. At present, some professional modeling software, such as Open Sim, poser, 3DS MAX and anybody, are constructed in a three-dimensional environment, and the three-dimensional human body modeling performance is better. Because the digital twin relates to multiple interdisciplines, the data related to human motion modeling is numerous, and the motion effect is diversified due to different motion environments and individual differences, and the simulation complexity is greatly enhanced.
Disclosure of Invention
Aiming at the defects in the prior art, the construction method of the virtual and real training device based on the digital twin solves the problem that the training process of the moving object cannot be analyzed digitally.
In order to achieve the purpose of the invention, the invention adopts the technical scheme that: a construction method of a virtual-real training device based on digital twins comprises the following steps:
s1, building a digital twinning system;
s2, respectively constructing a moving object digital three-dimensional twin body and a moving scene twin model;
s3, fusing the digital three-dimensional twin body of the moving object and the twin model of the moving scene to obtain a static training model, and importing the static training model into a digital twin system;
s4, acquiring real-time motion data of a moving object in a motion process, and transmitting the real-time motion data to a digital twin system;
and S5, iteratively optimizing the static training model according to the real-time motion data to obtain a real-time training model, and realizing virtual and real synchronous training.
The invention has the beneficial effects that: the digital twin model of the moving object and the moving scene is constructed by adopting a digital twin technology, real-time moving data of the moving process of the moving object is collected and reflected into the digital twin model, so that the digital three-dimensional twin of the moving object forms mapping of the moving object in a real space in the moving scene twin model, the data of the moving object in the training process is monitored and analyzed in real time, the digitization of the moving object in the training process is realized, the digitization support is provided for finding problems in the moving training process, and the training performance of the moving object is effectively improved.
Further, the digital twin system in the step S1 includes a physical layer, a virtual layer, a service layer, and a data layer;
the physical layer is used for acquiring motion data of a moving object, landmark data and weather data of a real motion scene, acquiring real-time motion data according to the motion data, the landmark data and the weather data, and respectively transmitting the real-time motion data to the data layer and the virtual layer;
the virtual layer is used for performing iterative optimization on the static training model according to the real-time motion data and the positioning information to obtain a simulation model and simulation data and construct a real-time training model; respectively transmitting the simulation data and the simulation model to a data layer and a service layer;
the service layer is used for respectively displaying a real motion process of a motion object and a virtual motion process of the digital three-dimensional twin body according to the real-time motion data and the simulation model, and simultaneously supporting a user to carry out simulation training on the digital three-dimensional twin body to obtain simulation data; the positioning information is used for acquiring the positioning information of the moving object; the monitoring system is used for monitoring the physical signs of the moving object according to the real-time movement data to obtain a monitoring result; judging whether an alarm module needs to be started or not according to the monitoring result; the system is used for seeking help for rescue workers when the physical signs of the moving object are abnormal, and sending the positioning information and the monitoring result to the rescue workers; obtaining service data according to the monitoring result, the positioning information and the simulation data, and transmitting the service data to a data layer;
the data layer is used for judging the timeliness of the real-time motion data, and if the time is larger than a preset value, historical data are obtained; storing real-time motion data, simulation data, service data, historical data and fusion derivative data; and transmitting the real-time motion data and the historical data to a service layer.
The beneficial effects of the above further scheme are: data of a moving object in a moving process are collected by a physical layer and then transmitted to a virtual layer, a data layer and a service layer of the digital twin system, model simulation is carried out on the virtual layer, data processing analysis is carried out on the data layer, and information interaction is carried out on the service layer; the digital twin system can react to the data change in the training process of the moving object in real time, and the accuracy and the real-time performance of the moving data of the moving object are ensured.
Furthermore, the service layer comprises a motion data monitoring service module, a positioning module, a motion posture display service module, an alarm module and a real-time control service module;
the motion data monitoring service module is used for monitoring the physical signs of the moving object according to the real-time motion data to obtain monitoring results and feeding the monitoring results back to the real-time control service module and the alarm module respectively;
the positioning module is used for acquiring positioning information of a moving object and transmitting the positioning information to the real-time control service module, the alarm module and the virtual layer respectively;
the motion posture display module is used for displaying a real motion process of a motion object and a virtual motion process of a digital three-dimensional twin body of the motion object according to the real-time motion data and the simulation model, and simultaneously supporting a user to perform simulation training on the digital three-dimensional twin body of the motion object according to historical data to obtain simulation data and transmit the simulation data to the real-time control service module;
the real-time control service module is used for judging whether an alarm module needs to be started or not according to the monitoring result; acquiring a simulation model and real-time motion data from a data layer, respectively transmitting the real-time motion data to a motion data monitoring service module and a motion attitude display module, and transmitting the simulation model to the motion attitude display module; controlling the postures of the digital three-dimensional twins when the digital three-dimensional twins are subjected to simulation training; obtaining service data according to the monitoring result, the positioning information and the simulation data;
the alarm module is used for asking for help to rescue workers when the physical signs of the moving object are abnormal, and sending the positioning information and the monitoring result to the rescue workers.
The beneficial effects of the above further scheme are: the training expert can check the motion condition of the motion object in real time through the motion posture display module, and can utilize historical data to carry out simulation training on the digital three-dimensional twin body of the motion object, a training plan is formulated, the starting of the positioning system also provides coordinate positioning for simulation training of the digital three-dimensional twin body of the motion object in the motion scene digital twin model, the setting of the alarm module enables the motion object to be timely called for help when an accident occurs in the training, safety guarantee is provided for the motion object, and the construction of the service layer provides a good interaction basis for the whole virtual and real training device.
Further, the data layer comprises a data transmission module, a database module and a data processing module;
the data transmission module is used for transmitting the real-time motion data, the simulation data, the service data and the historical data and the fusion derivative data generated in the operation process of the data processing module to the database module for storage; respectively transmitting the real-time motion data, the simulation data, the service data, the historical data and the fusion derivative data to a data processing module for processing; transmitting the real-time motion data and the historical data to a service layer;
the database module is used for storing real-time motion data, simulation data, service data, historical data and fusion derivative data;
and the data processing module is used for judging the timeliness of the real-time motion data, obtaining historical data if the time is greater than a preset value, and transmitting the historical data to the database module through the data transmission module, otherwise, not processing.
The beneficial effects of the above further scheme are: the data layer processes and transmits data fed back by the physical layer, the virtual layer and the service layer in time to provide data support for the virtual and real training device, the database module is responsible for storing data generated in the training process of the moving object, and the storage and support of a large amount of data ensure the accuracy of a digital three-dimensional twin body of the moving object and a digital twin model of a moving scene, so that the virtual space forms high-accuracy mapping to the real space.
Further, the method for constructing the digital three-dimensional twin body of the moving object in step S2 includes the following steps:
a1, capturing motion data of a human body in a certain time period;
a2, preprocessing the action data to obtain preliminary data;
a3, respectively obtaining a shape parameter beta and an attitude parameter according to the preliminary data, and taking the shape parameter beta and the attitude parameter as the input of a human body SMPL model;
a4, the shape parameter beta and a basic template of the human body SMPL model
Superposing, mixing and forming to obtain mesh of the silent posture of the moving object; wherein the expression of the superposition is as follows:
D∈R 6890×3×10
β∈R 10
V shape ∈R 6890×3
wherein D is the deviation of the principal component, beta is the magnitude of the deviation of the principal component,
mesh, V, as a base template shape The mesh of the silent posture of the moving object is shown, R is a real number set, and the mesh is a human body network;
a5, superposing the attitude parameters and the mesh of the silent attitude of the moving object, and mixing skins to obtain a digital three-dimensional twin body of the moving object;
and A6, judging whether the error between the digital three-dimensional twin organism of the moving object and the moving object is smaller than a preset value, if so, completing the construction of the digital three-dimensional twin organism of the moving object, otherwise, adjusting the shape parameter beta and the posture parameter, and returning to the step A4.
The beneficial effects of the above further scheme are: the human body SMPL model is a learnable model, can better fit the shape of a human body and the deformation under different postures through training, obtains a high-precision moving object digital three-dimensional twin body, and prepares for constructing a virtual and actual training device.
Further, the method for constructing the twin model of the motion scene in step S2 includes the following steps:
b1, collecting landmark data and climate data of a real motion scene;
b2, determining a coordinate system of the virtual motion scene, and building a motion scene twin model according to the landmark data and the climate data;
and B3, judging whether the error between the twin model of the motion scene and the actual motion scene is smaller than a preset value, if so, completing the construction of the twin model of the motion scene, otherwise, adjusting the landmark data and the climate data, and returning to the step B2.
The beneficial effects of the above further scheme are: the real-time acquisition of the data of the motion scene of the motion object ensures the accuracy of the twin model of the motion scene, and prepares for the construction of a virtual and real training device.
Further, the method for constructing the real-time training model in step S5 includes the following steps:
c1, acquiring positioning information of a moving object;
c2, backing up the real-time motion data and the positioning information to obtain comparison data;
c3, performing iterative optimization on the static training model according to the real-time motion data and the positioning information to obtain simulation data;
c4, comparing the simulation data with the comparison data to obtain an error value of the simulation data and the comparison data;
c5, judging whether the error value is smaller than a set value, if so, obtaining a simulation model, updating the static training model into the simulation model, and entering the step C6, otherwise, adjusting the real-time motion data and returning to the step C3;
and C6, judging whether the training of the moving object is finished, if so, completing the construction of the real-time training model, otherwise, returning to the step C3.
The beneficial effects of the above further scheme are: and updating the training condition of the moving object in real time, reflecting the training condition of the moving object on the digital three-dimensional twin body of the moving object and the digital twin model of the moving scene, simulating the training condition of the moving object to form a real-time training model, and preparing for a training expert to analyze the training condition of the moving object.
Drawings
FIG. 1 is a flow chart of the method of the present invention.
Fig. 2 is a structural view of a digital twinning system in the present invention.
FIG. 3 is a flow chart of a method for constructing a digital three-dimensional twin body of a moving object according to the present invention.
FIG. 4 is a flow chart of a method for constructing a twin model of a motion scene in the present invention.
FIG. 5 is a flowchart of a method for constructing a real-time training model according to the present invention.
Detailed Description
The following description of the embodiments of the present invention is provided to facilitate the understanding of the present invention by those skilled in the art, but it should be understood that the present invention is not limited to the scope of the embodiments, and it will be apparent to those skilled in the art that various changes may be made without departing from the spirit and scope of the invention as defined and defined by the appended claims, and all changes that can be made by the invention using the inventive concept are intended to be protected.
As shown in fig. 1, the method for constructing a virtual-real training device based on digital twins, provided by the invention, comprises the following steps:
s1, building a digital twin system;
s2, respectively constructing a moving object digital three-dimensional twin body and a moving scene twin model;
s3, fusing the digital three-dimensional twin body of the moving object and the twin model of the moving scene to obtain a static training model, and importing the static training model into a digital twin system;
in this embodiment, a moving object digital twin and a moving scene twin model are fused, a coordinate system of the moving scene twin model is established according to a coordinate system of a real space, and a start and end position, a moving track, a moving posture, a three-dimensional model, real-time data of a body and the like of the moving object digital twin when the moving object digital twin moves in the moving scene twin model are determined according to information of a moving posture, a track, a start and end position, physiological data in a moving process and the like of the moving object, so that the moving object digital twin forms mapping of the moving object in the real space in the moving scene twin model.
S4, acquiring real-time motion data of a moving object in a motion process, and transmitting the real-time motion data to a digital twin system;
and S5, iteratively optimizing the static training model according to the real-time motion data to obtain a real-time training model, and realizing virtual and real synchronous training.
In the embodiment, the method for constructing the virtual and real training device based on the digital twin solves the problem that influence factors cannot be efficiently judged when a moving object is damaged by moving due to reasons of physical fitness of the moving object, insufficient knowledge on training skills, unscientific training method and the like in the moving process; meanwhile, the exercise training process is subjected to digitalized, virtualized and transparent mapping, digitalized support is provided for the discovery of training problems, the training performance of the exercise object is improved scientifically and efficiently, and exercise injuries are effectively avoided.
In the embodiment, the data is the core driving force of the virtual and real training device, and is different from the traditional model, so that the mutual mapping and the high consistency with the entity space are more emphasized; the data source includes not only the physical space, the virtual space, but also the fused data of the virtual and real combination, and the connection between the physical entity and the virtual space must have bidirectionality and compatibility. Bidirectional connection, bidirectional driving and bidirectional interaction are embodied, and compatibility is embodied as cross-platform, cross-interface and cross-protocol. And effectively processing and analyzing massive real-time data and historical data to obtain the digital three-dimensional twin body of the moving object. The method comprises the steps of optimizing a training scheme according to motion data of a moving object digital three-dimensional twin body on a virtual-real training device in a moving scene twin model, and obtaining a training plan which is most suitable for a moving object according to the actual situation of the moving object, so that the moving object is helped to be far away from motion damage to the greatest extent, the training level is improved, and the risk of injury of the moving object in the motion process is reduced to the greatest extent; meanwhile, the training process of the moving object is digitalized and visualized, and in the training process, when the moving object is damaged, the degree of muscle-bone damage can be known in time, so that scientific guidance is provided for the next treatment, and the recovery period is shortened; through the application of the technology, a large amount of mechanical and physiological data in the movement process are accumulated, and a data base is laid for researching the body structure and body change in the movement process of a human body.
As shown in fig. 2, the digital twin system in step S1 includes a physical layer, a virtual layer, a service layer, and a data layer;
the physical layer is used for acquiring motion data of a moving object, landmark data and weather data of a real motion scene, acquiring real-time motion data according to the motion data, the landmark data and the weather data, and respectively transmitting the real-time motion data to the data layer and the virtual layer;
in this embodiment, the physical layer measures data such as respiratory behavior, sweat content, body temperature, a runner electrocardiogram (heart rate) and brain waves (central nervous system), a resting heart rate, a safe heart rate threshold during training, a central nervous system state, a cardiopulmonary function system state, heart Rate Variability (HRV), a muscle function state (strength, speed, endurance, skill), a fatigue index, an energy metabolism index, and an autonomic nervous system balance index of a current moving object through the flexible wearable sensor, and stores the data in the data layer.
The virtual layer is used for performing iterative optimization on the static training model according to the real-time motion data and the positioning information to obtain a simulation model and simulation data and construct a real-time training model; respectively transmitting the simulation data and the simulation model to a data layer and a service layer;
the service layer is used for respectively displaying a real motion process of a motion object and a virtual motion process of the digital three-dimensional twin body according to the real-time motion data and the simulation model, and simultaneously supporting a user to carry out simulation training on the digital three-dimensional twin body to obtain simulation data; the positioning information is used for acquiring the positioning information of the moving object; the monitoring system is used for monitoring the physical signs of the moving object according to the real-time movement data to obtain a monitoring result; judging whether an alarm module needs to be started or not according to the monitoring result; the system is used for seeking help for rescue workers when the physical signs of the moving object are abnormal, and sending the positioning information and the monitoring result to the rescue workers; obtaining service data according to the monitoring result, the positioning information and the simulation data, and transmitting the service data to a data layer;
the data layer is used for judging the timeliness of the real-time motion data, and if the time is larger than a preset value, historical data are obtained; storing real-time motion data, simulation data, service data, historical data and fusion derivative data; and transmitting the real-time motion data and the historical data to a service layer.
In this embodiment, the virtual-real training device based on digital twins is constructed by means of a digital twins system. The service layer determines the motion posture, muscle-bone balance, stability, difference and other performances in the motion process by capturing the video data set with the three-dimensional human body posture mark, so that the model of the digital twin is optimized, and when the motion posture of the motion object is found to be abnormal, the digital twin timely sends out a stimulation signal to the motion object to improve the training action.
The data layer selects a proper transmission mode according to the collected real-time motion data, historical data, fusion derivative data, service data, simulation data and other data, so that the problems of data loss, interference, tampering, distortion and the like are avoided; the data is correspondingly preprocessed, so that the accuracy, the connectivity and the timely communication of the data are ensured, and the data of a multi-platform and multi-user system can be communicated with each other; and the data fusion can be realized on the same motion platform by using a dynamic heterogeneous data fusion method.
The service layer comprises a motion data monitoring service module, a positioning module, a motion posture display service module, an alarm module and a real-time control service module;
the motion data monitoring service module is used for monitoring the physical signs of the moving object according to the real-time motion data to obtain monitoring results and feeding the monitoring results back to the real-time control service module and the alarm module respectively;
the positioning module is used for acquiring positioning information of the moving object and respectively transmitting the positioning information to the real-time control service module, the alarm module and the virtual layer;
the motion posture display module is used for displaying a real motion process of a motion object and a virtual motion process of a digital three-dimensional twin body of the motion object according to the real-time motion data and the simulation model, and simultaneously supporting a user to perform simulation training on the digital three-dimensional twin body of the motion object according to historical data to obtain simulation data and transmit the simulation data to the real-time control service module;
in this embodiment, the motion data on the virtual and real training device is subjected to data processing, and then the dynamic test shows that the character does not conform to the motion state of the actual physical space, and if the character conforms to the motion state, the simulation training is performed to obtain the training scheme most suitable for the moving object, so that a corresponding training plan is made. The moving object carries out corresponding training according to the training plan to achieve the expected training effect.
The real-time control service module is used for judging whether an alarm module needs to be started or not according to the monitoring result; acquiring a simulation model and real-time motion data from a data layer, respectively transmitting the real-time motion data to a motion data monitoring service module and a motion attitude display module, and transmitting the simulation model to the motion attitude display module; controlling the posture of the digital three-dimensional twin body when performing analog training on the digital three-dimensional twin body; obtaining service data according to the monitoring result, the positioning information and the simulation data;
in this embodiment, the real-time control service module includes not only a control sub-module for a digital twin of a moving object, but also a feedback control sub-module for a moving object in the physical world. The control submodule for controlling the digital twin body of the moving object mainly comprises control digital three-dimensional twin body movement conditions, such as running, jumping, walking, moving according to a marker track and the like; the feedback control sub-module guides the training of the moving object according to the movement data of the digital twin body, and sends a signal to the physical entity in time and stops an error training mode when various indexes of the digital twin body are disordered after an unreasonable training plan and training intensity are executed.
The alarm module is used for asking for help to rescue workers when the physical signs of the moving object are abnormal, and sending the positioning information and the monitoring result to the rescue workers.
In the embodiment, the positioning module and the alarm module are arranged on the service layer of the system, when the moving object moves in the open air, the moving object acts independently, the emergency situation occurs, the virtual and real training device can send out a distress signal through the alarm module in time, and the positioning module can be used for positioning quickly and accurately to enable nearby search and rescue workers and medical rescue teams to arrive at the scene at the first time and handle the emergency situation, so that the risk coefficient is reduced to the minimum.
The data layer comprises a data transmission module, a database module and a data processing module;
the data transmission module is used for transmitting the real-time motion data, the simulation data, the service data and the historical data and the fusion derivative data generated in the operation process of the data processing module to the database module for storage; respectively transmitting the real-time motion data, the simulation data, the service data, the historical data and the fusion derivative data to a data processing module for processing; transmitting the real-time motion data and the historical data to a service layer;
in the embodiment, the data transmission module selects a proper transmission mode according to the collected real-time driving data, historical data, fusion derivative data, service data, simulation data and other data, so that the problems of data loss, interference, tampering, distortion and the like are avoided; the data processing module carries out corresponding preprocessing on the data, so that the accuracy, the connectivity and the timely communication of the data are ensured, and the data of a multi-platform and multi-user system can be communicated with each other; and the data fusion can be realized on the same motion platform by using a dynamic heterogeneous data fusion method.
The database module is used for storing real-time motion data, simulation data, service data, historical data and fusion derivative data;
and the data processing module is used for judging the timeliness of the real-time motion data, obtaining historical data if the time is greater than a preset value, and transmitting the historical data to the database module through the data transmission module, otherwise, not processing.
As shown in fig. 3, the method for constructing a digital three-dimensional twin of a moving object in step S2 includes the following steps:
a1, capturing motion data of a human body in a certain time period;
a2, preprocessing the action data to obtain preliminary data;
a3, respectively obtaining a shape parameter beta and an attitude parameter according to the preliminary data, and taking the shape parameter beta and the attitude parameter as the input of a human body SMPL model;
a4, the shape parameter beta and a basic template of the human body SMPL model
Superposing, mixing and forming to obtain mesh of the silent posture of the moving object; wherein the expression of the superposition is as follows:
D∈R 6890×3×10
β∈R 10
V shape ∈R 6890×3
wherein D is the deviation of the principal component, beta is the magnitude of the deviation of the principal component,
mesh, V, as a base template shape The mesh of the silent posture of the moving object is shown, R is a real number set, and the mesh is a human body network;
a5, superposing the attitude parameters and the mesh of the silent attitude of the moving object, and mixing skins to obtain a digital three-dimensional twin body of the moving object;
and A6, judging whether the error between the digital three-dimensional twin organism of the moving object and the moving object is smaller than a preset value, if so, completing the construction of the digital three-dimensional twin organism of the moving object, otherwise, adjusting the shape parameter beta and the posture parameter, and returning to the step A4.
In the embodiment, body motion data of a moving object is obtained through a somatosensory technology, data preprocessing is carried out on the motion data, a model of a skeleton key point is obtained through visualization software, a muscle model is established on a simulation platform, a digital three-dimensional twin body of the moving object is obtained, virtual motion simulation is carried out on the digital three-dimensional twin body, motion characteristics of the virtual moving object are obtained, and parameterization and digitization are carried out on the obtained motion characteristics. And judging the similarity of the moving object and the moving object digital three-dimensional twin body according to the error threshold, if the similarity is within the error range, successfully constructing the moving object digital three-dimensional twin body, and if the similarity is not within the error range, adjusting parameters, re-optimizing the parameters, and further obtaining the accurate moving object digital three-dimensional twin body.
In this embodiment, a sensor (which may be an optical marker, a 3D scan, a flexible wearable device, or the like) captures data of a certain motion of a human body over a period of time, and constructs a human body SMPL model from the collected data. The human body SMPL model has shape parameters beta, a group of shape parameters beta has values of 10 dimensions to describe the shape of a person, and the value of each dimension can be interpreted as a certain index of the shape of the person, such as height, thickness and the like; pose parameters (posi parameters), a set of pose parameters with 24 × 3 dimensions of numbers, describing the motion pose of the human body at a certain moment, 24 of which represent 24 well-defined human body joint points, 3 of which is not the (x, y, z) spatial position coordinate (location) as defined in the recognition problem, but refers to the rotation angle of the node with respect to its parent node. The training process of the human body SMPL model comprises the following steps: shape-based Blend Shapes (Shape Blend Shapes), at which stage the base template T (otherwise known as the statistically mean template) is taken as the basic pose of the whole human body, which is statistically derived, representing the whole mesh with N =6890 end points (vertex), each with three spatial coordinates (x, y, z), which are different from the bone point join; then the needed human body posture and the offset of the basic posture are described through parameters, and the final expected human body posture is formed by superposition, and the process is a linear process. Basic model set by human body SMPL modelVersion T is the mean shape obtained by counting a large number of real human body mesh. Mesh of the quiet pose is formed by linearly combining the Principal Shape Components, otherwise known as endpoint offsets (Vertex deviances), and superimposing on the base template T. The main shape component here refers to a main variation component of the mesh statistically obtained in the data set. Specifically, each principal component is a 6890 × 3 matrix, where some (x, y, z) representation is relative to the corresponding base template
The offset of the endpoint above, the whole process can be represented by the following formula:
wherein D ∈ R 6890×3×10 Is a shift of 10 principal components, beta ∈ R 10 Is the size of the 10 principal component shifts,
is a base template->
Mesh, V of shape ∈R 6890×3 And R is a real number set and the mesh is a human body network after the mixed formation.
In this embodiment, 23 × 3 parameters are required to represent the relative rotation of the non-root node with respect to the parent node, and in order to represent the global rotation (also referred to as Orientation) and the spatial displacement of the whole body motion, such as walking, running, etc., of the body, it is also necessary to define the rotation and the displacement for the root node, and then, similarly, 3 parameters are required to express the rotation in an axis angle manner, and then 3 parameters are required to express the spatial displacement. Because different human bodies have larger differences in shape, after the mixed forming, bone points conforming to the mesh still need to be estimated according to the formed mesh, so that the bone points are rotated later to form the final expected posture. After the estimation of the positions of the skeleton points, control points, namely the skeleton points, for operating the whole human digital model are obtained. When the skeleton point is rotated, the human body in a silent posture can be put into a required posture like swinging a spherical joint doll. The human mesh end points can also change along with the joint points around the human mesh end points to form a final human digital model. Therefore, the covering is actually a process of enabling the human skeleton in the silent posture to move up and covering the human skeleton with the 'skin', so that three-dimensional character modeling can be realized.
In the embodiment, muscle physiological signal data acquired by flexible wearable equipment are used for correcting the digital three-dimensional twin body of the moving object, and the digital three-dimensional twin body of the moving object is continuously optimized in an iterative manner by combining historical data and related experimental data in the moving process, so that a moving model basis is provided for the next work. Finally, a three-dimensional integrated digital model of bones, muscles and skins in the motion process is formed, and a complete motion object digital twin is formed after the human body three-dimensional digital model is constructed.
In this embodiment, the digital three-dimensional twin of the moving object is the basis for establishing a digital twin system, and the final purpose of all designs is to serve the moving object. The service layer is provided with a real-time control service module, a motion attitude display service module, a positioning module, an alarm module and a motion data monitoring service module. The real-time control service module is used for carrying out real-time motion control on the digital twin motion object, such as running, jumping and directional movement. The motion posture display service module displays the motion posture of the motion object and the motion posture of the digital three-dimensional twin body of the motion object, and the motion data monitoring service module collects motion data in real time. The service layer transmits the service data to the data layer, and the data layer contains real-time motion data, historical data, fusion derivative data, service data and simulation data of the moving object digital three-dimensional twin body. And the physical layer and the virtual layer carry out real-time data interaction and real-time mapping. Obtaining a digital three-dimensional twin body of a moving object based on a physical method and a data-driven motion synthesis method; and performing dynamic modeling according to the motion data, constructing a motion model of the motion object, performing dynamic numerical analysis in the human motion process, continuously optimizing in a virtual layer by utilizing interactive data obtained by a physical layer and the virtual layer and combining a computer rendering technology, and finally obtaining a real-time training model of the motion object.
As shown in fig. 4, the method for constructing a twin model of a motion scene in step S2 includes the following steps:
b1, collecting landmark data and climate data of a real motion scene;
b2, determining a coordinate system of the virtual motion scene, and building a motion scene twin model according to the landmark data and the climate data;
and B3, judging whether the error between the twin model of the motion scene and the actual motion scene is smaller than a preset value, if so, completing the construction of the twin model of the motion scene, otherwise, adjusting the landmark data and the climate data, and returning to the step B2.
In this embodiment, the building of the digital twin model of the motion scene refers to building a virtual motion scene according to the geographic location and the building marker of the data, such as the temperature and humidity, the air quality, the air pressure, the illumination intensity, the wind speed, the ground type, the hardness degree and the like, acquired by the physical sensor. Firstly, a coordinate system of a virtual space is determined, secondly, frames of markers such as buildings and the like are established, the virtual motion environment is adjusted according to data collected by a sensor, and finally, the virtual motion environment is rendered to form a motion scene twin model. The twin model of the motion scene is mainly a static model, and can display the altitude, the oxygen content of air, the temperature and the humidity of the motion scene in real time by combining a Beidou positioning system, whether the motion scene is an inland or coastal area, whether the motion scene is an urban or suburban area, whether the motion scene is indoor or outdoor, and whether the ground type is a cement ground, a gravel road, a soil road or a plastic field. According to related data acquired by a sensor, a digital scene is constructed on a related software platform (Open Sim, poser, 3DS MAX and anybody) by utilizing a virtual reality VR technology, and then the digital scene is rendered, and model parameters are continuously optimized to enable the digital scene to conform to a real motion scene.
As shown in fig. 5, the method for constructing the real-time training model in step S5 includes the following steps:
c1, acquiring positioning information of a moving object;
c2, backing up the real-time motion data and the positioning information to obtain comparison data;
c3, performing iterative optimization on the static training model according to the real-time motion data and the positioning information to obtain simulation data;
c4, comparing the simulation data with the comparison data to obtain an error value of the simulation data and the comparison data;
c5, judging whether the error value is smaller than a set value, if so, obtaining a simulation model, updating the static training model into the simulation model, and entering the step C6, otherwise, adjusting the real-time motion data and returning to the step C3;
and C6, judging whether the training of the moving object is finished, if so, completing the construction of the real-time training model, otherwise, returning to the step C3.
In this embodiment, the method for constructing the real-time training model is to generate a three-dimensional motion process database of the digital three-dimensional twin body by a motion editing and synthesizing technology according to motion monitoring data of the virtual-real motion object, motion attitude data of the virtual-real motion object, real-time control data of the virtual object, motion capture data in the whole motion process, and the like.
Claims (7)
1. A construction method of a virtual-real training device based on digital twins is characterized by comprising the following steps:
s1, building a digital twinning system;
s2, respectively constructing a moving object digital three-dimensional twin body and a moving scene twin model;
s3, fusing the digital three-dimensional twin body of the moving object and the twin model of the moving scene to obtain a static training model, and importing the static training model into a digital twin system;
s4, acquiring real-time motion data of a moving object in a motion process, and transmitting the real-time motion data to a digital twin system;
and S5, iteratively optimizing the static training model according to the real-time motion data to obtain a real-time training model, and realizing virtual and real synchronous training.
2. The method for constructing a digital twin-based virtual-reality training device according to claim 1, wherein the digital twin system in step S1 comprises a physical layer, a virtual layer, a service layer and a data layer;
the physical layer is used for acquiring motion data of a moving object, landmark data and weather data of a real motion scene, acquiring real-time motion data according to the motion data, the landmark data and the weather data, and respectively transmitting the real-time motion data to the data layer and the virtual layer;
the virtual layer is used for performing iterative optimization on the static training model according to the real-time motion data and the positioning information to obtain a simulation model and simulation data and construct a real-time training model; respectively transmitting the simulation data and the simulation model to a data layer and a service layer;
the service layer is used for respectively displaying a real motion process of a motion object and a virtual motion process of the digital three-dimensional twin body according to the real-time motion data and the simulation model, and simultaneously supporting a user to carry out simulation training on the digital three-dimensional twin body to obtain simulation data; the positioning information is used for acquiring the positioning information of the moving object; the monitoring system is used for monitoring the physical signs of the moving object according to the real-time movement data to obtain a monitoring result; judging whether an alarm module needs to be started or not according to the monitoring result; the system is used for asking for help to rescue workers when the physical signs of the moving object are abnormal, and sending the positioning information and the monitoring result to the rescue workers; obtaining service data according to the monitoring result, the positioning information and the simulation data, and transmitting the service data to a data layer;
the data layer is used for judging the timeliness of the real-time motion data, and if the time is larger than a preset value, historical data are obtained; storing real-time motion data, simulation data, service data, historical data and fusion derivative data; and transmitting the real-time motion data and the historical data to a service layer.
3. The method for constructing the digital twin-based virtual and actual training device according to claim 2, wherein the service layer comprises a motion data monitoring service module, a positioning module, a motion posture display service module, an alarm module and a real-time control service module;
the motion data monitoring service module is used for monitoring the physical signs of the moving object according to the real-time motion data to obtain monitoring results, and feeding the monitoring results back to the real-time control service module and the alarm module respectively;
the positioning module is used for acquiring positioning information of the moving object and respectively transmitting the positioning information to the real-time control service module, the alarm module and the virtual layer;
the motion posture display module is used for displaying a real motion process of a motion object and a virtual motion process of a digital three-dimensional twin body of the motion object according to the real-time motion data and the simulation model, and simultaneously supporting a user to perform simulation training on the digital three-dimensional twin body of the motion object according to historical data to obtain simulation data and transmit the simulation data to the real-time control service module;
the real-time control service module is used for judging whether an alarm module needs to be started or not according to the monitoring result; acquiring a simulation model and real-time motion data from a data layer, respectively transmitting the real-time motion data to a motion data monitoring service module and a motion attitude display module, and transmitting the simulation model to the motion attitude display module; controlling the posture of the digital three-dimensional twin body when performing analog training on the digital three-dimensional twin body; obtaining service data according to the monitoring result, the positioning information and the simulation data;
and the alarm module is used for asking for help to rescue workers when the physical signs of the moving object are abnormal, and sending the positioning information and the monitoring result to the rescue workers.
4. The method for constructing the digital twin-based virtual reality training device according to claim 2, wherein the data layer comprises a data transmission module, a database module and a data processing module;
the data transmission module is used for transmitting the real-time motion data, the simulation data, the service data and the historical data and the fusion derivative data generated in the operation process of the data processing module to the database module for storage; respectively transmitting the real-time motion data, the simulation data, the service data, the historical data and the fusion derivative data to a data processing module for processing; transmitting the real-time motion data and the historical data to a service layer;
the database module is used for storing real-time motion data, simulation data, service data, historical data and fusion derivative data;
and the data processing module is used for judging the timeliness of the real-time motion data, obtaining historical data if the time is greater than a preset value, and transmitting the historical data to the database module through the data transmission module, otherwise, not processing.
5. The method for constructing a digital twin-based virtual-real training device according to claim 1, wherein the method for constructing a digital three-dimensional twin of a moving object in step S2 comprises the following steps:
a1, capturing motion data of a human body in a certain time period;
a2, preprocessing the action data to obtain preliminary data;
a3, respectively obtaining a shape parameter beta and an attitude parameter according to the preliminary data, and taking the shape parameter beta and the attitude parameter as the input of a human body SMPL model;
a4, combining the shape parameter beta with a basic template of a human body SMPL model
Superposing, mixing and forming to obtain mesh of the silent posture of the moving object; wherein the expression of the superposition is as follows:
D∈R 6890×3×10
β∈R 10
V shape ∈R 6890×3
wherein D is the deviation of the principal component, beta is the magnitude of the deviation of the principal component,
mesh, V, as a base template shape The mesh of the silent posture of the moving object is shown, R is a real number set, and the mesh is a human body network;
a5, superposing the attitude parameters and the mesh of the silent attitude of the moving object, and mixing skins to obtain a digital three-dimensional twin body of the moving object;
and A6, judging whether the error between the digital three-dimensional twin organism of the moving object and the moving object is smaller than a preset value, if so, completing the construction of the digital three-dimensional twin organism of the moving object, otherwise, adjusting the shape parameter beta and the posture parameter, and returning to the step A4.
6. The method for constructing the digital twin-based virtual-real training device according to claim 1, wherein the method for constructing the motion scene twin model in step S2 comprises the following steps:
b1, collecting landmark data and climate data of a real motion scene;
b2, determining a coordinate system of the virtual motion scene, and building a motion scene twin model according to the landmark data and the climate data;
and B3, judging whether the error between the twin model of the motion scene and the actual motion scene is smaller than a preset value, if so, completing the construction of the twin model of the motion scene, otherwise, adjusting the landmark data and the climate data, and returning to the step B2.
7. The method for constructing a digital twin-based virtual-real training device according to claim 1, wherein the method for constructing the real-time training model in step S5 comprises the following steps:
c1, acquiring positioning information of a moving object;
c2, backing up the real-time motion data and the positioning information to obtain comparison data;
c3, performing iterative optimization on the static training model according to the real-time motion data and the positioning information to obtain simulation data;
c4, comparing the simulation data with the comparison data to obtain an error value of the simulation data and the comparison data;
c5, judging whether the error value is smaller than a set value, if so, obtaining a simulation model, updating the static training model into the simulation model, and entering the step C6, otherwise, adjusting the real-time motion data and returning to the step C3;
and C6, judging whether the training of the moving object is finished, if so, completing the construction of the real-time training model, otherwise, returning to the step C3.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211473502.7A CN115964933A (en) | 2022-11-21 | 2022-11-21 | Construction method of virtual and real training device based on digital twins |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202211473502.7A CN115964933A (en) | 2022-11-21 | 2022-11-21 | Construction method of virtual and real training device based on digital twins |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN115964933A true CN115964933A (en) | 2023-04-14 |
Family
ID=87360695
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202211473502.7A Pending CN115964933A (en) | 2022-11-21 | 2022-11-21 | Construction method of virtual and real training device based on digital twins |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN115964933A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116822159A (en) * | 2023-06-06 | 2023-09-29 | 郑州轻工业大学 | Digital twin workshop rapid modeling method for dynamic and static fusion of man-machine environment |
| CN117131712A (en) * | 2023-10-26 | 2023-11-28 | 南开大学 | Virtual-real combined emergency rescue simulation system and method |
-
2022
- 2022-11-21 CN CN202211473502.7A patent/CN115964933A/en active Pending
Cited By (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN116822159A (en) * | 2023-06-06 | 2023-09-29 | 郑州轻工业大学 | Digital twin workshop rapid modeling method for dynamic and static fusion of man-machine environment |
| CN116822159B (en) * | 2023-06-06 | 2024-05-03 | 郑州轻工业大学 | Digital twin workshop rapid modeling method for dynamic and static fusion of man-machine environment |
| CN117131712A (en) * | 2023-10-26 | 2023-11-28 | 南开大学 | Virtual-real combined emergency rescue simulation system and method |
| CN117131712B (en) * | 2023-10-26 | 2024-01-16 | 南开大学 | Virtual-real combined emergency rescue simulation system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP7061694B2 (en) | Image processing methods and equipment, imaging equipment, and storage media | |
| CN115964933A (en) | Construction method of virtual and real training device based on digital twins | |
| CN107349594B (en) | A kind of action evaluation method of virtual Dance System | |
| CN101579238B (en) | Human motion capture three dimensional playback system and method thereof | |
| CN1168057C (en) | Method for following and imaging a subject's three-dimensional position and orientation, method for presenting a virtual space to a subject,and systems for implementing said methods | |
| CN109011508A (en) | A kind of intelligent coach system and method | |
| WO2017210654A2 (en) | Methods and devices for assessing a captured motion | |
| CN106648116A (en) | Virtual reality integrated system based on action capture | |
| US20210349529A1 (en) | Avatar tracking and rendering in virtual reality | |
| CN201431466Y (en) | Human motion capture and thee-dimensional representation system | |
| CN107330967A (en) | Knight's athletic posture based on inertia sensing technology is caught and three-dimensional reconstruction system | |
| CN206497423U (en) | A kind of virtual reality integrated system with inertia action trap setting | |
| CN107616898B (en) | Upper limb wearable rehabilitation robot based on daily actions and rehabilitation evaluation method | |
| CN104484574A (en) | Real-time human body gesture supervised training correction system based on quaternion | |
| CN113076002A (en) | Interconnected body-building competitive system and method based on multi-part action recognition | |
| CN109815930A (en) | A kind of action imitation degree of fitting evaluation method | |
| CN113542378A (en) | Remote rehabilitation service-oriented interactive exercise training method and device, computer equipment and storage medium | |
| CN112933581A (en) | Sports action scoring method and device based on virtual reality technology | |
| CN110502107A (en) | Wearable real-time action instructs system and method | |
| Wu et al. | Research on college gymnastics teaching model based on multimedia image and image texture feature analysis | |
| CN115040753A (en) | Fire fighter high altitude psychology adaptability training system based on VR | |
| Chakravarthi et al. | Real-time human motion tracking and reconstruction using IMU sensors | |
| Ida et al. | A study of kinematic cues and anticipatory performance in tennis using computational manipulation and computer graphics | |
| Nie et al. | [Retracted] The Construction of Basketball Training System Based on Motion Capture Technology | |
| CN109297507B (en) | Human body limb movement capturing method free of alignment action based on inertial sensor |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination |