CN115098100B - Digital twin body assembly driving method oriented to industrial application scene based on cloud protogenesis - Google Patents

Digital twin body assembly driving method oriented to industrial application scene based on cloud protogenesis Download PDF

Info

Publication number
CN115098100B
CN115098100B CN202210722116.0A CN202210722116A CN115098100B CN 115098100 B CN115098100 B CN 115098100B CN 202210722116 A CN202210722116 A CN 202210722116A CN 115098100 B CN115098100 B CN 115098100B
Authority
CN
China
Prior art keywords
digital twin
data
twin body
component
assembly
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210722116.0A
Other languages
Chinese (zh)
Other versions
CN115098100A (en
Inventor
康鹏飞
李迪
张春华
王世勇
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
South China University of Technology SCUT
Original Assignee
South China University of Technology SCUT
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by South China University of Technology SCUT filed Critical South China University of Technology SCUT
Priority to CN202210722116.0A priority Critical patent/CN115098100B/en
Publication of CN115098100A publication Critical patent/CN115098100A/en
Application granted granted Critical
Publication of CN115098100B publication Critical patent/CN115098100B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/30Creation or generation of source code
    • G06F8/38Creation or generation of source code for implementing user interfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00Computer-aided design [CAD]
    • G06F30/10Geometric CAD
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F8/00Arrangements for software engineering
    • G06F8/60Software deployment
    • G06F8/65Updates
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/44Arrangements for executing specific programs
    • G06F9/445Program loading or initiating
    • G06F9/44521Dynamic linking or loading; Link editing at or after load time, e.g. Java class loading
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F9/00Arrangements for program control, e.g. control units
    • G06F9/06Arrangements for program control, e.g. control units using stored programs, i.e. using an internal store of processing equipment to receive or retain programs
    • G06F9/46Multiprogramming arrangements
    • G06F9/50Allocation of resources, e.g. of the central processing unit [CPU]
    • G06F9/5061Partitioning or combining of resources
    • G06F9/5072Grid computing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T15/003D [Three Dimensional] image rendering
    • G06T15/005General purpose rendering architectures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T17/00Three dimensional [3D] modelling, e.g. data description of 3D objects
    • G06T17/20Finite element generation, e.g. wire-frame surface description, tesselation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2200/00Indexing scheme for image data processing or generation, in general
    • G06T2200/04Indexing scheme for image data processing or generation, in general involving 3D image data

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Software Systems (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Geometry (AREA)
  • Databases & Information Systems (AREA)
  • Computer Graphics (AREA)
  • Data Mining & Analysis (AREA)
  • Computing Systems (AREA)
  • Mathematical Physics (AREA)
  • Computer Security & Cryptography (AREA)
  • Human Computer Interaction (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Computer Hardware Design (AREA)
  • Evolutionary Computation (AREA)
  • Processing Or Creating Images (AREA)

Abstract

The invention discloses a digital twin body assembly driving method facing industrial application scenes based on cloud protogenesis, which comprises the following steps: constructing a digital twin body component based on cloud protogenesis, adopting a Collada format file based on an XML frame as a data format of the digital twin body component, and storing the kinematic data and the physical data of the physical object in a mode of element definition labels; the digital twin assembly is deployed in a cloud environment through the loading class of the Collada file of the digital twin assembly, data updating is carried out on the digital twin assembly according to data source data, updated digital twin assembly data is rendered and visualized to the cloud environment, the data of the digital twin assembly are circularly updated, and real-time driving of the digital twin assembly is achieved. The invention realizes accurate digital control, supports the storage of the multidimensional data of the digital twin body assembly, and facilitates the rapid deployment, development, updating and operation and maintenance collaborative cooperation of the digital twin body assembly at the cloud.

Description

Digital twin body assembly driving method oriented to industrial application scene based on cloud protogenesis
Technical Field
The invention relates to the technical field of digital twin component driving, in particular to a digital twin component driving method based on a cloud native oriented industrial application scene.
Background
The digital twin timely intercommunicates the running state data of the physical object and the virtual object through the data channel, and the monitoring, the prediction and the control of the running state of the physical object and the evolution of the virtual object are realized. A digital twin is a digital mapping system that is identical to a real object to be expressed in a virtual environment using data information using digitizing techniques or computer structured language. The cloud protogenesis technology is beneficial to each organization to construct and operate the application which can be elastically expanded in novel dynamic environments such as public cloud, private cloud, mixed cloud and the like. Representative technologies for cloud proto include containers, service grids, micro-services, immutable infrastructure, and declarative APIs. The characteristics of cloud native applications include distributed, high availability, high performance, elasticity, stateless, local light dependencies, and the like. These techniques enable the construction of loosely coupled systems that are fault tolerant, easy to manage and easy to observe.
Through the dual driving of the multidimensional virtual model and the fusion data and the interaction of the physical object and the virtual model, the digital twin can describe the multidimensional attribute of the physical object, describe the actual behavior and the real-time state of the physical object, and analyze the future development trend of the physical object, thereby realizing the actual functional services and application requirements of monitoring, simulation, prediction, optimization and the like of the physical object. The digital twin body is a digital mapping system which is completely consistent with the actual object and is expressed by using data information in a virtual environment by using an informatization technology, and a precondition is provided for realizing the digital twin technology. However, the implementation of the digital twin technology still faces a plurality of problems, such as light weight of structural data, multi-dimensional data expression, light weight deployment, development, updating and operation and maintenance collaborative problems, and limitations of cloud deployment of the digital twin, and the digital twin cannot realize multi-device interaction, single-device precise control, complex kinematic behavior expression and conform to various physical data of actual physical devices in a web environment, so that it is difficult to provide digital infrastructure services for small and medium enterprises and universities.
For the development situation of 3D model data storage, such as common file formats like 3ds, STL, OBJ, FBX, JT, STEP, solidWorks, etc., the universal URTF file for robot simulation is stored after the 3D model is articulated, and then the data of a mesh (which we commonly call as mesh data) is used for supporting the structural expression of the model, which is mostly universal, but if the kinematic behavior expression, simulation and virtual debugging of the 3D model are required, limitations are presented, special software is required to be used for completing the above tasks, and macroscopic model multidimensional expression at the cloud environment, such as web page end, is not required to be integrated, so that the functions of simulation and virtual debugging are realized.
At present, the 3D model has limited data of various physical components of actual physical equipment which can be represented in the web, and lacks descriptions of a digital twin kinematic model, a kinematic scene, a physical model and a physical scene, so that accurate digital control of digital twin components can not be realized in a dynamic environment, and as many digital users as possible can be adopted. And the GlTF file format focuses on skeletal animation, the motion visualization display is defined in a model file, the grid data of the model and the behavior change of a single model are written, and the model driving is realized by updating the animation data. The data storage mode is almost the same as the thought provided by the invention, and various data of the digital twin body assembly are stored by adopting a computer lightweight language.
In addition, for the visualization of the digital twin body in a simulation environment and the expression of the digital twin body in a behavior expression scheme, a kinematic model and a physical model are required to be defined in special software after modeling is completed (grid data of a digital twin body component are generated at the moment), the generated digital twin body has large limitation, various data of the digital twin body are mutually independent and have large data volume, deployment and control in a cloud (web) are not supported, meanwhile, the expression of the kinematic model of the digital twin body is preset, a process of synchronizing with actual equipment in real time is not realized, and expandability and accurate digital control in running are lacked.
Disclosure of Invention
In order to overcome the defects and shortcomings of the prior art, the invention provides a digital twin body component driving method based on cloud native oriented industrial application scenes, which is used as a model driver for creating a user of an authoring tool and contents of interactive applications, stores various data information in an XML element definition (label) mode, so that the data information is freely changed under the condition of not losing the information, realizes accurate digital control, can be combined into a tool chain with rich functions through a plurality of software packages (background drivers), and releases model data from a special binary format into a clear, XML-based and free open standard format, supports storage of multi-dimensional data of the digital twin body component, and is convenient for rapid deployment, development, updating and operation and maintenance collaborative cooperation of the digital twin body component in a cloud; the method can be directly used for being integrated in the existing content tool chain, and is adopted by as many digital users as possible, and the simple integration mechanism can enable all data of the model to become the basis of general data exchange between application programs in a XML element definition mode, thereby providing services for developers, hardware and middleware manufacturers.
A second object of the invention is to provide a digital twin body assembly driving system based on cloud native oriented industrial application scenarios.
A third object of the present invention is to provide a computer-readable storage medium.
It is a fourth object of the present invention to provide a computing device.
In order to achieve the above purpose, the present invention adopts the following technical scheme:
the invention provides a digital twin body assembly driving method facing industrial application scenes based on cloud protogenesis, which comprises the following steps:
constructing a digital twin body component based on cloud protogenesis, adopting a Collada format file based on an XML frame as a data format of the digital twin body component, constructing a Collada architecture of the digital twin body component oriented to an industrial application scene, storing kinematic data and physical data of a physical object in a mode of element definition labels, and describing a reference relation among various library data and a combined hierarchical relation with other digital twin body components in a mode of identifiers;
the digital twin assembly is deployed in a cloud environment through the loading class of the Collada file of the digital twin assembly, all tag data under the digital twin assembly are traversed, the data of the digital twin assembly are updated according to the data in a data source, new homogeneous coordinates of the digital twin assembly in the cloud environment are obtained through matrix change, the updated data of the digital twin assembly are rendered, the data of the digital twin assembly are visualized to the cloud environment, the data of the digital twin assembly are continuously and circularly updated, and real-time driving of the digital twin assembly is achieved.
As a preferred embodiment, the structural data of the digital twin body assembly includes: a visual scene library, a geometric structure library, a model function library, a model material library, a kinematic model, a kinematic system library, a kinematic scene, a physical scene and a physical model;
the mesh data in the geometric model library replicates mesh data generated by the three-dimensional modeling software.
As an preferable technical solution, a visual scene is defined by a node in a visual scene library, where the node is a node in the visual scene, and the visual scene library is provided with: referencing a combined structural library, referencing a model material library, and expressing child nodes of a combined hierarchical relationship of digital twin component structural components in a visual scene;
wherein the reference incorporates a structural library to instantiate the geometric grid data into the visual scene and the reference model material library instantiates the geometric grid data into the visual scene;
the geometric structure library comprises grid data of each component structure of the model, wherein the grid data comprises structure point data information, triangular relation data and vertex data;
the model function stores material data of the digital twin body component;
the model material library comprises reference material data of each component structure of the model, and the reference material data refers to the material data stored in the model function library;
The kinematic model library defines the kinematic structure parameters of the structural components of the digital twin body assembly through joints and Link;
the kinematic system library comprises element definition label motion and element definition label kinematics, wherein the element definition label motion comprises a physical model identifier of each structure and physical system parameters of each motion center shaft, and the element definition label kinematics comprises a kinematic model identifier of each structure and information of each center shaft;
the kinematic scene library instantiates a kinematic system in the kinematic scene by defining information of parameter constraints to the referenced kinematic system.
As a preferred technical solution, the updating the data of the digital twin body assembly according to the data in the data source specifically includes:
performing motion control on the digital twin body component by combining the data source through updating all the kinematic data about the component under the Joint information element label in the kinematic model library;
combining a patch animation method, after a new data source is acquired, positioning the current digital twin body component, and searching all Joint information element labels defined as non-static under the digital twin body component, wherein the kinematic data labels need to be modified;
If the joint is judged to be a non-static joint, the data parameter variable of the joint in the data source is transmitted to the current non-static joint angle parameter and is set to be a new variable, the current variable and the supplementary animation parameter are transmitted to the digital twin body component animation method, and the grid data of the digital twin body component is updated through matrix change to obtain the new homogeneous coordinates of the digital twin body component in the cloud environment.
As an optimal technical scheme, the new homogeneous coordinates of the digital twin body component in the cloud environment are obtained through matrix change, the grid coordinate data of the digital twin body component are transformed by adopting a four-dimensional matrix, and a specific calculation formula is expressed as follows:
wherein,W ij for a four-dimensional transformation matrix, C is a scaling parameter, X, Y and Z are camera coordinate parameters in a world coordinate system respectively, and X x ,X y ,X z Three basis vectors, Y, which are X-axis vectors, respectively x ,Y y ,Y z Three basis vectors, Z, which are Y-axis vectors, respectively x ,Z y ,Z z Three basis vectors, which are Z-axis vectors, respectively;
W new =W base *W ij
W new for transforming the mesh coordinate data of the resulting new digital twin body assembly, W base Grid coordinate data for a base digital twin component.
As a preferred technical solution, the method performs rendering on the updated data of the digital twin component, and re-renders the updated grid data of the digital twin component through a renderer interface provided by a thread. Js library of encapsulated OpenGL, webGL and shaders.
In order to achieve the second object, the present invention adopts the following technical scheme:
a cloud native industry-oriented application scenario based digital twin body assembly drive system, comprising: the system comprises a digital twin body component building module, a cloud deployment module, an operation and maintenance coordination module, a data updating module and a visualization module;
the digital twin body component construction module is used for constructing a digital twin body component based on cloud protogenesis, a digital twin body component data format based on XML framework is adopted, a digital twin body component coloada framework oriented to industrial application scenes is constructed, the kinematic data and the physical data of a physical object are stored in an element definition label mode, and the reference relation among all library data and the combined hierarchical relation with other digital twin body components are described in an identifier mode;
the cloud deployment module is used for deploying the digital twin component in a cloud environment through the loading class of the Collada file of the digital twin component;
the operation and maintenance coordination module is used for realizing operation and maintenance coordination of the digital twin body assembly in a cloud environment, continuously and circularly updating data of the digital twin body assembly, and driving the digital twin body assembly through a background motion control method;
The data updating module is used for updating grid coordinate data of the digital twin body assembly in the cloud environment, traversing all tag data under the digital twin body assembly, updating the data of the digital twin body assembly according to the data in the data source, and obtaining new homogeneous coordinates of the digital twin body assembly in the cloud environment through matrix change;
the visualization module is used for realizing the visualization of the digital twin body assembly in the cloud environment, rendering the updated data of the digital twin body assembly and visualizing the data to the cloud environment.
In order to achieve the third object, the present invention adopts the following technical scheme:
a computer readable storage medium storing a program which when executed by a processor implements a digital twin component driving method for industrial application scenarios based on cloud native as described above.
In order to achieve the fourth object, the present invention adopts the following technical scheme:
a computing device comprising a processor and a memory for storing a processor executable program, the processor implementing a digital twin component driving method as described above based on a cloud native industrial application oriented scenario when executing the program stored in the memory.
Compared with the prior art, the invention has the following advantages and beneficial effects:
(1) The existing method for constructing the digital twin body is not called as a digital twin body assembly, and is characterized in that the existing concept of the digital twin body is generally a 3D mathematical model of an actual physical model, and is essentially grid data, the kinematic data and the grid data are two parts of contents, the method for constructing the digital twin body is to model by application software (such as SolidWorks, inventor, CATIA and other three-dimensional modeling software), the software automatically generates the grid data, and other information data such as the kinematic data and the physical data of a physical object are defined in other corresponding software, so that the application scope is small, the limitation is large, and related APIs are not provided to change the data information of the physical object in a cloud environment;
for other file formats supporting 3D visualization in the cloud, such as XML,3Ds, and GlTF file formats, the file formats can be generated through application software, or a developer manually writes data information such as materials, animation skins and the like, but cannot express the data information such as a kinematic model, a kinematic system, a physical model and the like of a digital twin body, and cannot fully express the complete functions and behavior components of the digital twin body assembly;
According to the method for constructing the digital twin-body component based on the Collada file format of the XML framework, other information such as kinematic data and physical data of the digital twin-body component can be written in a software development and editing environment after grid data of the digital twin-body component are generated through an application program, the method has data integrity, openness and high universality, light weight deployment of large-scale data of the digital twin-body component in a cloud environment can be met, and all digital users can update bottom data of the digital twin-body component through the application program, so that secondary development is greatly improved; the method supports data analysis and modification under the general compiling environment, and can meet the requirement of accurately expressing various data of a physical model by using computer language in the digital manufacturing field.
(2) In the prior art, a driving method of a digital twin body is generally called a simulation and virtual debugging process, in the simulation process, the digital twin body is imported into a simulation environment, after a kinematic model of the digital twin body is defined in a corresponding application program, a motion scheme at a certain production beat, such as how many angles each movable joint moves, how much motion gesture changes, speed and acceleration changes, and beat time are preset in advance in digital manufacturing application software. In the virtual debugging process, a simulation process is included, meanwhile, start-stop data of field physical equipment are collected, for example, equipment motion data from a PLC or a controller is used as a signal basis for starting and stopping a digital twin body, the motion process is set in the simulation process, accurate data control is not realized in a real sense, after the driving is defined by kinematics, start-stop signals in running are captured through simulated motion postures, and the driving of the digital twin body is realized by setting motion data and motion beats;
The driving method provided by the invention can better realize the real-time requirement in the digital manufacturing process, accurately display the real motion control process of the physical object in the manufacturing process, and is close to the software and hardware driving process of the physical object in the real environment; the simulation, real-time monitoring and motion trend prediction functions can be realized according to different data sources, different motion control methods can be developed according to different physical objects, and the application range is wide.
Drawings
FIG. 1 is a schematic flow chart of a digital twin body assembly driving method based on a cloud native oriented industrial application scene;
FIG. 2 is a schematic diagram of an element definition tag and identifier employed by the digital twinner assembly of the present invention;
FIG. 3 is a schematic view of a digital twin component Collada architecture of the present invention for industrial application scenarios;
FIG. 4 is a schematic diagram of the kinematic data transformation process of the digital twin body assembly of the present invention;
fig. 5 is a schematic diagram of a visual technology framework of a digital twin body in a cloud environment.
Detailed Description
The present invention will be described in further detail with reference to the drawings and examples, in order to make the objects, technical solutions and advantages of the present invention more apparent. It should be understood that the specific embodiments described herein are for purposes of illustration only and are not intended to limit the scope of the invention.
Example 1
As shown in fig. 1, the present embodiment provides a digital twin body assembly driving method facing industrial application scene based on cloud native, which includes the following steps:
s1: constructing a digital twin body component based on cloud protogenesis;
in the embodiment, on the basis of referring to an industrial 4.0 component, the concept of a digital twin body component is innovatively provided, the characteristics of the existing cloud resources are combined, and the digital twin body component is divided according to three aspects of structure, function and behavior by adopting the multi-view concept, so that the elasticity and the distributed advantages of the cloud primary are fully exerted. The structural view angle of the digital twin body component describes the grid data (3D grid), kinematic parameters, connecting rod parameters, joint types, joint limits, physical structure models such as speed, acceleration, materials, appearance and the like of the actual equipment of the current component and the combined hierarchical relationship with other digital twin body components, and data information such as the aggregate characteristics, material properties, kinematic models and the like of the physical components of the digital twin body component is constructed; the functional view describes an information set which can be accessed by the components under the dynamic environment, provides conditions for the interaction of data among the components, and meets the complex requirements of behavior expression; the behavioral perspectives are based on a kinematic model of the current component itself, as well as dynamic characterization of the digital twin component driven by the data source.
In this embodiment, a Collada format file based on an XML framework is adopted as a data format of a digital twin component, in a prepared software development and editing environment, structural data of the digital twin component is stored in an XML element definition (tag) mode, including data such as a visual scene library, a geometric structure library, a model function library, a model material library, a kinematic model, a kinematic system, a kinematic scene, a physical model and the like, physical object kinematic data and physical data and the like are written in an element definition tag mode, grid data in the geometric model library can copy grid data generated by three-dimensional modeling software, and in addition, a reference relationship between each library data and a combined hierarchical relationship with other digital twin components are described in an identifier mode, so that a specific data range can be accurately positioned when the digital twin component is driven, and a condition for realizing accurate digital control is realized.
As shown in FIG. 2, to solve the data association problem of digital twin component structure, function, behavior, the relationship between them is described by means of element definition tags or identifiers.
In this embodiment, for the description of the digital twin component data, the data information of each part may be placed under an element tag, for example, a kinematic model and a kinematic system may be placed under an element tag, and distinguished by a form of a sub-tag or an identifier, which is to accurately locate a certain data under the component when the digital twin component is driven, so as to achieve accurate control, and the essence of which is to distinguish various data information of the digital twin component by an element definition tag or identifier.
The method for constructing the digital twin-body component based on the Collada file format of the XML architecture provided by the invention not only inherits the advantages of free XML, development and light-weight deployment, a developer can freely enable the data of a kinematic model, a kinematic system, a kinematic scene, a physical model and the like of the digital twin-body component to be expressed in a cloud environment through an element definition (tag) mode, but also can directly update the data in the background, and meanwhile, the reference relation between each structural library data and the combined hierarchical relation with other digital twin-body components can be described through an identifier mode, so that the data information can be freely changed under the condition of not losing information, the method can be directly used for being integrated in the existing content tool chain, is adopted by a digitalized user as much as possible, is convenient for secondary development from a background developer, and can enable all data of the model to become the basis of general data exchange between application programs through the element definition mode.
As shown in fig. 3, a digital twin component Collada architecture is constructed for industrial application scenarios, wherein:
(1): the visual scene is defined by a node in the visual scene library, which is a node in the visual scene. It may contain a transformation (such as rotation or translation) that references more child nodes. It may refer to meshes and cameras, as well as skins describing mesh transformations. instance geometry is a reference-binding structure library, i.e. mesh data (mesh), which instantiates mesh data of a geometric structure to a visual scene; instance material is a library of referenced model materials, expressing the appearance of the model, instantiating mesh data of the geometry into the visual scene; the remaining child nodes express the combined hierarchical relationship of the digital twin component structural components in the visual scene.
(2): the geometry library mainly comprises grid data of each component structure of the model, namely mesh. mesh is a collection of points (also denoted by Vertex), normal vectors (Normal vectors), faces (faces) that define the shape of a 3D object. Its mesh data contains structure point data information (source), triangle relation data (triangules), vertex data (vertetics). OpenGL forms the topological structure of the Mesh by three modes of Triangle List (a Triangle is formed by every three vertexes), triangle Strip (a Triangle is formed by every two added vertexes and the former vertexes), triangle Fan (a Triangle is formed by every two added vertexes and the 0 th vertex), and forms a vertex array according to one of the three modes, so that the GPU can Draw all the triangles, and the visual basis of the digital twin body assembly in the environment is completed.
(3): the model function library is to be pointed out that the function or behavior component of the digital twin component is not written here, but the function required to be represented by the structure of the component itself is written here, and the appearance expression of the component, namely the material data of the digital twin component, can be continuously expanded according to the requirement, and the material data comprise color data of ambient light of the model and scattered light of the diffuse.
(4): the model material library contains texture data of the respective component structures of the model, where the texture data refers to texture data written in the function library.
(5) The kinematic model library defines the kinematic structure parameters of the digital twin component structure components through joints and links.
(6) The kinematic system library needs to be matched with a physical engine, is a description and expression of dynamic physical parameters of the digital twin body assembly, and mainly comprises two element definition tags of motion and kinematics. The motion element definition comprises a physical model identifier of each structure and physical system parameters of each motion center shaft, such as the motion speed and the acceleration of the center shaft; the kinematics element definition includes a kinematic model identification for each of its structures, which is referenced herein as data information in a kinematic model library, and may be instantiated into a kinematic system, and also contains information for each central axis, such as whether the central axis is a live joint or a dead joint.
(7) The kinematic scene library constrains information to the referenced kinematic system by defining parameters and then instantiates the kinematic system in the kinematic scene.
S2, deploying the digital twin body assembly in a cloud environment;
after the digital twin component is built, the method is imported into a main program which needs to be applied to cloud environment deployment through a loading class (colladaLoader) of a colladaarchitecture file of the digital twin component, the front end html is taken as an example, a new variable is inherited into the colladaLoader class in the html file, the written digital twin component colladafile address is indexed by using a load method of the class, the digital twin component is imported into the cloud environment, all tag data under the digital twin component are traversed by using a load method of the class, sub-class tags meeting the judgment of grid model data are rendered by using a renderer, and the basic behavior of the digital twin component can be controlled.
S3: the operation and maintenance of the digital twin body assembly in the cloud environment are coordinated;
after the digital twin body assembly is deployed in a cloud environment, the embodiment provides a sample of a motion control method for accessing the digital twin body assembly, the digital twin body assembly can be driven by a background motion control method, the digital twin body assembly is motion-controlled by combining a data source through updating all kinematic data about the assembly under a Joint information element label of a point Joint in a kinematic model library, the digital twin body assembly is combined with a supplementary animation method (firstly, the duration of the supplementary animation is defined, the supplementary animation indicates the duration of the digital twin body assembly moving from a current motion gesture to a next motion gesture, the motion speed of the current digital twin body assembly in the cloud environment is calculated through Joint motion speed and acceleration parameters of a kinematic system under the assembly, after a new data source is acquired, the digital twin body assembly is accurately positioned to a kinematic data label, all the Joint information element labels defined as non-static (OpenOpenface) under the digital twin body assembly are searched, if the Joint labels are non-static joints, the data parameters of the Joint in the digital twin body assembly are static variables are set to be the static parameters, the digital twin body parameters are set to be the digital coordinates of the digital twin-dimensional model assembly after the digital twin-parallel-channel is updated, and the digital coordinate-converter is rendered to the digital-parallel-coordinate-converter is provided to the digital-parallel-environment, and the digital-converter is then rendered by the digital-parallel-coordinate-converter;
According to the embodiment, the visualization step of the digital twin body assembly in the cloud environment is executed, rendering and updating are carried out, and the processes are circulated along with continuous input of the data source, so that the operation and maintenance coordination of the digital twin body assembly in the cloud environment is realized.
S4: updating grid coordinate data of the digital twin body assembly in a cloud environment;
in the present embodiment, the geometry library includes structure point data (source), triangle relationship data (triangle), and vertex data (vertices). OpenGL composes the topology of the Mesh by a Draw array mode, wherein all vertex data of triangles in the Mesh can be composed by three modes, namely a Triangle List (every three vertexes compose a Triangle), a Triangle Strip (every two added vertexes and the former two vertexes compose a Triangle), a Triangle Fan (every two added vertexes and the 0 th vertex compose a Triangle), and the Triangle array is formed by arranging one of the three modes, so that the GPU can Draw all the triangles.
As shown in fig. 4, when the background updates the kinematic data of the digital twin body according to the data of the data source, the motion gesture and the grid data of the digital twin body are correspondingly changed, and the change is completed by using a four-dimensional transformation matrix. When the kinematic data (such as coordinates, joint angle information, motion speed or acceleration information) of the digital twin body component in the cloud environment is updated, new digital twin body grid data is obtained by multiplying the new kinematic data by a four-dimensional transformation matrix such as a model matrix, a view matrix and a projection matrix, which enables a Vector3 representing one point in three-dimensional space to be converted by multiplying the matrix, such as translation, rotation, shearing, scaling, reflection, orthogonal or perspective projection and the like. The matrix is applied to the vector, namely the new visual information such as visual angles, coordinates and the like of the digital twin under the cloud environment.
For example, the four-dimensional matrix transforms the grid coordinate data of the digital twin body assembly, and is shown as follows:
W ij for a four-dimensional transformation matrix, C is a scaling parameter, X, Y and Z are camera coordinate parameters in a world coordinate system respectively, and X x ,X y ,X z Three basis vectors, Y, which are X-axis vectors, respectively x ,Y y ,Y z Three basis vectors, Z, which are Y-axis vectors, respectively x ,Z y ,Z z Three basis vectors, respectively, of the Z-axis vector.
W new =W base *W ij
W new For transforming the mesh coordinate data of the resulting new digital twin body assembly, W base Grid coordinate data for a base digital twin component.
S5: the digital twin body assembly is visualized in a cloud environment;
as shown in fig. 5, in the visualization process of the digital twin component at the cloud, the updated grid data of the digital twin component is re-rendered through a renderer render interface provided by a thread. Js library of packaged OpenGL, webGL and shaders, so as to visualize the cloud environment.
Firstly, a JavaScript binding of OpenGL ES 2.0 is added by WebGL to provide hardware 3D accelerated rendering for HTML5Canvas, so that complex navigation and data visualization can be created in a cloud environment by means of a system display card. And then drawing graphics and rectangles on the grid data of the digital twin body in a cloud environment through a canvas element of the HTML5 by combining a render interface with a renderer execution method, filling the regions with patterns and colors, and controlling each pixel by using JavaScript to realize the rendering of the digital twin body component.
In this embodiment, for the motion control method of the digital twin body component, the user may develop a custom to be specific to different devices, but basically, the data under a certain label of the digital twin body component is updated according to the data of the data source by using a data source and a motion control method (or an animation method), and then the dynamic visual update in the motion environment is realized by using a renderer method and matrix transformation.
This embodiment is not a game engine or delivery format at the time of game execution, but rather is a model driven for content creation by users of authoring tools and interactive applications, e.g. most games will use proprietary, size optimized, friendly streaming binary files. For digital twin service software developers, it is desirable to develop and test relatively simple content and test models quickly, which still needs to include advanced rendering technologies such as vertex and pixel programs (shaders), on the other hand, if the simulation, virtual debugging and real-time monitoring functions of the cloud are to be realized, the final model delivery format needs to support complex motion transformation of the model, lightweight deployment of the cloud, integrality, strong expandability, dynamic environment change, quick prototyping of the content, and precise digital control, which means that the model contains grid data of the model, behavior change of a single model, and control of each component structure of the model is also required to be supported quickly, efficiently and strongly.
According to the method, various data information is stored in an XML element definition (label) mode, so that the data information is freely changed under the condition of no information loss, accurate digital control is realized, meanwhile, a tool chain with rich functions can be formed by combining a plurality of software packages (background drivers), and the special binary format is liberated from model data by combining the digital twin body component driving method provided by the invention, so that the digital twin body component multi-dimensional data is definitely and freely opened standard format based on XML, storage of the digital twin body component multi-dimensional data is supported, and rapid deployment, development, updating and operation and maintenance collaborative cooperation of the digital twin body component multi-dimensional data in a cloud are facilitated; the method can be directly used for being integrated in the existing content tool chain, and is adopted by as many digital users as possible, and the simple integration mechanism can enable all data of the model to become the basis of general data exchange between application programs in a XML element definition mode, thereby providing services for developers, hardware and middleware manufacturers.
The driving method of the digital twin body assembly provided by the invention adopts flexible data sources, whether real-time data acquired on site or analog data from any database or optimal motion track data obtained through machine learning can be used as the driving data of the digital twin body assembly, only the data sources need to be changed in the driving method, the driving process is perfectly fitted with the driving process of real equipment, the real-time requirement of industrial application scenes can be realized, and the real-time visualization and monitoring requirements on site are met. The method can be fully applied to different types of digital twin body assemblies, and has wide application range.
Example 2
A cloud native industry-oriented application scenario based digital twin body assembly drive system, comprising: the system comprises a digital twin body component building module, a cloud deployment module, an operation and maintenance coordination module, a data updating module and a visualization module;
in this embodiment, the digital twin component building module is configured to build a digital twin component based on cloud native, and use a file in a Collada format based on an XML framework as a data format of the digital twin component, build a Collada architecture of the digital twin component oriented to an industrial application scene, store kinematic data and physical data of a physical object in a manner of element definition tags, and describe a reference relationship between each library data and a combined hierarchical relationship with other digital twin components in a manner of identifiers;
in this embodiment, the cloud deployment module is configured to deploy the digital twin component in a cloud environment through a loading class of a Collada file of the digital twin component;
in this embodiment, the operation and maintenance coordination module is configured to implement operation and maintenance coordination of the digital twin component in the cloud environment, continuously and circularly update data of the digital twin component, and drive the digital twin component through a background motion control method;
In this embodiment, the data updating module is configured to update grid coordinate data of the digital twin component in the cloud environment, traverse all tag data under the digital twin component, update the data of the digital twin component according to data in the data source, and obtain new homogeneous coordinates of the digital twin component in the cloud environment through matrix change;
in this embodiment, the visualization module is configured to implement visualization of the digital twin component in the cloud environment, render the updated data of the digital twin component, and visualize the data to the cloud environment.
Example 3
The present embodiment provides a storage medium, which may be a storage medium such as a ROM, a RAM, a magnetic disk, or an optical disk, where the storage medium stores one or more programs, and when the programs are executed by a processor, the digital twin body component driving method based on the cloud native industrial application scenario of embodiment 1 is implemented.
Example 4
The present embodiment provides a computing device, which may be a desktop computer, a notebook computer, a smart phone, a PDA handheld terminal, a tablet computer, or other terminal devices with display functions, where the computing device includes a processor and a memory, where the memory stores one or more programs, and when the processor executes the programs stored in the memory, the digital twin body component driving method based on the cloud native facing industrial application scenario of embodiment 1 is implemented.
The above examples are preferred embodiments of the present invention, but the embodiments of the present invention are not limited to the above examples, and any other changes, modifications, substitutions, combinations, and simplifications that do not depart from the spirit and principle of the present invention should be made in the equivalent manner, and the embodiments are included in the protection scope of the present invention.

Claims (7)

1. The digital twin body assembly driving method based on the cloud primary oriented industrial application scene is characterized by comprising the following steps of:
constructing a digital twin body component based on cloud protogenesis, adopting a Collada format file based on an XML frame as a data format of the digital twin body component, constructing a Collada architecture of the digital twin body component oriented to an industrial application scene, storing kinematic data and physical data of a physical object in a mode of element definition labels, and describing a reference relation among various library data and a combined hierarchical relation with other digital twin body components in a mode of identifiers;
the digital twin assembly is deployed in a cloud environment through the loading class of the Collada file of the digital twin assembly, all tag data under the digital twin assembly are traversed, the data of the digital twin assembly are updated according to the data in a data source, new homogeneous coordinates of the digital twin assembly in the cloud environment are obtained through matrix change, the updated data of the digital twin assembly are rendered, the data of the digital twin assembly are visualized to the cloud environment, the data of the digital twin assembly are continuously and circularly updated, and real-time driving of the digital twin assembly is achieved;
The updating of the data of the digital twin body assembly according to the data in the data source comprises the following specific steps:
performing motion control on the digital twin body component by combining the data source through updating all the kinematic data about the component under the Joint information element label in the kinematic model library;
combining a patch animation method, after a new data source is acquired, positioning the current digital twin body component, and searching all Joint information element labels defined as non-static under the digital twin body component, wherein the kinematic data labels need to be modified;
if the joint is judged to be a non-static joint, transmitting a data parameter variable of the joint in a data source to a current non-static joint angle parameter, setting the joint as a new variable, transmitting the current variable and a supplementary animation parameter to a digital twin body component animation method, and updating grid data of the digital twin body component through matrix change to obtain a new homogeneous coordinate of the digital twin body component in a cloud environment;
the new homogeneous coordinates of the digital twin body component in the cloud environment are obtained through matrix change, the grid coordinate data of the digital twin body component are transformed by adopting a four-dimensional matrix, and a specific calculation formula is expressed as follows:
Wherein W is ij For a four-dimensional transformation matrix, C is a scaling parameter, X, Y and Z are camera coordinate parameters in a world coordinate system respectively, and X x ,X y ,X z Three basis vectors, Y, which are X-axis vectors, respectively x ,Y y ,Y z Three basis vectors, Z, which are Y-axis vectors, respectively x ,Z y ,Z z Three basis vectors, which are Z-axis vectors, respectively;
W new =W base *W ij
W new for transforming the mesh coordinate data of the resulting new digital twin body assembly, W basr Grid coordinate data for a base digital twin component.
2. The cloud-native industrial application scenario-based digital twin assembly driving method of claim 1, wherein the structural data of the digital twin assembly comprises: a visual scene library, a geometric structure library, a model function library, a model material library, a kinematic model, a kinematic system library, a kinematic scene, a physical scene and a physical model;
the mesh data in the geometric model library replicates mesh data generated by the three-dimensional modeling software.
3. The method for driving a digital twin body component based on a cloud native industrial application scene according to claim 2, wherein a visual scene is defined by a node in a visual scene library, the node is a node in the visual scene, and the visual scene library is provided with: referencing a combined structural library, referencing a model material library, and expressing child nodes of a combined hierarchical relationship of digital twin component structural components in a visual scene;
Wherein the reference incorporates a structural library to instantiate the geometric grid data into the visual scene and the reference model material library instantiates the geometric grid data into the visual scene;
the geometric structure library comprises grid data of each component structure of the model, wherein the grid data comprises structure point data information, triangular relation data and vertex data;
the model function stores material data of the digital twin body component;
the model material library comprises reference material data of each component structure of the model, and the reference material data refers to the material data stored in the model function library;
the kinematic model library defines the kinematic structure parameters of the structural components of the digital twin body assembly through joints and Link;
the kinematic system library comprises element definition label motion and element definition label kinematics, wherein the element definition label motion comprises a physical model identifier of each structure and physical system parameters of each motion center shaft, and the element definition label kinematics comprises a kinematic model identifier of each structure and information of each center shaft;
the kinematic scene library instantiates a kinematic system in the kinematic scene by defining information of parameter constraints to the referenced kinematic system.
4. The cloud native and industrial application scene-oriented digital twin component driving method according to claim 1, wherein the updated digital twin component data is rendered, and the updated digital twin component grid data is re-rendered through a renderer render interface provided by a thread js library of packaged OpenGL, webGL and shaders.
5. A digital twin body assembly driving system based on a cloud primary oriented industrial application scene, comprising: the system comprises a digital twin body component building module, a cloud deployment module, an operation and maintenance coordination module, a data updating module and a visualization module;
the digital twin body component construction module is used for constructing a digital twin body component based on cloud protogenesis, a digital twin body component data format based on XML framework is adopted, a digital twin body component coloada framework oriented to industrial application scenes is constructed, the kinematic data and the physical data of a physical object are stored in an element definition label mode, and the reference relation among all library data and the combined hierarchical relation with other digital twin body components are described in an identifier mode;
The cloud deployment module is used for deploying the digital twin component in a cloud environment through the loading class of the Collada file of the digital twin component;
the operation and maintenance coordination module is used for realizing operation and maintenance coordination of the digital twin body assembly in a cloud environment, continuously and circularly updating data of the digital twin body assembly, and driving the digital twin body assembly through a background motion control method;
the data updating module is used for updating grid coordinate data of the digital twin body assembly in the cloud environment, traversing all tag data under the digital twin body assembly, updating the data of the digital twin body assembly according to the data in the data source, and obtaining new homogeneous coordinates of the digital twin body assembly in the cloud environment through matrix change;
the updating the data of the digital twin body assembly according to the data in the data source specifically comprises the following steps:
performing motion control on the digital twin body component by combining the data source through updating all the kinematic data about the component under the Joint information element label in the kinematic model library;
combining a patch animation method, after a new data source is acquired, positioning the current digital twin body component, and searching all Joint information element labels defined as non-static under the digital twin body component, wherein the kinematic data labels need to be modified;
If the joint is judged to be a non-static joint, transmitting a data parameter variable of the joint in a data source to a current non-static joint angle parameter, setting the joint as a new variable, transmitting the current variable and a supplementary animation parameter to a digital twin body component animation method, and updating grid data of the digital twin body component through matrix change to obtain a new homogeneous coordinate of the digital twin body component in a cloud environment;
the new homogeneous coordinates of the digital twin body component in the cloud environment are obtained through matrix change, the grid coordinate data of the digital twin body component are transformed by adopting a four-dimensional matrix, and a specific calculation formula is expressed as follows:
wherein W is ij For a four-dimensional transformation matrix, C is a scaling parameter, X, Y and Z are camera coordinate parameters in a world coordinate system respectively, and X x ,X y ,X z Three basis vectors, Y, which are X-axis vectors, respectively x ,Y y ,Y z Three basis vectors, Z, which are Y-axis vectors, respectively x ,Z y ,Z z Three basis vectors, which are Z-axis vectors, respectively;
W new =W base *W ij
W new for transforming the mesh coordinate data of the resulting new digital twin body assembly, W base Grid coordinate data for the base digital twin component;
the visualization module is used for realizing the visualization of the digital twin body assembly in the cloud environment, rendering the updated data of the digital twin body assembly and visualizing the data to the cloud environment.
6. A computer readable storage medium storing a program, wherein the program when executed by a processor implements the cloud native industrial application scenario-based digital twin component driving method according to any one of claims 1-4.
7. A computing device comprising a processor and a memory for storing a program executable by the processor, wherein the processor, when executing the program stored in the memory, implements the cloud native industry-oriented digital twin component driving method according to any one of claims 1-4.
CN202210722116.0A 2022-06-24 2022-06-24 Digital twin body assembly driving method oriented to industrial application scene based on cloud protogenesis Active CN115098100B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210722116.0A CN115098100B (en) 2022-06-24 2022-06-24 Digital twin body assembly driving method oriented to industrial application scene based on cloud protogenesis

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210722116.0A CN115098100B (en) 2022-06-24 2022-06-24 Digital twin body assembly driving method oriented to industrial application scene based on cloud protogenesis

Publications (2)

Publication Number Publication Date
CN115098100A CN115098100A (en) 2022-09-23
CN115098100B true CN115098100B (en) 2024-03-26

Family

ID=83292060

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210722116.0A Active CN115098100B (en) 2022-06-24 2022-06-24 Digital twin body assembly driving method oriented to industrial application scene based on cloud protogenesis

Country Status (1)

Country Link
CN (1) CN115098100B (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115314499B (en) * 2022-10-10 2023-01-24 国网浙江省电力有限公司嵊州市供电公司 Multi-terminal cooperative working method and system suitable for electric power field
CN115686464B (en) * 2023-01-03 2023-03-21 山东矩阵软件工程股份有限公司 Multi-dimensional data space-based functional module management method and system

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008026003A2 (en) * 2006-09-01 2008-03-06 Visual Technology Services Ltd 3 d pdf document generator and method for generating 3d pdf documents
CN110992452A (en) * 2019-12-13 2020-04-10 张恒 Skill trisection method human body dynamic structure drawing system software
CN113673948A (en) * 2021-08-02 2021-11-19 江苏杰瑞信息科技有限公司 Industrial Internet-based digital twin foundation development platform
CN113703407A (en) * 2021-08-29 2021-11-26 河海大学 Method, system and equipment for constructing robot production line operating system based on digital twin

Family Cites Families (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR102546065B1 (en) * 2020-10-30 2023-06-22 주식회사 플럭시티 Realization of digital twin using xml parsing of building information modeling and energy visualization system using thereof

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2008026003A2 (en) * 2006-09-01 2008-03-06 Visual Technology Services Ltd 3 d pdf document generator and method for generating 3d pdf documents
CN110992452A (en) * 2019-12-13 2020-04-10 张恒 Skill trisection method human body dynamic structure drawing system software
CN113673948A (en) * 2021-08-02 2021-11-19 江苏杰瑞信息科技有限公司 Industrial Internet-based digital twin foundation development platform
CN113703407A (en) * 2021-08-29 2021-11-26 河海大学 Method, system and equipment for constructing robot production line operating system based on digital twin

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
遥操作机器人运动的实时三维监控方法研究;曹文明;王耀南;印峰;伍锡如;夏汉民;;仪器仪表学报;20100415(第04期);全文 *

Also Published As

Publication number Publication date
CN115098100A (en) 2022-09-23

Similar Documents

Publication Publication Date Title
CN115098100B (en) Digital twin body assembly driving method oriented to industrial application scene based on cloud protogenesis
US7676356B2 (en) System, method and data structure for simulated interaction with graphical objects
CN107016170A (en) A kind of LED lamp three-dimensional customization emulation mode based on WebGL
Bijl et al. Advanced 3D visualization for simulation using game technology
CN111476882A (en) Browser-oriented robot virtual graphic modeling method
CN113066160B (en) Method for generating scene data of indoor mobile robot
Pütz et al. Tools for visualizing, annotating and storing triangle meshes in ROS and RViz
CN112184857B (en) Data generation system
Kamat et al. Scene graph and frame update algorithms for smooth and scalable 3D visualization of simulated construction operations
Roach et al. Computer aided drafting virtual reality interface
Conlan The blender python API: Precision 3D modeling and add-on development
US11625900B2 (en) Broker for instancing
Dong et al. Real-time large crowd rendering with efficient character and instance management on GPU
Raje Evaluation of ros and gazebo simulation environment using turtlebot3 robot
US20210241540A1 (en) Applying Non-Destructive Edits To Nested Instances For Efficient Rendering
Guo et al. Exploration of human-computer interaction system for product design in virtual reality environment based on computer-aided technology
Edelkamp et al. Surface inspection via hitting sets and multi-goal motion planning
Whyte et al. Visualization and information: a building design perspective
Ruffaldi et al. Coco-a framework for multicore visuo-haptics in mixed reality
Horváth et al. A use case of the simulation-based approach to mobile robot algorithm development
Buonocore et al. Digital Twin, from Industry to Cultural Heritage
Snyder ObjectSim-A Reusable Object Oriented DIS Visual Simulation
Kłodkowski et al. Simulating human motion using Motion Model Units–example implementation and usage
Hu et al. Construction of virtual interactive assembly system
Huang 3D Animation Simulation Based on Computer Virtual Simulation Technology

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant