WO2022134775A1 - 数字孪生模型的运行方法、装置和电子设备 - Google Patents

数字孪生模型的运行方法、装置和电子设备 Download PDF

Info

Publication number
WO2022134775A1
WO2022134775A1 PCT/CN2021/124492 CN2021124492W WO2022134775A1 WO 2022134775 A1 WO2022134775 A1 WO 2022134775A1 CN 2021124492 W CN2021124492 W CN 2021124492W WO 2022134775 A1 WO2022134775 A1 WO 2022134775A1
Authority
WO
WIPO (PCT)
Prior art keywords
digital twin
model
blueprint
twin model
physical
Prior art date
Application number
PCT/CN2021/124492
Other languages
English (en)
French (fr)
Inventor
黄晓庆
马世奎
张站朝
Original Assignee
达闼机器人股份有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 达闼机器人股份有限公司 filed Critical 达闼机器人股份有限公司
Publication of WO2022134775A1 publication Critical patent/WO2022134775A1/zh

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F30/00—Computer-aided design [CAD]
    • G06F30/20—Design optimisation, verification or simulation
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/30—Semantic analysis
    • G—PHYSICS
    • G10—MUSICAL INSTRUMENTS; ACOUSTICS
    • G10L—SPEECH ANALYSIS TECHNIQUES OR SPEECH SYNTHESIS; SPEECH RECOGNITION; SPEECH OR VOICE PROCESSING TECHNIQUES; SPEECH OR AUDIO CODING OR DECODING
    • G10L15/00—Speech recognition
    • G10L15/26—Speech to text systems
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2111/00—Details relating to CAD techniques
    • G06F2111/10—Numerical modelling
    • Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
    • Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
    • Y02P—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN THE PRODUCTION OR PROCESSING OF GOODS
    • Y02P90/00—Enabling technologies with a potential contribution to greenhouse gas [GHG] emissions mitigation
    • Y02P90/02—Total factory control, e.g. smart factories, flexible manufacturing systems [FMS] or integrated manufacturing systems [IMS]

Definitions

  • Embodiments of the present invention relate to the field of robotics, and in particular, to a method, device and electronic device for running a digital twin model.
  • cloud robots have been more and more widely used. Among them, in some application scenarios that are dangerous, dirty, repetitive and difficult to implement, the requirements for cloud robots are also higher, resulting in a market demand for intelligent robots that can replace humans in function.
  • an embodiment of the present invention provides a method for running a digital twin model, where the digital twin model is a physical model with the same physical attributes as a physical robot, and the method includes:
  • the business behavior blueprint drives the behavioral interaction between the digital twin model and the digital twin world or other digital twin models, so as to synchronously control the physical robot to perform behavioral interaction with the physical environment or other physical robots.
  • the business behavior blueprint is presented in the form of a behavior script, and the business behavior blueprint includes the behavior logic of the digital twin model, the behavior logic is defined in the packaged digital twin model, and the behavior logic Used to drive the behavioral interaction of the digital twin model with the environment or other digital twin models.
  • the business behavior blueprint drives the behavioral interaction between the digital twin model and the environment or other digital twin models, so as to synchronously control the behavioral interaction between the physical robot and the physical environment or other physical robots, including:
  • the digital twin model receives control instructions
  • the business behavior blueprint parses the control instruction to obtain one or more behaviors matching the control instruction
  • the business behavior blueprint controls the digital twin model to execute the one or more behaviors, so as to synchronously control the physical robot to execute the one or more behaviors.
  • the business behavior blueprint parses the control instruction to obtain one or more behaviors matching the control instruction, including:
  • the business behavior blueprint analyzes the intention of the control instruction, and determines, according to the type of the entity robot corresponding to the digital twin model, one or more actions that the digital twin model needs to execute that matches the control instruction.
  • the business behavior blueprint supports an overloading mechanism
  • the method further includes:
  • the type of the entity robot corresponding to the digital twin model determine whether the action of the entity robot is the same as that of the general-purpose robot when performing the actions;
  • the controlling the digital twin to perform the one or more actions includes:
  • the method also includes:
  • the business behavior blueprint supports an inheritance mechanism, wherein the business behavior blueprint of the child model in the digital twin model can inherit the business behavior blueprint of the parent model.
  • the method also includes:
  • the sensor of the entity robot is simulated to obtain a second model including the sensor model
  • Embodiments of the present invention also provide a device for realizing a digital twin model, wherein the digital twin model is a physical model identical to the physical properties of an entity robot, and the device includes:
  • a packaging module for packaging the digital twin model and the business behavior blueprint of the digital twin model to obtain a packaged digital twin model, wherein the packaged digital twin model includes the business behavior blueprint;
  • a loading module for loading the packaged digital twin model and running the business behavior blueprint
  • a driving module configured to drive the digital twin model to perform behavior interaction with the digital twin world or other digital twin models through the business behavior blueprint, so as to synchronously control the physical robot to perform behavior interaction with the physical environment or other physical robots.
  • An embodiment of the present invention further provides an electronic device, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface communicate with each other through the communication bus;
  • the memory is used to store at least one executable instruction, and the executable instruction enables the processor to execute the operations of the above-mentioned method for running the digital twin model.
  • An embodiment of the present invention further provides a computer-readable storage medium, where at least one executable instruction is stored in the storage medium, and when the executable instruction is executed on the electronic device, the electronic device executes the digital twin as described above. The operation of the model's run method.
  • the business behavior blueprint and the digital twin model are packaged together.
  • the digital twin model When the digital twin model is loaded, the business behavior blueprint will run dynamically, so that the business behavior blueprint drives the digital twin model to interact with the digital twin world or other digital twin models. Behavioral interaction to synchronously control the physical robot to perform behavioral interaction with the physical environment or other physical robots.
  • Fig. 1 is the application schematic diagram of the operation method of the digital twin model provided by the embodiment of the present invention
  • FIG. 2 is a flowchart of a method for operating a digital twin model provided by an embodiment of the present invention
  • FIG. 3 is a schematic structural diagram of an operating device for a digital twin model provided by an embodiment of the present invention.
  • FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.
  • FIG. 1 is an application schematic diagram of a method for running a digital twin model provided by an embodiment of the present invention.
  • the method is applied to cloud servers.
  • the communication between the cloud server 10 and the physical robot 20 is performed through a dedicated network 30 .
  • Various robot services are trained on the cloud server 10, and the cloud server 10 controls the physical robot 20 to execute the trained robot services.
  • robot service refers to performing preset actions in different application scenarios to complete preset functions, such as welcome reception, mobile grabbing, security patrols, and distribution.
  • a service needs to be composed of an application, and a number of skills are combined into the logic of the application.
  • playing ping-pong, cutting the ball, pulling the ball, etc. is a skill
  • the application refers to the actual playing of ping-pong by the physical robot.
  • the service refers to the service that the physical robot can provide the service of playing table tennis sparring.
  • the physical robot grasping items is a skill, and the physical robot can use the grasping skills to complete the application of delivering coffee to people, and the physical robot can complete the reception service of serving tea and pouring water.
  • FIG. 2 is a flowchart of a method for running a digital twin model provided by an embodiment of the present invention. As shown in Figure 2, the method includes the following steps:
  • Step 201 Package the digital twin model and the business behavior blueprint of the digital twin model to obtain a packaged digital twin model, wherein the packaged digital twin model includes a business behavior blueprint.
  • the digital twin model is a physical model with the same physical properties as the physical robot, which can also be called a digital twin.
  • a digital twin model that maps physical characteristics one by one for the physical robot, and constructing a digital twin world that is a virtual mirror of the physical world where the physical robot is located, the digital twin model is used to train robot skills and applications in the digital twin world.
  • the digital twin model synchronously controls the behavior of the physical robot.
  • the multi-source data collected by various sensors of the physical robot that is, the acquisition of environmental change data
  • the digital twin world is a 3D semantic map data service that is a virtual mirror of the physical world where the physical robot lives. It is a digital representation of the 3D environmental semantics that the physical robot can recognize and understand in various application scenarios, helping the robot to perceive and recognize the physical world. Provide an interactive digital semantic environment for the real-time online operation service of the robot in the cloud server. The digital twin world is also used to train various digital twin models in the background (offline) to ensure that the physical robot has the best operating strategy and behavior and actions when it goes online.
  • a digital twin can be built in the following ways:
  • Step a1 Perform geometric appearance modeling and joint simulation on the shape, structure and appearance of the entity robot to obtain a first model.
  • Step a2 simulating the sensor of the physical robot on the basis of the first model to obtain a second model including the sensor model.
  • Step a3 Perform physical simulation on the physical properties of the physical robot on the basis of the second model to obtain a digital twin model.
  • the constructed digital twin model is not only consistent with the physical robot in geometric appearance, but also consistent with the dynamic control model and spatial position. In addition, some control capability interfaces of the physical robot are also simulated and provided on the digital twin model.
  • the business behavior blueprint is presented in the form of a behavior script file.
  • the business behavior blueprint includes the behavior logic of the digital twin model.
  • the behavior logic is defined in the packaged digital twin model.
  • the behavior logic is used to drive the digital twin model and the environment or other digital twins. Models interact with behavior.
  • the business behavior blueprint also collects information about the state of the physical robot and its surrounding physical environment, and updates the digital twin model and the digital twin world synchronously. In this way, the packaged digital twin model is no longer pure data, but a model with behavioral logic.
  • a digital twin model can correspond to multiple business behavior blueprints, and each business behavior blueprint can include one or more behaviors.
  • a behavior can consist of one or more actions.
  • Unreal Engine can be used as the digital twin rendering and collaborative processing engine, and the Unreal Pack tool can be used to package the digital twin model and business behavior blueprint. Blueprints are essentially a visual way of programming. After the digital twin model contains the business behavior blueprint, it will become an active agent, actively sensing environmental changes and giving feedback in the virtual world. That is, by running the business behavior blueprint, the GameAI capability of the UE engine can be called to detect the surrounding environment, such as sending rays to detect whether there are other robots ahead, so as to further determine how to execute the behavior of the digital twin model in the digital twin world.
  • the packaged digital twin model can be distributed and shared through the robot development platform.
  • Step 202 Load the packaged digital twin model and run the business behavior blueprint.
  • the business behavior blueprints packaged with it are loaded and up and running.
  • the digital twin model containing the business behavior blueprint can be dynamically loaded in the form of the Unreal Pak resource package. Since the business behavior blueprint is not associated or bound with the digital twin model, it is equivalent to an empty script that is not executable. Therefore, when the resource is loaded successfully, the business behavior blueprint will be bound to the digital twin model, so that the business behavior blueprint will be instantiated, can be executed, and is ready to receive engine calls to events and interfaces. Among them, the data perceived by the entity robot will be converted into blueprint events, which are monitored and processed by the business behavior blueprint.
  • Step 203 The business behavior blueprint drives the behavioral interaction between the digital twin model and the digital twin world or other digital twin models, so as to synchronously control the behavioral interaction between the physical robot and the physical environment or other physical robots.
  • the behavioral interaction between the digital twin model and the digital twin world or other digital twin models mainly means that the digital twin model performs some behaviors in the digital twin world to realize its service functions.
  • the physical robot obtains the user's voice command in the physical environment, synchronizes the voice command to the digital twin model, and the business behavior blueprint analyzes the voice command to determine some actions that the digital twin model needs to perform, and then controls the digital twin model to perform these actions. , and synchronously control the physical robot to perform the same behavior.
  • step 203 further includes the following steps:
  • Step a1 The digital twin model receives control instructions.
  • control instruction can be an instruction issued by the user in the physical environment, for example, the user says "I'm thirsty" to the physical robot; or the physical robot automatically determines the instruction to be executed through other surrounding environment data obtained by the sensor, such as the doorbell rings, The sensor of the physical robot acquires the sound of the doorbell and determines that it is necessary to open the door; or it is an instruction sent by the cloud server during the training of the skills and applications of the digital twin model.
  • Step a2 The business behavior blueprint parses the control instruction to obtain one or more behaviors matching the control instruction.
  • parsing includes direct semantic analysis and intent analysis.
  • the control instruction can directly include the behavior that needs to be performed by the physical robot. For example, if the user says to the physical robot, "please come to the kitchen, bring the mineral water on the table and pass it to me", the control instruction directly includes the behavior that needs to be executed by the physical robot. Three actions: 1. Go to the kitchen, 2. Bring the mineral water on the table, 3. Hand it to the user. At this time, a direct semantic analysis can obtain the behavior matching the control instruction.
  • the control instruction may also not directly include the behavior that needs to be performed by the physical robot, but only a random statement of the user. If the control instruction does not directly include the behavior that needs to be performed by the physical robot, a business behavior blueprint is required to analyze the intention of the control instruction to determine the behavior that the physical robot needs to perform. For example, if the user says "I'm thirsty" to the physical robot, the control instruction does not directly include the behavior that needs to be performed by the physical robot. At this time, the business behavior blueprint needs to be analyzed by intent, and the following three behaviors need to be performed to obtain the physical robot: 1. Go Go to the kitchen, 2. bring the mineral water on the table, 3. hand it to the user.
  • the action can be one or more actions.
  • the above examples are all cases of multiple behaviors.
  • the following is a case where there is only one behavior. For example, if the user and the physical robot say "let's shake hands", the physical robot only needs to perform the behavior of shaking hands with the user.
  • the business behavior blueprint analyzes the intent of the control instruction, and can also determine one or more behaviors that the digital twin model needs to execute that matches the control instruction according to the type of the entity robot corresponding to the digital twin model.
  • the types of physical robots can include welcome robots, security robots, housework robots, companion robots, and so on. For example, if the control command is "greeting", for a physical robot with only a screen and a body but no hands, the determined behavior is nodding on the screen; for a physical robot with not only a head but also hands, its determined behavior is The behaviors are nodding and waving; for a physical robot with not only a head, but also hands and facial expressions, the determined behaviors are nodding, waving, and smiling.
  • the business behavior blueprint can perform differential processing on behavior determination according to the characteristics of the entity robot corresponding to its own digital twin model, and the difference can be defined in advance in the business behavior blueprint. Therefore, since the digital twin model contains the business behavior blueprint, it can realize the adaptation to the differences of different physical robots, and can realize the adaptation of standard action instructions or control data based on the ability differences of the physical robots, so as to realize the use of a unified control interface to achieve Differentiated control performance.
  • Step a3 The business behavior blueprint controls the digital twin model to execute one or more behaviors, so as to synchronously control the physical robot to execute one or more behaviors.
  • the digital twin model can be controlled to execute one or more behaviors to synchronously control the physical robot to execute one or more behaviors.
  • the business behavior blueprint supports inheritance and/or overloading mechanisms.
  • Unreal Engine has an inheritance and/or overloading blueprint mechanism, which is similar to class inheritance and function overloading in object-oriented programming.
  • the business behavior blueprint of the child model in the digital twin model can inherit the business behavior blueprint of the parent class model, so that the properties and methods of the parent class blueprint can be inherited into the child model, and the mesh (model's mesh) of the child model can be inherited. storage file) can be replaced and updated to the mesh of the parent model, thus achieving model inheritance.
  • the blueprint of the submodel can also redefine and append properties and methods. For example, both a chassis mobile robot model and a four-wheeled robot model can be inherited from a more general or more capable humanoid robot model, and a robot model with fewer joints can be inherited from a robot model with more joints.
  • the digital twin model can call the business behavior blueprint faster. It can directly call the corresponding business behavior blueprint without traversing all the business behavior blueprints, which speeds up the response to events.
  • Overloading means that the child model can redefine the function defined by the same method name of the parent model. For example, for the behavior of walking from place A to place B, for ordinary robots, you can directly perform the walking action; for security robots, according to the type of the robot, its business behavior blueprint needs to overload the behavior, except for the walking action. It is also necessary to add the patrol action of shaking his head left and right.
  • This method of redefining the behavior (or function) of the same name is called overloading, that is, different actions can be performed for the same behavior through overloading. Therefore, in some embodiments, the method further includes:
  • Step b1 For the determined one or more actions, according to the type of the entity robot corresponding to the digital twin model, determine whether the action of the entity robot is the same as that of the general-purpose robot when performing the action.
  • Step b2 If different, overload the behavior.
  • controlling the digital twin model to execute one or more behaviors includes: controlling the digital twin model to execute reloaded behaviors.
  • the method further includes:
  • Step c1 The business behavior blueprint detects the digital twin world around the digital twin model, and obtains object information around the digital twin model in the digital twin world.
  • Step c2 Control the real-time action change or movement route change of the behavioral interaction between the digital twin model and the environment or other digital twin models according to the object information.
  • the business behavior blueprint and the digital twin model are packaged together.
  • the digital twin model When the digital twin model is loaded, the business behavior blueprint will run dynamically, so that the business behavior blueprint drives the digital twin model to interact with the digital twin world or other digital twin models. , to synchronously control the physical robot to interact with the physical environment or other physical robots.
  • FIG. 3 is a schematic structural diagram of an apparatus for running a digital twin model provided by an embodiment of the present invention.
  • the digital twin model is a physical model with the same physical properties as the physical robot, and the device 300 includes:
  • the packaging module 301 is configured to package the digital twin model and the business behavior blueprint of the digital twin model to obtain a packaged digital twin model, wherein the packaged digital twin model includes the business behavior blueprint.
  • the loading module 302 is configured to load the packaged digital twin model and run the business behavior blueprint.
  • the driving module 303 is used for the business behavior blueprint to drive the digital twin model to perform behavioral interaction with the digital twin world or other digital twin models, so as to synchronously control the physical robot to perform behavioral interaction with the physical environment or other physical robots.
  • the business behavior blueprint is presented in the form of a behavior script file, the business behavior blueprint includes the behavior logic of the digital twin model, and the packaged digital twin model defines the Behavioral logic, the behavioral logic is used to drive the behavioral interaction of the digital twin model with the environment or other digital twin models.
  • the business behavior blueprint drives the digital twin model to perform behavioral interaction with the environment or other digital twin models, so as to synchronously control the physical robot to perform behavioral interaction with the physical environment or other physical robots, including: :
  • the digital twin model receives control instructions.
  • the business behavior blueprint parses the control instruction to obtain one or more behaviors matching the control instruction.
  • the business behavior blueprint root controls the digital twin model to execute the one or more behaviors, so as to synchronously control the physical robot to execute the one or more behaviors.
  • the business behavior blueprint parses the control instruction to obtain one or more behaviors matching the control instruction, including:
  • the business behavior blueprint analyzes the intention of the control instruction, and determines, according to the type of the entity robot corresponding to the digital twin model, one or more actions that the digital twin model needs to execute that matches the control instruction.
  • the business behavior blueprint supports an overload mechanism
  • the driver module 303 is further configured to:
  • the action of the entity robot is the same as that of the general-purpose robot when performing the actions.
  • the business behavior blueprint supports an inheritance mechanism, wherein the business behavior blueprint of the child model in the digital twin model can inherit the business behavior blueprint of the parent model.
  • the apparatus 300 further includes a modeling module for:
  • a first model is obtained by performing geometric appearance modeling and joint simulation on the shape, structure and appearance of the entity robot.
  • the sensor of the physical robot is simulated to obtain a second model including the sensor model.
  • the business behavior blueprint and the digital twin model are packaged together.
  • the digital twin model When the digital twin model is loaded, the business behavior blueprint will run dynamically, so that the business behavior blueprint drives the digital twin model to interact with the digital twin world or other digital twin models. Behavioral interaction to synchronously control the physical robot to perform behavioral interaction with the physical environment or other physical robots.
  • FIG. 4 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention.
  • the specific embodiment of the present invention does not limit the specific implementation of the electronic device.
  • the electronic device may include: a processor (processor) 402 , a communication interface (Communications Interface) 404 , a memory (memory) 406 , and a communication bus 408 .
  • processor processor
  • Communication interface Communication Interface
  • memory memory
  • the processor 402 , the communication interface 404 , and the memory 406 communicate with each other through the communication bus 408 .
  • the communication interface 404 is used for communicating with network elements of other devices such as clients or other servers.
  • the processor 402 is configured to execute the program 410, and specifically may execute the relevant steps in the foregoing embodiments of the method for running the digital twin model.
  • program 410 may include program code, which includes computer-executable instructions.
  • the processor 402 may be a central processing unit CPU, or an Application Specific Integrated Circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention.
  • the one or more processors included in the electronic device may be the same type of processors, such as one or more CPUs; or may be different types of processors, such as one or more CPUs and one or more ASICs.
  • the memory 406 is used to store the program 410 .
  • Memory 406 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.
  • the business behavior blueprint and the digital twin model are packaged together.
  • the digital twin model When the digital twin model is loaded, the business behavior blueprint will run dynamically, so that the business behavior blueprint drives the digital twin model to interact with the digital twin world or other digital twin models. Behavioral interaction to synchronously control the physical robot to perform behavioral interaction with the physical environment or other physical robots.
  • An embodiment of the present invention provides a computer-readable storage medium, where the storage medium stores at least one executable instruction.
  • the executable instruction is executed on an electronic device, the electronic device can execute any of the above method embodiments. How the digital twin works.
  • An embodiment of the present invention provides an apparatus for running a digital twin model, which is used to execute the above-mentioned method for running a digital twin model.
  • An embodiment of the present invention provides a computer program, and the computer program can be invoked by a processor to cause an electronic device to execute the method for running a digital twin model in any of the foregoing method embodiments.
  • An embodiment of the present invention provides a computer program product.
  • the computer program product includes a computer program stored on a computer-readable storage medium, and the computer program includes program instructions.
  • the program instructions When the program instructions are run on a computer, the computer is made to execute any of the above.
  • modules in the device in the embodiment can be adaptively changed and arranged in one or more devices different from the embodiment.
  • the modules or units or components in the embodiments may be combined into one module or unit or component, and they may be divided into multiple sub-modules or sub-units or sub-assemblies. All features disclosed in this specification (including accompanying claims, abstract and drawings) and any method so disclosed may be employed in any combination unless at least some of such features and/or procedures or elements are mutually exclusive. All processes or units of equipment are combined.
  • Each feature disclosed in this specification may be replaced by alternative features serving the same, equivalent or similar purpose, unless expressly stated otherwise.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Computational Linguistics (AREA)
  • Audiology, Speech & Language Pathology (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Geometry (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Computer Hardware Design (AREA)
  • General Health & Medical Sciences (AREA)
  • Human Computer Interaction (AREA)
  • Acoustics & Sound (AREA)
  • Multimedia (AREA)
  • Manipulator (AREA)

Abstract

本发明实施例涉及机器人技术领域，公开了一种数字孪生模型的运行方法、装置和电子设备。所述数字孪生模型为与实体机器人物理属性相同的物理模型，所述方法包括：将数字孪生模型和所述数字孪生模型的业务行为蓝图进行打包，得到打包后的数字孪生模型，其中，所述打包后的数字孪生模型包括所述业务行为蓝图；加载所述打包后的数字孪生模型，运行所述业务行为蓝图；所述业务行为蓝图驱动所述数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，以同步控制所述实体机器人与物理环境或其他实体机器人进行行为交互。通过上述方式，本发明实施例实现了数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互。

Description

数字孪生模型的运行方法、装置和电子设备
相关申请的交叉引用
本公开要求在2020年12月22日提交中国专利局、申请号为202011531979.7、名称为“数字孪生模型的运行方法、装置和电子设备”的中国专利申请的优先权，其全部内容通过引用结合在本公开中。
技术领域
本发明实施例涉及机器人技术领域，具体涉及一种数字孪生模型的运行方法、装置和电子设备。
背景技术
目前，在机器人的实现方式中，云端机器人得到了越来越广泛的应用。其中，在危险的、肮脏的、重复性的以及实现较为困难的一些应用场景中，对云端机器人的要求也更高，产生了功能上可以替代人类的智能机器人的市场需求。
若为了提升智能机器人的训练效率及降低试错成本，采用数字孪生模型进行机器人技能和应用的训练，那么，如何数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，是需要解决的问题。
发明内容
鉴于上述问题，本发明实施例提供了一种数字孪生模型的运行方法，所述数字孪生模型为与实体机器人物理属性相同的物理模型，所述方法包括：
将数字孪生模型和所述数字孪生模型的业务行为蓝图进行打包，得到打包后的数字孪生模型，其中，所述打包后的数字孪生模型包括所述业务行为蓝图；
加载所述打包后的数字孪生模型，运行所述业务行为蓝图；
所述业务行为蓝图驱动所述数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，以同步控制所述实体机器人与物理环境或其他实体机器人进行行为交互。
其中，所述业务行为蓝图以行为脚本的文件方式呈现，所述业务行为蓝图包括所述数字孪生模型的行为逻辑，所述打包后的数字孪生模型中定义了所述行为逻辑，所述行为逻辑用于驱动所述数字孪生模型与环境或其他数字孪生模型进行行为交互。
其中，所述业务行为蓝图驱动所述数字孪生模型与环境或其他数字孪生模型进行行为交互，以同步控制所述实体机器人与物理环境或其他实体机器人进行行为交互，包括：
所述数字孪生模型接收控制指令；
所述业务行为蓝图对所述控制指令进行解析，得到与所述控制指令匹配的一个或多个行为；
所述业务行为蓝图控制所述数字孪生模型执行所述一个或多个行为，以同步控制所述实体机器人执行所述一个或多个行为。
其中，所述业务行为蓝图对所述控制指令进行解析，得到与所述控制指令匹配的一个或多个行为，包括：
所述业务行为蓝图对所述控制指令进行意图分析，并根据所述数字孪生模型对应的实体机器人的类型，确定该数字孪生模型需要执行的与所述控制指令匹配的一个或多个行为。
其中，所述业务行为蓝图支持重载机制，所述方法还包括：
对于确定的所述一个或多个行为，根据该数字孪生模型对应的实体机器人的类型，判断该实体机器人在执行所述行为时是否与通用机器人的动作相同；
若不同，对所述行为进行重载；
所述控制所述数字孪生模型执行所述一个或多个行为，包括：
控制所述数字孪生模型执行所述重载后的行为。
其中，所述方法还包括：
所述业务行为蓝图支持继承机制，其中，数字孪生模型中的子模型的业务行为蓝图可以继承父类模型的业务行为蓝图。
其中，所述方法还包括：
对所述实体机器人的形状、结构和外观进行几何外观建模和关节仿真，得到第一模型；
在所述第一模型的基础上对所述实体机器人的传感器进行仿真，得到包含传感器模型的第二模型；
在所述第二模型的基础上对所述实体机器人的物理属性进行物理仿真，得到数字孪生模型。
本发明实施例还提供了一种数字孪生模型的实现装置，所述数字孪生模型为与实体 机器人物理属性相同的物理模型，所述装置包括：
打包模块，用于将数字孪生模型和所述数字孪生模型的业务行为蓝图进行打包，得到打包后的数字孪生模型，其中，所述打包后的数字孪生模型包括所述业务行为蓝图；
加载模块，用于加载所述打包后的数字孪生模型，运行所述业务行为蓝图；
驱动模块，用于通过所述业务行为蓝图驱动所述数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，以同步控制所述实体机器人与物理环境或其他实体机器人进行行为交互。
本发明实施例还提供了一种电子设备，包括：处理器、存储器、通信接口和通信总线，所述处理器、所述存储器和所述通信接口通过所述通信总线完成相互间的通信；
所述存储器用于存放至少一可执行指令，所述可执行指令使所述处理器执行如上所述的数字孪生模型的运行方法的操作。
本发明实施例还提供了一种计算机可读存储介质，所述存储介质中存储有至少一可执行指令，所述可执行指令在电子设备上运行时，使得电子设备执行如上所述的数字孪生模型的运行方法的操作。
本发明实施例通过将业务行为蓝图与数字孪生模型一起打包，加载数字孪生模型时，业务行为蓝图会动态运行，从而通过业务行为蓝图驱动所述数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，以同步控制所述实体机器人与物理环境或其他实体机器人进行行为交互。
上述说明仅是本发明实施例技术方案的概述，为了能够更清楚了解本发明实施例的技术手段，而可依照说明书的内容予以实施，并且为了让本发明实施例的上述和其它目的、特征和优点能够更明显易懂，以下特举本发明的具体实施方式。
附图说明
附图仅用于示出实施方式，而并不认为是对本发明的限制。而且在整个附图中，用相同的参考符号表示相同的部件。在附图中：
图1是本发明实施例提供的数字孪生模型的运行方法的应用示意图；
图2是本发明实施例提供的数字孪生模型的运行方法的流程图；
图3是本发明实施例提供的数字孪生模型的运行装置的结构示意图；
图4是本发明实施例提供的电子设备的结构示意图。
具体实施方式
下面将参照附图更详细地描述本发明的示例性实施例。虽然附图中显示了本发明的示例性实施例，然而应当理解，可以以各种形式实现本发明而不应被这里阐述的实施例所限制。
从机器人向人类的智能化发展角度来看，如果要制造一个跟人脑一样聪明的电子大脑，该电子大脑将会是巨大的，不可能在单体机器人上实现。此外，由于单体机器人所能接触的数据有限，无法完成需要有大数据训练的机器学习和深度学习。人工智能的深度学习必须由大量机器人提供数据，汇聚到云端，由云端的巨大的“机器大脑”来完成，这进一步说明机器人的部分感知和认知系统必须放在云端，这是智能机器人发展的必然方向。
基于此，本发明实施例提供了一种云端机器人系统，图1是本发明实施例提供的数字孪生模型的运行方法的应用示意图。该方法应用于云服务器。如图1所示，云服务器10和实体机器人20之间通过专用网络30进行通信。各项机器人服务在云服务器10训练完成，并由云服务器10控制实体机器人20执行训练好的各项机器人服务。其中，机器人服务是指在不同应用场景中执行预设动作，完成预设功能，例如迎宾接待、移动抓取、安防巡逻和配送等。服务需要由应用组成，而若干技能组合成应用的逻辑。例如，会打乒乓球入削球、拉球等，属于技能，应用是指由实体机器人实际去打乒乓球。服务是指实体机器人可以提供打乒乓球陪练的服务。再如，实体机器人抓物品属于技能，实体机器人使用抓物品的技能可以完成送咖啡给人的应用，则实体机器人可以完成端茶倒水的接待服务。
本发明实施例提供了一种数字孪生模型的运行方法。图2是本发明实施例提供的数字孪生模型的运行方法的流程图。如图2所示，该方法包括如下步骤：
步骤201：将数字孪生模型和数字孪生模型的业务行为蓝图进行打包，得到打包后的数字孪生模型，其中，打包后的数字孪生模型包括业务行为蓝图。
其中，数字孪生模型为与实体机器人物理属性相同的物理模型，也可以称为数字孪生体。通过为实体机器人建立物理特性上一一映射的数字孪生模型，以及构建实体机器人所处物理世界的虚拟镜像的数字孪生世界，采用数字孪生模型在数字孪生世界中进行机器人技能和应用的训练，基于数字孪生模型同步控制实体机器人的行为。同时实体机器人的各种传感器采集的多源数据(也即获取环境变化数据)也将同步到数字孪生世界， 用于数字孪生模型的机器人技能和应用的训练和在线运行，实现动态闭环、持续进化的智能云端机器人系统。这样，可以提升智能训练的效率，降低试错成本。
数字孪生世界是实体机器人所处物理世界的虚拟镜像的三维语义地图数据服务，是实体机器人在各种应用场景下可认知理解的三维环境语义的数字化表示，帮助机器人感知、认知物理世界，为云服务器的机器人实时在线运行服务提供可交互的数字语义化环境。数字孪生世界也被用于后台(离线)训练各种数字孪生模型，保证在实体机器人上线运行时具有最佳运行策略和行为和动作。
数字孪生模型可以通过如下方式构建：
步骤a1：对实体机器人的形状、结构和外观进行几何外观建模和关节仿真，得到第一模型。
步骤a2：在第一模型的基础上对实体机器人的传感器进行仿真，得到包含传感器模型的第二模型。
步骤a3：在第二模型的基础上对实体机器人的物理属性进行物理仿真，得到数字孪生模型。
构建好的数字孪生模型，不仅几何外观上与实体机器人保持一致，而且在动力学控制模型及空间位置上也是保持一致的。此外，实体机器人的一些控制能力接口在数字孪生模型上也进行了仿真和提供。
其中，业务行为蓝图以行为脚本的文件方式呈现，业务行为蓝图包括数字孪生模型的行为逻辑，打包后的数字孪生模型中定义了行为逻辑，行为逻辑用于驱动数字孪生模型与环境或其他数字孪生模型进行行为交互。业务行为蓝图还采集实体机器人的状态以及其周围物理环境信息，对数字孪生模型和数字孪生世界进行同步更新。这样，打包后的数字孪生模型不再是纯数据，而是带行为逻辑的模型。通过将数字孪生模型与业务行为蓝图合并为一个资源包，可以实现模型及行为能力的高度开放与定制，从而实现一套平台或引擎可以支持接入与控制各种差异化的实体机器人。
当然，一个数字孪生模型可以对应多个业务行为蓝图，每个业务行为蓝图中可以包括一个或多个行为。而一个行为可以由一个或多个动作组成。
具体实现时，可以采用虚幻引擎(Unreal Engine，UE)作为数字孪生渲染与协作处理引擎，采用Unreal Pack工具对数字孪生模型及业务行为蓝图进行打包。蓝图(blueprint)本质上是指一种可视化的编程方式。数字孪生模型包含业务行为蓝图后，将变为主动的 智能体，在虚拟世界中主动感知环境变化并作出反馈。也即通过运行业务行为蓝图，可以调用UE引擎的GameAI能力探测周围环境，比如发送射线探测前方是否有其他机器人等，从而进一步确定如何执行数字孪生模型在该数字孪生世界中的行为。
将数字孪生模型和业务行为蓝图打包成一个文件包后，可以通过机器人开发平台将打包后的数字孪生模型进行分发共享。
步骤202：加载打包后的数字孪生模型，运行业务行为蓝图。
在数字孪生模型加载后，与其一起打包的业务行为蓝图将一并加载并启动运行。
具体实现时，可以采用Unreal Pak资源包的形式对包含业务行为蓝图的数字孪生模型进行动态加载。由于业务行为蓝图并未和数字孪生模型关联或绑定，相当于业务行为蓝图是空脚本，不具备可执行性。因此当资源加载成功后，将绑定业务行为蓝图与该数字孪生模型，使业务行为蓝图将被实例化，可以被执行，并准备好接收引擎对事件和接口的调用。其中，实体机器人感知的数据会转化为蓝图事件，由业务行为蓝图进行监听和处理。
步骤203：业务行为蓝图驱动数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，以同步控制实体机器人与物理环境或其他实体机器人进行行为交互。
数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，主要是指数字孪生模型在数字孪生世界中执行一些行为，从而实现其服务功能。例如，实体机器人在物理环境中获取用户的语音指令，将该语音指令同步到数字孪生模型，业务行为蓝图通过分析该语音指令判断数字孪生模型需要执行的一些行为，然后控制数字孪生模型执行这些行为，并同步控制实体机器人执行相同的行为。
在一些实施例中，步骤203进一步包括如下步骤：
步骤a1：数字孪生模型接收控制指令。
其中，控制指令可以是物理环境中用户发出的指令，例如用户对实体机器人说“我渴了”；或者实体机器人通过传感器获取的其他周围环境数据自动判断出需要执行的指令，例如门铃响了，实体机器人的传感器获取到门铃声音，判断出需要去开门；或者在由云服务器在进行数字孪生模型的技能和应用的训练时发送的指令。
步骤a2：业务行为蓝图对控制指令进行解析，得到与控制指令匹配的一个或多个行为。
其中，解析包括直接的语义分析以及意图分析。
控制指令可以直接包括需要实体机器人执行的行为，例如，用户对实体机器人说“请到厨房，把桌子上的矿泉水拿过来，递给我”，则该控制指令直接包括了需要实体机器人执行的三个行为：1.走到厨房，2.把桌子上的矿泉水拿过来，3.递给用户。此时进行直接的语义分析则可以得到与控制指令匹配的行为。
控制指令也可以不直接包括需要实体机器人执行的行为，而仅仅是用户的一个随意的语句。若控制指令没有直接包括需要实体机器人执行的行为，则需要业务行为蓝图对该控制指令进行意图分析，判断出实体机器人需要执行的行为。例如，用户对实体机器人说“我渴了”，则该控制指令没有直接包括需要实体机器人执行的行为，此时业务行为蓝图需进行意图解析，得到实体机器人需执行如下三个行为：1.走到厨房，2.把桌子上的矿泉水拿过来，3.递给用户。
该行为可以是一个或多个行为。上述举例均为行为为多个的情况。如下为行为仅为一个的情况，例如，用户和实体机器人说“我们握手吧”，则实体机器人只需执行和用户握手这一个行为。
在一些实施例中，业务行为蓝图对控制指令进行意图分析，还可以根据数字孪生模型对应的实体机器人的类型，确定该数字孪生模型需要执行的与控制指令匹配的一个或多个行为。实体机器人的类型可以包括迎宾机器人、安保机器人、家务机器人、陪伴机器人等。例如，若控制指令为“打招呼”，对于仅具备屏幕和身体、不具备手部的实体机器人，其确定的行为为屏幕进行点头；对于不仅具有头部还具有手部的实体机器人，其确定的行为为点头和挥手；对于不仅具有头部、还具有手部、且具备面部表情的实体机器人，其确定的行为为点头、挥手以及微笑。通过上述方式，可以使业务行为蓝图可根据自身的数字孪生模型所对应的实体机器人的特性对行为确定进行差异处理，而差异可以在业务行为蓝图中预先进行定义。因此，由于数字孪生模型包含业务行为蓝图，可以实现对不同实体机器人差异的适配，可以实现基于实体机器人的能力差异对标准的动作指令或控制数据进行适配，从而实现使用统一的控制接口达到差异化的控制表现。
步骤a3：业务行为蓝图控制数字孪生模型执行一个或多个行为，以同步控制实体机器人执行一个或多个行为。
确定好与控制指令匹配的行为后，就可以控制数字孪生模型执行一个或多个行为，以同步控制实体机器人执行一个或多个行为。
在一些实施例中，业务行为蓝图支持继承和/或重载机制。其中，Unreal Engine具备 继承和/或重载蓝图机制，这一机制类似面向对象编程中类的继承和函数重载。
关于继承机制，是指数字孪生模型中的子模型的业务行为蓝图可以继承父类模型的业务行为蓝图，这样父类蓝图的属性与方法可以被继承到子模型中，子模型的mesh(模型的存储文件)可以被替换更新为父模型的mesh，从而实现模型的继承。此外，子模型的蓝图还可以重新定义、追加属性和方法。例如，底盘移动机器人模型和四轮机器人模型都可以继承至一个更通用的或者行为能力更强的类人机器人模型，关节少的机器人模型可以继承至关节更多的机器人模型。通过继承机制，使数字孪生模型在调用业务行为蓝图时速度更快，无需遍历所有的业务行为蓝图，就可以直接调用到相应的业务行为蓝图，加快了对事件的响应速度。
重载是指子模型可以重新定义父类模型的相同方法名所定义的功能。例如对于从A地走到B地的行为，对于普通机器人，直接执行行走的动作即可；对于安保机器人，根据该机器人类型，其业务行为蓝图需要对该行为进行重载，除了行走的动作，还需加入左右摇头查看的巡视动作，这一针对相同名称的行为(或功能)进行重新定义的方式称为重载，也即对于同样的行为可通过重载执行不同的动作。因此，在一些实施例中，方法还包括：
步骤b1：对于确定的一个或多个行为，根据该数字孪生模型对应的实体机器人的类型，判断该实体机器人在执行行为时是否与通用机器人的动作相同。
步骤b2：若不同，对行为进行重载。
相应的，控制数字孪生模型执行一个或多个行为，包括：控制数字孪生模型执行重载后的行为。
前文已提及，实体机器人的各种传感器采集的多源数据也将同步到数字孪生世界。因此，在一些实施例中，该方法还包括：
步骤c1：业务行为蓝图探测数字孪生模型周围的数字孪生世界，得到数字孪生世界中数字孪生模型周围的物体信息。
步骤c2：根据物体信息控制数字孪生模型与环境或其他数字孪生模型进行行为交互的实时动作变化或移动路线变化。
本发明实施例通过将业务行为蓝图与数字孪生模型一起打包，加载数字孪生模型时，业务行为蓝图会动态运行，从而通过业务行为蓝图驱动数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，以同步控制实体机器人与物理环境或其他实体机器人 进行行为交互。
图3是本发明实施例提供的数字孪生模型的运行装置的结构示意图。如图3所示，所述数字孪生模型为与实体机器人物理属性相同的物理模型，该装置300包括：
打包模块301，用于将数字孪生模型和所述数字孪生模型的业务行为蓝图进行打包，得到打包后的数字孪生模型，其中，所述打包后的数字孪生模型包括所述业务行为蓝图。
加载模块302，用于加载所述打包后的数字孪生模型，运行所述业务行为蓝图。
驱动模块303，用于所述业务行为蓝图驱动所述数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，以同步控制所述实体机器人与物理环境或其他实体机器人进行行为交互。
在一种可选的方式中，所述业务行为蓝图以行为脚本的文件方式呈现，所述业务行为蓝图包括所述数字孪生模型的行为逻辑，所述打包后的数字孪生模型中定义了所述行为逻辑，所述行为逻辑用于驱动所述数字孪生模型与环境或其他数字孪生模型进行行为交互。
在一种可选的方式中，所述业务行为蓝图驱动所述数字孪生模型与环境或其他数字孪生模型进行行为交互，以同步控制所述实体机器人与物理环境或其他实体机器人进行行为交互，包括：
所述数字孪生模型接收控制指令。
所述业务行为蓝图对所述控制指令进行解析，得到与所述控制指令匹配的一个或多个行为。
所述业务行为蓝图根控制所述数字孪生模型执行所述一个或多个行为，以同步控制所述实体机器人执行所述一个或多个行为。
在一种可选的方式中，所述业务行为蓝图对所述控制指令进行解析，得到与所述控制指令匹配的一个或多个行为，包括：
所述业务行为蓝图对所述控制指令进行意图分析，并根据所述数字孪生模型对应的实体机器人的类型，确定该数字孪生模型需要执行的与所述控制指令匹配的一个或多个行为。
在一种可选的方式中，所述业务行为蓝图支持重载机制，所述驱动模块303还用于：
对于确定的所述一个或多个行为，根据该数字孪生模型对应的实体机器人的类型，判断该实体机器人在执行所述行为时是否与通用机器人的动作相同。
若不同，对所述行为进行重载。
控制所述数字孪生模型执行所述重载后的行为。
在一种可选的方式中，所述业务行为蓝图支持继承机制，其中，数字孪生模型中的子模型的业务行为蓝图可以继承父类模型的业务行为蓝图。
在一种可选的方式中，所述装置300还包括建模模块，用于：
对所述实体机器人的形状、结构和外观进行几何外观建模和关节仿真，得到第一模型。
在所述第一模型的基础上对所述实体机器人的传感器进行仿真，得到包含传感器模型的第二模型。
在所述第二模型的基础上对所述实体机器人的物理属性进行物理仿真，得到数字孪生模型。
本发明实施例通过将业务行为蓝图与数字孪生模型一起打包，加载数字孪生模型时，业务行为蓝图会动态运行，从而通过业务行为蓝图驱动所述数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，以同步控制所述实体机器人与物理环境或其他实体机器人进行行为交互。
图4是本发明实施例提供的电子设备的结构示意图，本发明具体实施例并不对电子设备的具体实现做限定。
如图4所示，该电子设备可以包括：处理器(processor)402、通信接口(Communications Interface)404、存储器(memory)406、以及通信总线408。
其中：处理器402、通信接口404、以及存储器406通过通信总线408完成相互间的通信。通信接口404，用于与其它设备比如客户端或其它服务器等的网元通信。处理器402，用于执行程序410，具体可以执行上述用于数字孪生模型的运行方法实施例中的相关步骤。
具体地，程序410可以包括程序代码，该程序代码包括计算机可执行指令。
处理器402可能是中央处理器CPU，或者是特定集成电路ASIC(Application Specific Integrated Circuit)，或者是被配置成实施本发明实施例的一个或多个集成电路。电子设备包括的一个或多个处理器，可以是同一类型的处理器，如一个或多个CPU；也可以是不同类型的处理器，如一个或多个CPU以及一个或多个ASIC。
存储器406，用于存放程序410。存储器406可能包含高速RAM存储器，也可能还 包括非易失性存储器(non-volatile memory)，例如至少一个磁盘存储器。
本发明实施例通过将业务行为蓝图与数字孪生模型一起打包，加载数字孪生模型时，业务行为蓝图会动态运行，从而通过业务行为蓝图驱动所述数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，以同步控制所述实体机器人与物理环境或其他实体机器人进行行为交互。
本发明实施例提供了一种计算机可读存储介质，所述存储介质存储有至少一可执行指令，该可执行指令在电子设备上运行时，使得所述电子设备执行上述任意方法实施例中的数字孪生模型的运行方法。
本发明实施例提供一种数字孪生模型的运行装置，用于执行上述数字孪生模型的运行方法。
本发明实施例提供了一种计算机程序，所述计算机程序可被处理器调用使电子设备执行上述任意方法实施例中的数字孪生模型的运行方法。
本发明实施例提供了一种计算机程序产品，计算机程序产品包括存储在计算机可读存储介质上的计算机程序，计算机程序包括程序指令，当程序指令在计算机上运行时，使得所述计算机执行上述任意方法实施例中的数字孪生模型的运行方法。
在此提供的算法或显示不与任何特定计算机、虚拟系统或者其它设备固有相关。各种通用系统也可以与基于在此的示教一起使用。根据上面的描述，构造这类系统所要求的结构是显而易见的。此外，本发明实施例也不针对任何特定编程语言。应当明白，可以利用各种编程语言实现在此描述的本发明的内容，并且上面对特定语言所做的描述是为了披露本发明的最佳实施方式。
在此处所提供的说明书中，说明了大量具体细节。然而，能够理解，本发明的实施例可以在没有这些具体细节的情况下实践。在一些实例中，并未详细示出公知的方法、结构和技术，以便不模糊对本说明书的理解。
类似地，应当理解，为了精简本发明并帮助理解各个发明方面中的一个或多个，在上面对本发明的示例性实施例的描述中，本发明实施例的各个特征有时被一起分组到单个实施例、图、或者对其的描述中。然而，并不应将该公开的方法解释成反映如下意图：即所要求保护的本发明要求比在每个权利要求中所明确记载的特征更多的特征。
本领域技术人员可以理解，可以对实施例中的设备中的模块进行自适应性地改变并且把它们设置在与该实施例不同的一个或多个设备中。可以把实施例中的模块或单元或 组件组合成一个模块或单元或组件，以及可以把它们分成多个子模块或子单元或子组件。除了这样的特征和/或过程或者单元中的至少一些是相互排斥之外，可以采用任何组合对本说明书(包括伴随的权利要求、摘要和附图)中公开的所有特征以及如此公开的任何方法或者设备的所有过程或单元进行组合。除非另外明确陈述，本说明书(包括伴随的权利要求、摘要和附图)中公开的每个特征可以由提供相同、等同或相似目的的替代特征来代替。
应该注意的是上述实施例对本发明进行说明而不是对本发明进行限制，并且本领域技术人员在不脱离所附权利要求的范围的情况下可设计出替换实施例。在权利要求中，不应将位于括号之间的任何参考符号构造成对权利要求的限制。单词“包含”不排除存在未列在权利要求中的元件或步骤。位于元件之前的单词“一”或“一个”不排除存在多个这样的元件。本发明可以借助于包括有若干不同元件的硬件以及借助于适当编程的计算机来实现。在列举了若干装置的单元权利要求中，这些装置中的若干个可以是通过同一个硬件项来具体体现。单词第一、第二、以及第三等的使用不表示任何顺序。可将这些单词解释为名称。上述实施例中的步骤，除有特殊说明外，不应理解为对执行顺序的限定。

Claims (12)

  1. 一种数字孪生模型的运行方法，其特征在于，所述数字孪生模型为与实体机器人物理属性相同的物理模型，所述方法包括：
    将所述数字孪生模型和所述数字孪生模型的业务行为蓝图进行打包，得到打包后的数字孪生模型；
    加载所述打包后的数字孪生模型，以加载所述数字孪生模型并运行所述业务行为蓝图；
    所述数字孪生模型接收控制指令；
    所述业务行为蓝图根据所述控制指令驱动所述数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，以同步控制所述实体机器人与物理环境或其他实体机器人进行行为交互。
  2. 根据权利要求1所述的方法，其特征在于，
    所述业务行为蓝图为行为脚本，所述业务行为蓝图包括所述数字孪生模型的行为逻辑，所述打包后的数字孪生模型中定义了所述行为逻辑，所述行为逻辑用于根据所述控制指令驱动所述数字孪生模型与所述数字孪生世界或所述其他数字孪生模型进行行为交互。
  3. 根据权利要求1所述的方法，其特征在于，所述业务行为蓝图根据所述控制指令驱动所述数字孪生模型与环境或其他数字孪生模型进行行为交互，以同步控制所述实体机器人与物理环境或其他实体机器人进行行为交互，包括：
    所述业务行为蓝图对所述控制指令进行解析，得到与所述控制指令匹配的一个或多个行为；
    所述业务行为蓝图控制所述数字孪生模型执行所述一个或多个行为，以同步控制所述实体机器人执行所述一个或多个行为。
  4. 根据权利要求3所述的方法，其特征在于，所述业务行为蓝图对所述控制指令进行解析，得到与所述控制指令匹配的一个或多个行为，包括：
    所述业务行为蓝图对所述控制指令进行语义分析，和/或意图分析；
    根据分析结果和所述数字孪生模型对应的实体机器人的类型，确定该数字孪生模型需要执行的与所述控制指令匹配的一个或多个行为。
  5. 根据权利要求4所述的方法，其特征在于，所述业务行为蓝图支持重载机制，所述 方法还包括：
    对于确定的所述一个或多个行为，根据该数字孪生模型对应的实体机器人的类型，判断该实体机器人在执行所述行为时是否与通用机器人的动作相同；
    若不同，根据该数字孪生模型对应的实体机器人的类型对所述行为进行重载；
    所述控制所述数字孪生模型执行所述一个或多个行为，包括：
    控制所述数字孪生模型执行所述重载后的行为。
  6. 根据权利要求1所述的方法，其特征在于，所述业务行为蓝图支持继承机制，所述方法还包括：
    所述数字孪生模型中的子模型的业务行为蓝图，继承自该子模型的父类模型的业务行为蓝图。
  7. 根据权利要求1所述的方法，其特征在于，所述加载所述打包后的数字孪生模型，以加载所述数字孪生模型并运行所述业务行为蓝图，包括：
    加载所述数字孪生模型和所述业务行为蓝图；
    将所述业务行为蓝图与所述数字孪生模型进行绑定，以对所述业务行为蓝图进行实例化。
  8. 根据权利要求1所述的方法，其特征在于，所述方法还包括：
    所述业务行为蓝图监听所述实体机器人采集的数据，并将所述数据同步到所述数字孪生世界，所述数字孪生世界用于支持所述数字孪生模型的训练和运行。
  9. 根据权利要求1-8任一项所述的方法，其特征在于，所述方法还包括：
    根据所述实体机器人的形状、结构和外观进行几何外观建模，并进行关节仿真，得到第一模型；
    在所述第一模型的基础上根据所述实体机器人的传感器进行仿真，得到包含传感器模型的第二模型；
    在所述第二模型的基础上根据所述实体机器人的物理属性进行物理仿真，并根据所述实体机器人的控制能力接口进行仿真，得到所述数字孪生模型。
  10. 一种数字孪生模型的实现装置，其特征在于，所述数字孪生模型为与实体机器人物理属性相同的物理模型，所述装置包括：
    打包模块，用于将所述数字孪生模型和所述数字孪生模型的业务行为蓝图进行打包，得到打包后的数字孪生模型；
    加载模块，用于加载所述打包后的数字孪生模型，以加载所述数字孪生模型并运行所述业务行为蓝图；
    驱动模块，用于通过所述数字孪生模型接收控制指令，并通过所述业务行为蓝图根据所述控制指令驱动所述数字孪生模型与数字孪生世界或其他数字孪生模型进行行为交互，以同步控制所述实体机器人与物理环境或其他实体机器人进行行为交互。
  11. 一种电子设备，其特征在于，包括：处理器、存储器、通信接口和通信总线，所述处理器、所述存储器和所述通信接口通过所述通信总线完成相互间的通信；
    所述存储器用于存放至少一可执行指令，所述可执行指令使所述处理器执行如权利要求1-9任意一项所述的数字孪生模型的运行方法的操作。
  12. 一种计算机可读存储介质，其特征在于，所述存储介质中存储有至少一可执行指令，所述可执行指令在电子设备上运行时，使得电子设备执行如权利要求1-9任意一项所述的数字孪生模型的运行方法的操作。
PCT/CN2021/124492 2020-12-22 2021-10-18 数字孪生模型的运行方法、装置和电子设备 WO2022134775A1 (zh)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202011531979.7A CN112632778B (zh) 2020-12-22 2020-12-22 数字孪生模型的运行方法、装置和电子设备
CN202011531979.7 2020-12-22

Publications (1)

Publication Number Publication Date
WO2022134775A1 true WO2022134775A1 (zh) 2022-06-30

Family

ID=75321899

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2021/124492 WO2022134775A1 (zh) 2020-12-22 2021-10-18 数字孪生模型的运行方法、装置和电子设备

Country Status (2)

Country Link
CN (1) CN112632778B (zh)
WO (1) WO2022134775A1 (zh)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115460485A (zh) * 2022-08-26 2022-12-09 烽火通信科技股份有限公司 一种基于数据交互的层次化光网络模型及建模方法
CN117289624A (zh) * 2023-09-20 2023-12-26 广东省电信规划设计院有限公司 一种应用于数据孪生技术的数据采集方法及系统

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112632778B (zh) * 2020-12-22 2023-07-18 达闼机器人股份有限公司 数字孪生模型的运行方法、装置和电子设备
CN113959444A (zh) * 2021-09-30 2022-01-21 达闼机器人有限公司 用于无人设备的导航方法、装置、介质及无人设备
CN114310870A (zh) * 2021-11-10 2022-04-12 达闼科技（北京）有限公司 智能体的控制方法、装置、电子设备及存储介质

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110765635A (zh) * 2019-11-04 2020-02-07 上海电气集团股份有限公司 数字孪生系统的协同方法、系统、电子设备和介质
CN111300381A (zh) * 2020-01-20 2020-06-19 兰州理工大学 基于数字孪生三维模型的软体机器人或驱动器系统及其建模方法
CN111538294A (zh) * 2019-11-15 2020-08-14 武汉理工大学 基于数字孪生的工业机器人制造系统可重构系统与方法
CN111552238A (zh) * 2020-04-17 2020-08-18 达闼科技（北京）有限公司 机器人控制方法、装置、计算设备及计算机存储介质
CN112632778A (zh) * 2020-12-22 2021-04-09 达闼机器人有限公司 数字孪生模型的运行方法、装置和电子设备

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9671777B1 (en) * 2016-06-21 2017-06-06 TruPhysics GmbH Training robots to execute actions in physics-based virtual environment
US20190138662A1 (en) * 2017-11-07 2019-05-09 General Electric Company Programmatic behaviors of a contextual digital twin
CN108919760B (zh) * 2018-07-05 2020-01-24 长安大学 一种基于数字孪生的智能车间自治生产过程动态联动控制方法
CN108919765B (zh) * 2018-07-20 2021-06-04 王德权 一种基于数字孪生的智能制造工厂虚拟调试和虚拟监控方法及系统
CN109326003B (zh) * 2018-11-28 2022-09-20 合肥哈工热气球数字科技有限公司 一种基于虚拟仿真的工厂三维监控系统
CN109799974A (zh) * 2018-12-14 2019-05-24 广州优态科技有限公司 数据及逻辑的开发方法
CN109857078A (zh) * 2019-04-17 2019-06-07 中船第九设计研究院工程有限公司 一种船厂生产车间数字孪生仿真系统
CN110222450B (zh) * 2019-06-14 2022-12-02 中国电子科技集团公司第三十八研究所 一种物理行为仿真系统的构建方法及运动机构的控制方法
US11196964B2 (en) * 2019-06-18 2021-12-07 The Calany Holding S. À R.L. Merged reality live event management system and method
CN111061232A (zh) * 2019-12-09 2020-04-24 中国科学院沈阳自动化研究所 基于数字孪生的生产线设计与优化方法
CN110929422B (zh) * 2019-12-09 2020-09-22 中国人民解放军军事科学院国防科技创新研究院 一种机器人集群仿真方法及装置
CN111258571A (zh) * 2020-02-11 2020-06-09 达闼科技（北京）有限公司 机器人应用程序开发方法、系统、装置及存储介质

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110765635A (zh) * 2019-11-04 2020-02-07 上海电气集团股份有限公司 数字孪生系统的协同方法、系统、电子设备和介质
CN111538294A (zh) * 2019-11-15 2020-08-14 武汉理工大学 基于数字孪生的工业机器人制造系统可重构系统与方法
CN111300381A (zh) * 2020-01-20 2020-06-19 兰州理工大学 基于数字孪生三维模型的软体机器人或驱动器系统及其建模方法
CN111552238A (zh) * 2020-04-17 2020-08-18 达闼科技（北京）有限公司 机器人控制方法、装置、计算设备及计算机存储介质
CN112632778A (zh) * 2020-12-22 2021-04-09 达闼机器人有限公司 数字孪生模型的运行方法、装置和电子设备

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115460485A (zh) * 2022-08-26 2022-12-09 烽火通信科技股份有限公司 一种基于数据交互的层次化光网络模型及建模方法
CN115460485B (zh) * 2022-08-26 2024-05-28 烽火通信科技股份有限公司 一种基于数据交互的层次化光网络模型及建模方法
CN117289624A (zh) * 2023-09-20 2023-12-26 广东省电信规划设计院有限公司 一种应用于数据孪生技术的数据采集方法及系统
CN117289624B (zh) * 2023-09-20 2024-03-15 广东省电信规划设计院有限公司 一种应用于数据孪生技术的数据采集方法及系统

Also Published As

Publication number Publication date
CN112632778A (zh) 2021-04-09
CN112632778B (zh) 2023-07-18

Similar Documents

Publication Publication Date Title
WO2022134775A1 (zh) 数字孪生模型的运行方法、装置和电子设备
CN106997235A (zh) 用于实现增强现实交互和展示的方法、设备
JP2022550167A (ja) アニメーションを作成する方法、装置、コンピューティング機器及び記憶媒体
van Oijen et al. CIGA: A middleware for intelligent agents in virtual environments
WO2022218085A1 (zh) 获取虚拟图像的方法、装置、计算机设备、计算机可读存储介质以及计算机程序产品
CN108942919A (zh) 一种基于虚拟人的交互方法及系统
JP2001229392A (ja) 少ないメッセージ交信により会話式キャラクタを実施する合理的アーキテクチャ
US20220156433A1 (en) Industrial network communication emulation
JP2020534567A (ja) ロボット対話方法およびデバイス
CN106660209B (zh) 智能机器人控制系统、方法及智能机器人
Scheutz et al. Systematic integration of cognitive and robotic architectures
WO2023082949A1 (zh) 智能体的控制方法、装置、电子设备、程序及存储介质
Peters et al. Smart objects for attentive agents
Du et al. A ROS/Gazebo based method in developing virtual training scene for upper limb rehabilitation
Pineda et al. Ioca: Interaction-oriented cognitive architecture
CN111421517A (zh) 一种智能终端增强的编程教育机器人
WO2023098090A1 (zh) 智能设备控制方法、装置、服务器和存储介质
KR20240055025A (ko) 실용 3d 에셋을 위한 추론된 골격 구조
Obdržálek Mobile agents and their use in a group of cooperating autonomous robots
EP3617852A1 (en) Method and system for creating aesthetic design at scale
Roth Structured representations for behaviors of autonomous robots
Demeter et al. Cognitive robotics software development aspects based on experiments of future software engineers
Takimoto et al. A simple bi-layered architecture to enhance the liveness of a robot
Schoen et al. Lively: Enabling Multimodal, Lifelike, and Extensible Real-time Robot Motion
García Rojas et al. Individualized reaction movements for virtual humans

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 21908776

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

32PN Ep: public notification in the ep bulletin as address of the adressee cannot be established

Free format text: NOTING OF LOSS OF RIGHTS PURSUANT TO RULE 112(1) EPC (EPO FORM 1205A DATED 22.11.2023)