KR102659118B1 - Digital twin-based robot programming method using asset administration shell - Google Patents

Abstract

A digital twin-based robot programming method using the asset management shell is provided. The robot manipulator control system according to an embodiment of the present invention models an actual robot manipulator with an information model, uses an information model operating unit that receives motion data from the actual robot manipulator using the information model, and an information model of the actual robot manipulator. A virtual robot manipulator is built on the digital twin at the manufacturing site, and a digital twin operating unit is included that controls the actual robot manipulator using the digital twin. As a result, when data related to the production process changes and the work details of the robot manipulator change, the work details can be reflected without interruption.

Description

Digital twin-based robot programming method using asset administration shell}

The present invention relates to robot manipulator programming, and more specifically, to a method of exchanging process elements and data based on AAS and programming a robot manipulator through digital twin.

Recently, manufacturing companies are actively introducing industrial robots in various processes such as assembly, welding, packaging, inspection, and palletizing to secure corporate competitiveness. Automation of production systems using industrial robots has various advantages such as minimum cycle time, improved quality, increased productivity, safety, and worker protection.

However, as requirements for product function and quality become more sophisticated and diverse, the product life cycle is becoming shorter. In this situation, process automation using existing static robot programming (online and offline programming) methods lacks the flexibility needed to transition to a high-mix, small-scale production and mass-customized production system.

To solve these limitations, we are actively conducting research on CNN-based business robot control research using camera images, robot control research based on hand gesture recognition using EMG signals, robot control research using joysticks, and robot control research based on deep reinforcement learning. .

Nevertheless, robot programming is still very complex and takes a lot of time and money, and as a result, automation using robots is still inefficient in building a flexible manufacturing system that can produce a variety of products with high efficiency.

The present invention was created to solve the above problems, and the purpose of the present invention is to provide a robot manipulator that can interact flexibly with other elements of the process according to rapidly changing market changes and product requirements. As a programming method, it provides a method of exchanging process elements and data based on AAS and programming and controlling the robot manipulator through digital twin.

A robot manipulator control system according to an embodiment of the present invention for achieving the above object includes an information model operating unit that models an actual robot manipulator as an information model and receives motion data from the actual robot manipulator using the information model; and a digital twin operating unit that builds a virtual robot manipulator in the digital twin of the manufacturing site using the information model of the actual robot manipulator and controls the actual robot manipulator using the digital twin.

The digital twin operating unit can generate control data for controlling the actual robot manipulator based on simulation results for the virtual robot manipulator and transmit the control data to the actual robot manipulator through the information model.

The digital twin operating unit may control the virtual robot manipulator using a program written to generate control data for the virtual robot manipulator based on data acquired from information models of other elements constituting the manufacturing site.

Other elements may include vision equipment to photograph the product that the robot manipulator must pick.

The digital twin operation unit can identify product information from the product image acquired through the information model of the vision equipment, and identify the target location of the product through the product information model based on the identified product information.

The digital twin operating unit can check in advance whether there is a collision between the virtual robot manipulator and surrounding elements and calculate how the virtual robot manipulator will move to the target point.

The information model operation department and the digital twin operation department can synchronize the information model and digital twin environment through topic communication based on the ROS2 standard.

The information model may be an Asset Administration Shell (AAS) information model.

The information model operation unit can transmit data to the actual robot manipulator using OPC UA (Open Platform Communication Unified Architecture).

According to another aspect of the invention, modeling an actual robot manipulator into an information model; Building a virtual robot manipulator on a digital twin at a manufacturing site using an information model of a real robot manipulator: receiving motion data from a real robot manipulator using an information model; A robot manipulator control method is provided, comprising: controlling an actual robot manipulator using a digital twin.

According to another aspect of the present invention, an actual robot manipulator placed at a manufacturing site; an information model operating unit that models the actual robot manipulator as an information model and receives motion data from the actual robot manipulator using the information model; and a digital twin operating unit that builds a virtual robot manipulator in the digital twin of the manufacturing site using the information model of the actual robot manipulator and controls the actual robot manipulator using the digital twin. A robot manipulator control system comprising a. This is provided.

According to another aspect of the present invention, building a virtual robot manipulator in a digital twin of a manufacturing site using an information model of a real robot manipulator: receiving motion data from a real robot manipulator using the information model; A robot manipulator control method is provided, comprising: controlling an actual robot manipulator using a digital twin.

As described above, according to embodiments of the present invention, by exchanging data with process elements based on AAS and programming the robot manipulator through the digital twin, data related to the production process is changed and the work content of the robot manipulator is changed. When changes are made, work can be reflected without interruption.

In addition, according to embodiments of the present invention, information and real-time data of the robot manipulator are modeled with a standardized data model structure and exchanged using a standard communication protocol to increase interoperability and facilitate expansion, and the person in charge can program the robot manipulator. It can be used even if you lack expertise.

Figure 1. System architecture configuration
Figure 2. System architecture configuration procedure
Figure 3. Relationship between system architecture and procedures
Figure 4. Scenario execution procedure sequence diagram

Hereinafter, the present invention will be described in more detail with reference to the drawings.

An embodiment of the present invention presents a digital twin-based robot programming method using AAS (Asset Administration Shell). It is a technology that exchanges data with process elements based on AAS and programmatically controls the robot manipulator through digital twin.

1. System architecture

Figure 1 is a diagram showing the configuration of a programming control system for a robot manipulator according to an embodiment of the present invention.

As shown, the robot manipulator programming control system according to an embodiment of the present invention includes the robot manipulator 110 corresponding to the physical asset layer and the digital twin operation corresponding to the digital twin layer. It is configured to include a unit 120 and an AAS operation unit 130 corresponding to the AAS Digital Twin Model Layer.

The robot manipulator 110 refers to an actual manipulator placed at a manufacturing site and is controlled by the digital twin operation unit 120, which is an upper layer.

The AAS operating unit 130 models the robot manipulator 110 with AAS, an information model, receives motion data from the robot manipulator 110 using AAS, and generates the data generated by the digital twin operating unit 120, which will be described later. Control data is transmitted to the robot manipulator 110 using AAS.

AAS is a standard digital twin representation model for creating virtual robot manipulator models that are interoperable in the digital world. Additionally, the AAS operation unit 130 may acquire and transmit data using AAS of process elements at the manufacturing site in addition to the robot manipulator 110.

The AAS operation unit 130 performs data exchange, information sharing, and interaction with AAS using OPC UA (Open Platform Communication Unified Architecture), a standard communication protocol.

The digital twin operation unit 120 uses the AAS of the robot manipulator 100 to build a virtual robot manipulator in the digital twin at the manufacturing site and controls the robot manipulator 100 using the digital twin.

To this end, the digital twin operating unit 120 generates control data for controlling the robot manipulator 100 based on simulation results for the virtual robot manipulator, and controls data generated through the AAS of the robot manipulator 100. Delivered to the robot manipulator (100).

Furthermore, the digital twin operating unit 120 simulates the virtual robot manipulator using a program written to generate control data of the virtual robot manipulator based on data acquired from the AAS of other elements constituting the manufacturing site, and operates the robot manipulator ( 100) can also be controlled.

In this process, the digital twin operating unit 120 checks in advance whether there is a collision between the virtual robot manipulator and surrounding elements (Collision Checking), calculates how the virtual robot manipulator will move to the target point (Kinematics), and runs a path creation algorithm. Motion planning is used to enable the virtual robot manipulator to recognize the surrounding environment and safely reach the desired point without human intervention or separate programming.

2. Design procedure

In order to apply the above system architecture, a total of four procedures were constructed as shown in Figure 2: digital twin construction, data modeling, real-time data synchronization, and standard communication protocol application. Applying these procedures to the system architecture is shown in Figure 3.

2.1 Digital twin construction (S210)

To build an offline programming environment for controlling the robot manipulator 110, a digital twin environment can be built using MoveIt2.

In general, digital twin construction, referred to in robot programming systems, builds a digital twin environment based on the 3D model of the robot manipulator and the 3D model of the elements that make up the process (equipment, process layout, etc.) to control the robot manipulator, and controls the movement of the robot. is created automatically without human intervention.

The offline programming (OLP) method, which is one of the methods of programming an actual robot manipulator, uses a 3D model of the robot in a digital twin environment to identify and correct errors during programming before applying operations to the robot manipulator, thereby preventing damage to the robot system and its consequences. It has the advantage of reducing downtime.

Additionally, in a digital twin environment, a path generation algorithm is utilized to consider obstacles and calculate trajectories to create an optimal path for the robot to automatically perform tasks without human intervention.

The programming method according to an embodiment of the present invention supports an offline programming method through building a digital twin. Collision Checking, a function that detects the surroundings by organizing the relationship between each joint and the components of the surrounding process and checks in advance whether there is a collision based on this information, and factors such as the robot's joint angle and link length Kinematics, which calculates how the joints of the robot manipulator will move to the target point by analyzing the relationships between them, and path creation algorithms, allow the robot manipulator to recognize the surrounding environment and safely reach the desired point without human intervention or separate programming. Many functions for robot control, such as motion planning, are performed by building a digital twin.

In addition, the constructed digital twin is linked to the actual robot manipulator, so it can be simulated and applied immediately in the digital twin environment, enabling flexible and quick response to variable environments.

2.2 Data Modeling (S220)

Standardized data modeling is needed so that the information of the robot manipulator described above can be expressed and interacted with in the digital world. Accordingly, data modeling should be conducted based on AAS, a standardization system proposed by Industry 4.0 that expresses all physical assets existing in the real world as a digital information model.

The proposed system can model AAS-based data based on the AAS metamodel proposed by Industrial Digital Twin AASociation (IDTA). IDTA defines MetaModel as an object-oriented UML diagram.

Looking at the structure of MetaModel, the structure of AAS has a submodel that groups asset information such as manufacturer information, asset specification information, operation information for control, and design drawings. At the lower level, each submodel is divided into properties that contain asset information. A configured SubmodelElement exists. The Property that defines the actual value of the assets grouped in the Submodel must be interpreted with the same meaning everywhere, and for this purpose, it is linked to a ConceptDescription that references a unique identifier such as CDD or ECLAAS of IEC61360. Through this, it is possible to build an ontology information model that computers can understand.

Based on the MetaModel proposed by IDTA as above, the information required by the robot manipulator can be modeled using the AAS data meta model standard, and a standardized information model can be built using a user-defined submodel for robot control. As a result, smooth information sharing and data exchange can be achieved while ensuring interoperability between assets or stakeholders.

2.3 Real-time data synchronization (S230)

In order to share real-time data from the modeled robot manipulator AAS and control the robot through data from other models, the AAS digital twin layer and the digital twin layer must be connected and synchronized through an internal interface.

Accordingly, it is possible to build an internal interface that can synchronize the AAS information model and the digital twin environment through Topic communication based on the ROS2 standard.

Through this, the dynamic data of the actual robot manipulator is updated in the property of the operation submodel of the AAS information model, making it possible to check dynamic data such as the moving speed and current position of the robot manipulator's joint through the AAS model.

In addition, in order to reflect in real time the work contents of the robot manipulator received from other process elements, it was linked with offline programming functions such as Collision Checking, Kinematics, Motion Planning, and Robot Control of the digital twin through an internal interface module, and when the work contents are changed, the digital twin The robot manipulator can be controlled by utilizing its offline programming function.

2.4 Application of standard communication protocol (S240)

In order to share and interact with AAS-based data with other models, a communication protocol that can autonomously exchange data is needed.

The IDTA standard expresses that there are three types of data exchange methods based on AAS. In the embodiment of the present invention, the Server/Client structure of OPC UA is adopted to apply Type 2 Reactive AAS among the three types of methods.

Type 2 Reactive AAS can exchange data with other AAS or applications through server/client structured APIs (HTTP, MQTT, etc.). By enabling the exchange of real-time data through a server-client structure, it maintains interoperability between heterogeneous systems or applications, enables interaction through data exchange, and provides EndPoint to enable expansion to other services and applications.

3. Pick & Place application scenario and configuration procedure

In order to program a robot manipulator without human intervention using the robot manipulator programming system according to an embodiment of the present invention, vision equipment and production products are assumed with process elements modeled in AAS.

In Pick & Place, there is a production product AAS that has place location information and a vision AAS that can photograph the produced product.

In addition, there is a Product AAS Registry that has the endpoint of the product AAS in order to apply it to various manufactured products. And there is a Workflow Manager that controls the flow of all scenarios and exchanges data. The data exchange process and operation procedure are shown in Figure 4.

(1) In order to receive the production product number and pick location from Vision AAS, connect to the OPC UA Server and request a photo shoot.

(2) A pick command is issued to the system proposed in the present invention based on the pick position of the produced product received from vision AAS.

(3) Based on the production product number information received from Vision AAS, access the Product AAS Registry to receive the production product AAS Endpoint.

(4) Access the production product AAS Endpoint received through the Product AAS Registry and receive the place location.

(5) Connect to the robot AAS and issue a place command based on the place location of the manufactured product.

According to the above method, even if the specifications of the product picked and placed by the robot manipulator or the palletizing position of the product are changed, the product modeled based on the standard can be recognized with vision, and the digital product can be digitalized based on the changed work contents without separate programming modification. Work content can be simulated in the twin layer and applied to the robot.

4. Variation example

So far, the digital twin-based robot programming method using the asset management shell has been explained in detail with preferred embodiments.

In the above embodiment, by exchanging data with process elements based on AAS and programming the robot manipulator through the digital twin, the work content is reflected without interruption when the work content of the robot manipulator changes due to changes in production process-related data. made it possible to do so.

By modeling the information and real-time data of the robot manipulator in a standardized data model structure and exchanging it using standard communication protocols, interoperability is improved and expansion is easy, and it can be used even if the person in charge lacks expertise in programming the robot manipulator.

Furthermore, since the robot is programmed using an open source-based software platform, it can be applied regardless of the robot vendor.

Meanwhile, of course, the technical idea of the present invention can be applied to a computer-readable recording medium containing a computer program that performs the functions of the device and method according to this embodiment. Additionally, the technical ideas according to various embodiments of the present invention may be implemented in the form of computer-readable code recorded on a computer-readable recording medium. A computer-readable recording medium can be any data storage device that can be read by a computer and store data. For example, of course, computer-readable recording media can be ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical disk, hard disk drive, etc. Additionally, computer-readable codes or programs stored on a computer-readable recording medium may be transmitted through a network connected between computers.

In addition, although preferred embodiments of the present invention have been shown and described above, the present invention is not limited to the specific embodiments described above, and the technical field to which the invention pertains without departing from the gist of the present invention as claimed in the claims. Of course, various modifications can be made by those skilled in the art, and these modifications should not be understood individually from the technical idea or perspective of the present invention.

110: Physical asset layer / robot manipulator
120: Digital twin layer / Digital twin operation department
130: AAS digital twin model layer / AAS operation department

Claims (12)

an information model operating unit that models the actual robot manipulator as an information model and receives motion data from the actual robot manipulator using the information model; and
It includes a digital twin operating unit that builds a virtual robot manipulator on the digital twin at the manufacturing site using the information model of the actual robot manipulator, and controls the actual robot manipulator using the digital twin;
The digital twin operation department,
Based on the simulation results for the virtual robot manipulator, control data for controlling the actual robot manipulator is generated, and the control data is transmitted to the actual robot manipulator through the information model.
Controlling the virtual robot manipulator using a program written to generate control data of the virtual robot manipulator based on data obtained from information models of other elements constituting the manufacturing site,
A robot manipulator control system that checks in advance whether there is a collision between the virtual robot manipulator and surrounding elements and calculates how the virtual robot manipulator will move to the target point.
delete delete In claim 1,
Other factors are:
A robot manipulator control system comprising vision equipment for photographing products that the robot manipulator must pick.
an information model operating unit that models the actual robot manipulator as an information model and receives motion data from the actual robot manipulator using the information model; and
It includes a digital twin operating unit that builds a virtual robot manipulator on the digital twin of the manufacturing site using the information model of the actual robot manipulator and controls the actual robot manipulator using the digital twin;
Digital twin operation department,
Based on the simulation results for the virtual robot manipulator, control data for controlling the actual robot manipulator is generated, and the control data is transmitted to the actual robot manipulator through the information model.
Controlling the virtual robot manipulator using a program written to generate control data of the virtual robot manipulator based on data obtained from information models of other elements constituting the manufacturing site,
Other factors are:
Includes vision equipment to photograph products that the robot manipulator must pick,
The digital twin operation department,
A robot manipulator control system characterized by identifying product information from a product image acquired through the information model of a vision equipment, and identifying the target position of the product through the product information model based on the identified product information.
delete In claim 1,
Information model operation department and digital twin operation department,
A robot manipulator control system that synchronizes the information model and digital twin environment through Topic communication based on the ROS2 standard.
In claim 1,
The information model is,
A robot manipulator control system characterized by an AAS (Asset Administration Shell) information model.
In claim 1,
The information model operation department,
A robot manipulator control system characterized by transmitting data to an actual robot manipulator using OPC UA (Open Platform Communication Unified Architecture).
modeling a real robot manipulator into an information model;
Steps to build a virtual robot manipulator on the digital twin of the manufacturing floor using the information model of the real robot manipulator:
Receiving motion data from an actual robot manipulator using an information model;
Comprising: controlling a real robot manipulator using a digital twin,
The control step is,
Based on the simulation results for the virtual robot manipulator, control data for controlling the actual robot manipulator is generated, and the control data is transmitted to the actual robot manipulator through the information model.
Controlling the virtual robot manipulator using a program written to generate control data of the virtual robot manipulator based on data obtained from information models of other elements constituting the manufacturing site,
A robot manipulator control method characterized by checking in advance whether there is a collision between the virtual robot manipulator and surrounding elements and calculating how the virtual robot manipulator will move to the target point.
Real robotic manipulators deployed on manufacturing floors;
an information model operating unit that models the actual robot manipulator as an information model and receives motion data from the actual robot manipulator using the information model; and
It includes a digital twin operating unit that builds a virtual robot manipulator on the digital twin of the manufacturing site using the information model of the actual robot manipulator and controls the actual robot manipulator using the digital twin;
The digital twin operation department,
Based on the simulation results for the virtual robot manipulator, control data for controlling the actual robot manipulator is generated, and the control data is transmitted to the actual robot manipulator through the information model.
Controlling the virtual robot manipulator using a program written to generate control data of the virtual robot manipulator based on data obtained from information models of other elements constituting the manufacturing site,
A robot manipulator control system that checks in advance whether there is a collision between the virtual robot manipulator and surrounding elements and calculates how the virtual robot manipulator will move to the target point.
Steps to build a virtual robot manipulator on the digital twin of the manufacturing floor using the information model of the real robot manipulator:
Receiving motion data from an actual robot manipulator using an information model;
Comprising: controlling a real robot manipulator using a digital twin,
The control step is,
Based on the simulation results for the virtual robot manipulator, control data for controlling the actual robot manipulator is generated, and the control data is transmitted to the actual robot manipulator through the information model.
Controlling the virtual robot manipulator using a program written to generate control data of the virtual robot manipulator based on data obtained from information models of other elements constituting the manufacturing site,
A robot manipulator control method characterized by checking in advance whether there is a collision between the virtual robot manipulator and surrounding elements and calculating how the virtual robot manipulator will move to the target point.