KR102222468B1 - Interaction System and Interaction Method for Human-Robot Interaction - Google Patents

Abstract

An interaction system for human-robot interaction is provided. An interaction system for human-robot interaction includes: a work command receiving unit for receiving a work command; A work command inference unit for inferring the work command and determining at least one interaction type constituting the work command; And a robot motion command generation unit that generates a robot motion command based on the determined one or more interaction types, and provides the generated robot motion command to the robot.

Description

Interaction System and Interaction Method for Human-Robot Interaction}

The present invention relates to an interaction system and method for human-robot interaction (HRI), and in an interface supporting human-robot interaction, an interaction system and method capable of supporting human-centered interaction It is about.

[Explanation of nationally supported R&D]

This study is under the management of the National Research Foundation of Korea, "A Study on the Development of Hand-Based Seamless CoUI Technology for Collaboration between Remote Users" of the Global Frontier Support (R&D) Project of the Ministry of Science and ICT. 2011-0031425).

With the advancement of robot technology, many tasks that humans used in the industrial field are being replaced by robots, manipulators, etc., and the industry is developing in the direction that humans cooperate with, control, and manage robots and manipulators. . In addition, cases where robots are applied not only to industrial fields, but also to medical services, housekeeping services, and social services in which human interaction is important are increasing. Therefore, the interaction between humans and robots is becoming more important.

In order for the human-robot interaction to be efficiently performed, human-robot communication, that is, human commands must be naturally transmitted to the robot. However, since the language used by humans and the language used by robots are different, humans are inconvenient to provide detailed commands centering on the movement of the robot in order to transmit commands to the robot. For example, a human wants the robot to perform an action of "Put object A on object B", but the details of the robot that have set specific coordinates in consideration of the physical characteristics of the robot, such as position, speed, acceleration, torque, etc. Commands related to motion must be transmitted to the robot. In other words, due to the discrepancy between the rather abstract human language instructing the overall robot's work content and the robot's language for the detailed physical motions of the robot, it is somewhat limited for the efficient interaction between humans and robots have.

Therefore, there is a need for a system and method capable of supporting human-centered interaction for more efficient interaction between humans and robots.

M.R. Cutkosky, On Grasp Choice, Grasp Models, and the Design of Hands for Manufacturing Tasks, IEEE Trans. on Robotics and Automation, Vol. 5, No. 3, 1989

The present invention has been devised to solve the above-described problems, and provides an interaction system and method for human-robot interaction that can support human-centered interaction for more efficient interaction between human and robot.

An interaction system for human-robot interaction according to an embodiment of the present specification includes: a work command receiving unit for receiving a work command; A work command inference unit for inferring the work command and determining at least one interaction type constituting the work command; And a robot motion command generation unit that generates a robot motion command based on the determined one or more interaction types, and provides the generated robot motion command to the robot.

In one embodiment, the work command inference unit first classifies the work command into at least one sub-task, classifies the classified at least one sub-task into the at least one or more interaction types, and determines the work structure of the work command. You can decide.

In one embodiment, further comprising a database unit including an interaction type generation interface, robot information, object information, and human information, wherein the work command inference unit is the robot information read out from the database unit, The at least one interaction type may be determined by further considering the object information and the human information.

In an embodiment, the work command inference unit may determine the work structure of the work command by further considering the robot information, object information, and human information.

In an embodiment, current object information, current robot information, and current human information reflecting a result of the robot operation according to the robot operation command may be fed back to the database and stored.

In one embodiment, the robot motion command generator generates the robot motion command in consideration of a characteristic of the robot and an error generated by performing the interaction type, and the current robot is properly changed while appropriately changing a parameter applied to the interaction type. Estimates can be made as to whether the work order can be properly performed.

An interaction method for human-robot interaction according to another embodiment of the present invention includes the steps of: receiving a work instruction; Inferring the work command to determine at least one interaction type constituting the work command; And generating a robot motion command based on the determined one or more interaction types, and providing the generated robot motion command to the robot.

In one embodiment, the step of determining at least one interaction type constituting the work command by inferring the work command includes first classifying the work command into at least one sub-task, and the classified at least one sub-task Classifying the at least one interaction type may include determining a work structure of the work command.

In one embodiment, the step of inferring the work command and determining at least one or more interaction types constituting the work command may include reading an interaction type generation interface, robot information, object information, and human information from the database unit. -Out), and determining the at least one interaction type by further considering the robot information, the object information, and the human information.

In one embodiment, the step of inferring the work command and determining at least one or more interaction types constituting the work command may include reading an interaction type generation interface, robot information, object information, and human information from the database unit. -Out), and determining the work structure of the work command in consideration of the robot information, the object information, and the human information.

In an embodiment, after the operation of the robot is performed according to the provided robot operation command, the step of feeding back and storing object information, robot information, and human information to a database unit may be further included.

In one embodiment, generating a robot motion command based on the determined one or more interaction types and providing the generated robot motion command to the robot includes: The robot motion command may be generated in consideration of, and may include estimating whether the current robot can properly perform the work command while appropriately changing a parameter applied to the interaction type.

A storage medium according to another embodiment of the present invention is stored in a medium to execute the interaction method for human-robot interaction according to any one of claims 7 to 12 by being combined with hardware.

In the interaction system and method for human-robot interaction according to an embodiment of the present invention, a work command can be inferred and provided to a robot by generating an instruction in an interaction type unit, which is a minimum unit of interaction, and the robot is an interaction type unit. Based on the command, you will perform the indicated task.

In other words, since commands are delivered in the smallest unit of meaning of interaction between humans and robots, convenience for humans to instruct tasks can be improved. In addition, as the interaction type unit, which is the smallest semantic unit of interaction, is used, the target of the task becomes clear and the robot can more easily understand the entire task, thereby performing an efficient human-robot interaction.

1 is an exemplary diagram for explaining a relationship between an interaction system for human-robot interaction, a human and a robot.
2 is a block diagram of an interaction system for human-robot interaction.
3 is a structural diagram showing a task structure (Task Tree) of a task command.
4 is a flowchart of an interaction method for human-robot interaction according to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The detailed description of the present invention described below refers to the accompanying drawings, which illustrate specific embodiments in which the present invention may be practiced. These embodiments are described in detail enough to enable a person skilled in the art to practice the present invention. The various embodiments of the present invention are different from each other, but need not be mutually exclusive. For example, specific shapes, structures, and characteristics described herein may be implemented in other embodiments without departing from the spirit and scope of the present invention in relation to one embodiment. Further, the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the present invention. Accordingly, the detailed description to be described below is not described in a limiting sense, and the scope of the present invention is limited only by the appended claims along with all scopes equivalent to those claimed by the claims. Like reference numerals in the drawings refer to the same or similar functions in various aspects.

The terms used in the present specification have selected general terms that are currently widely used as possible while considering functions, but this may vary according to the intention or custom of a technician working in the field, or the emergence of new technologies. In addition, in certain cases, there are terms arbitrarily selected by the applicant, and in this case, the meaning will be described in the description of the corresponding specification. Therefore, terms used in the present specification should be interpreted based on the actual meaning of the term and the entire contents of the present specification, not a simple name of the term.

1 is an exemplary diagram for explaining a relationship between an interaction system 10 for human-robot interaction, a human H, and a robot R.

Referring to FIG. 1, the interaction system for human-robot interaction may be a system for controlling an operation of a robot R according to a command of a human H. Here, the robot R may be a robot arm or a manipulator, but is not limited thereto, and may be a service robot that performs various services (medical services, housekeeping services, etc.) performed by humans as a robot having a human form. . The present invention provides a human-centered interaction system in which a human (H) and a robot (R) can interact more effectively and efficiently. That is, the human (H) can transmit a human-centered somewhat abstract work command to the robot (R) to the interaction system 10 for human-robot interaction, and the interaction system 10 for human-robot interaction. Can infer this, generate a command suitable for the robot's motion, and provide it to the robot. The interaction system for human-robot interaction of the present invention may be a system included in a robot, but is not limited thereto, and may be implemented as a system separate from the robot R. Hereinafter, the configuration and operation of the interaction system 10 for human-robot interaction according to the present embodiment will be described in more detail.

2 is a block diagram of an interaction system 10 for human-robot interaction. 3 is a structural diagram showing a task structure (Task Tree) of a task command.

2 and 3, the interaction system 10 for human-robot interaction includes a work command reception unit 110, a work command inference unit 120, a database unit 130, and a robot motion command generation unit ( 140).

The interaction system 10 for human-robot interaction according to the embodiments and each device or unit constituting the same may have an aspect that is entirely hardware or partially hardware and partially software. For example, each component of the interaction system 10 for human-robot interaction refers to a combination of hardware and software driven by the hardware. The hardware may be a data processing device including a central processing unit (CPU) or another processor. In addition, software driven by hardware may refer to an executing process, an object, an executable file, a thread of execution, a program, and the like. For example, the work command inference unit 120 may refer to a combination of hardware for inferring a work command and software for the same.

In addition, in the present specification, each unit constituting the interaction system 10 for human-robot interaction is not intended to refer to separate components that are physically separated. That is, in FIG. 2, each part of the interaction system 10 for human-robot interaction is shown as separate blocks, which are distinguished from each other, but this allows the interaction system 10 for human-robot interaction to be executed by the operation. It is functionally classified by Depending on the embodiment, some or all of the above-described units may be integrated into the same single device, or one or more units may be implemented as separate devices that are physically separated from other units. For example, each unit may be components that are communicatively connected to each other in a distributed computing environment.

The work command receiving unit 110 receives a work command. The work command may be a command for a work to be performed by the robot. For example, it may be a command for a rather abstract task, such as delivering a specific object to a specific human, not a command that causes the robot to move to a specific coordinate. The command for the entire task that the robot must perform may be a more familiar communication method to humans, and may be a human-centered interaction means rather than a robot-centric.

The work order may be an order provided by a human. The work command may be provided through an input means such as a keyboard or a mouse, but is not limited thereto, and may be generated by recognizing a human voice or a human motion. The work command receiving unit 110 may include at least one of an input module for inputting a work command, a voice recognition module for recognizing a human voice, and a motion recognition module for recognizing a human motion. The work command received by the work command receiving unit 110 is provided to the work command inferring unit 120.

The work command inference unit 120 infers the work command and determines a work structure (Task Tree). The work order consists of at least one or more subtasks, and each subtask consists of at least one or more interaction types. The work command inference unit 120 determines at least one or more sub-tasks constituting the work command, and determines at least one or more interaction types required to perform the determined sub-task.

For example, "transfer object A to user 1"-The work command may be composed of three sub-tasks: lifting object A, handing object A to user 1, and home position (GoToHome). Lifting Object A-In order to perform the sub-task, three types of interaction may be required: MoveTo, Grasp, and LiftUp. After receiving the work command "transfer object A to user 1", the work command inference unit 120 determines a plurality of sub-tasks as described above and an interaction type required to perform each sub-task.

Here, in the present embodiment, an interaction type such as MoveTo, Grasp, and LiftUp may be a command of a minimum unit that causes an interaction between a human and a robot. The interaction system 10 for human-robot interaction can infer a work command, generate a command in an interaction type unit, which is the smallest unit of interaction, and provide it to the robot. Will be performed. In other words, since commands are delivered in the smallest unit of meaning of interaction between humans and robots, convenience for humans to instruct tasks can be improved. In addition, as the interaction type unit, which is the smallest semantic unit of interaction, is used, the target of the task becomes clear and the robot can understand the whole task more easily.

Such an interaction type may be a predefined state for natural interaction with the robot. In addition, detailed operations of the robot for the robot to perform the defined interaction may be in a state defined according to each interaction type. These interaction types can be added by being created inside the system or outside the system. The interaction type interface is in a state stored in the database unit 130, and the work command inference unit 120 dynamically determines an interaction type suitable for the current work command from among the interaction type interfaces stored in the database unit 130. You can decide.

The database unit 130 may include an interaction type interface, robot information, object information, and human information. The information stored in the database unit 130 may be updated by reflecting the latest state of a robot, an object, or a human after the robot executes a work command.

The robot information refers to information related to a robot performing a task, and includes at least information on the position of the robot. Here, the object is an object included in an environment in which the robot and the human belong, and means an object capable of interacting with the robot and/or human. The object information includes location information of all objects located in the environment, and physical information such as size and weight. The human information refers to information on the robot and a human capable of interacting with an object, and includes at least human location information and a location where the human interaction is performed (eg, the location of a hand).

The work command inference unit 120 may determine the work structure of the work command by further considering robot information, object information, human information, and the like. That is, the work command inference unit 120 may additionally generate a sub task required to perform the work command by further considering robot information, object information, human information, and the like. As shown in FIG. 3, the work command provided by the work command receiving unit 110 is "transmit object A to user 1", or object B is selected in consideration of information on an object located under object B. You can create additional subtasks to move. That is, the work command inference unit 120 considers information on a robot, an object, a human, and the like, and performs a sub-task for performing a work command for transferring the object A to the user 1, "Lifting the object B, putting down the object B, and Lifting A, passing object A to user 1, and going to home (GoToHome).

In addition, the work command inference unit 120 may dynamically determine an interaction type required to perform a sub-task by further considering robot information, object information, and human information stored in the database unit 130.

The interaction type may have different values set according to at least one of information on a work command, a robot, a human, and an object. In other words, the LiftUp motion of Lifting Object A-Lifting Sub-task and Lifting Lifting Object B-Lifting Sub-task consist of the same LiftUp interface, but the LiftUp motion of lifting Object A and LiftUp motion of lifting Object B are detailed. There are differences in parameters.

As another example, even if the interaction type corresponding to the same MoveTo, the moving direction and speed may be different depending on the object and the object. In addition, when there is an obstacle that is expected to collide in the moving process, MovoTo must be created to avoid the obstacle. In addition, when the object A is handed to the user, the HandOver interaction type may be changed according to the location where the interaction with the human is performed. That is, interaction types such as MoveTo and HandOver must be dynamically generated to have different parameter values according to the work command and the surrounding environment. Accordingly, the work command inference unit 120 reads out information and an interaction type interface stored in the database unit 130, such as robots, objects, and humans, and provides a plurality of interactions constituting the currently provided work command. Types can be created dynamically.

The work command inference unit 120 provides information on the interaction type dynamically generated through inference on the work command to the robot operation command generation unit 140.

The robot motion command generation unit 140 may determine an order of robot tasks so that the robot performs a plurality of dynamically generated interaction types, and may generate a robot motion command. The robot motion command generation unit 140 may plan the motion of the robot based on the dynamically generated interaction type, and may plan a process in which each interaction type is performed as a whole. The robot motion command generation unit 140 may provide the generated robot motion command to the robot.

In addition, the robot motion command generation unit 140 may generate a robot motion command to be provided to the robot in consideration of an error generated by performing a characteristic of the robot and an interaction type. That is, the robot motion command generation unit 140 may estimate the motion of the robot according to the generated robot motion command. The robot motion command generation unit 140 may estimate whether the current robot can properly perform a work command while appropriately changing a parameter applied to each interaction type unit. The interaction system 10 for human-robot interaction according to the present embodiment generates a robot motion command in units of interaction type, so that the estimated logic configuration can be united and the number of parameters can be minimized.

The robot can perform the entire task according to the robot motion command, which is a combination of interaction types, and the object, human, and current information of the robot reflecting the result of the robot's task execution can be fed back to the database unit 130 and stored. It can be used to dynamically create an interaction type corresponding to the next work command.

Hereinafter, an interaction method for human-robot interaction according to another embodiment of the present invention will be described.

4 is a flowchart of an interaction method for human-robot interaction according to an embodiment of the present invention. The interaction method for human-robot interaction is a method performed in the interaction system for human-robot interaction according to FIG. 2, and the description of FIG. 2 may be referred to.

4, the interaction method for human-robot interaction includes receiving a work command (S100), inferring a work command to determine at least one interaction type constituting the work command (S110), and And generating a robot motion command based on the determined one or more interaction types, and providing the generated robot motion command to the robot (S120).

First, a work command is received (S100).

Receiving the work command may be performed by the work command receiving unit 110. The work order may be an order provided by a human. The work command may be provided through an input means such as a keyboard or a mouse, but is not limited thereto, and may be generated and provided by recognizing a human voice or a human motion.

Subsequently, at least one interaction type constituting the work command is determined by inferring a work command (S110).

The step of determining the work structure by inferring the work command may be performed by the work command inference unit 120. The work order consists of at least one or more subtasks, and each subtask consists of at least one or more interaction types. The work command inference unit 120 determines at least one or more sub-tasks constituting the work command, and determines at least one or more interaction types required to perform the determined sub-task. The interaction type may be a command of a minimum unit that causes an interaction between a human and a robot. Such an interaction type may be a predefined state for natural interaction with the robot. That is, the interaction type interface is in a state stored in the database unit 130, and the work command inference unit 120 infers an interaction type suitable for the current work command from among the interaction type interfaces stored in the database unit 130. Can be determined dynamically.

In the step of determining at least one interaction type constituting the work command by inferring the work command (S110), the interaction type generation interface, robot information, object information, and human information are read from the database unit (Read-Out ), and further considers the robot information, the object information, and the human information to determine the work structure of the work command.

In addition, in the step of inferring the work command and determining at least one interaction type constituting the work command (S110), the interaction type generation interface, robot information, object information, and human information are read from the database unit. -Out), and further considering the robot information, the object information, and the human information to determine the at least one interaction type.

Next, a robot motion command is generated based on the determined one or more interaction types, and the generated robot motion command is provided to the robot (S120).

The order of robot tasks may be determined so that the robot performs a plurality of dynamically generated interaction types, and a robot motion command may be generated. This step is performed by the robot motion command generation unit 140. The robot motion command generation unit 140 may plan the motion of the robot based on the dynamically generated interaction type, and may plan a process in which each interaction type is performed as a whole. The robot motion command generation unit 140 may provide the generated robot motion command to the robot.

In addition, the step (S120) of generating a robot motion command based on the determined one or more interaction types and providing the generated robot motion command to the robot (S120) includes: The robot motion command may be generated in consideration of, and may include estimating whether the current robot can properly perform the work command while appropriately changing a parameter applied to the interaction type.

In the interaction method for human-robot interaction according to the present embodiment, after the operation of the robot is performed according to the provided robot operation command, object information, robot information, and human information are fed back to and stored in a database unit. It may contain more.

That is, according to the robot operation command, the robot can perform the entire operation, and the object, human, and current information of the robot reflecting the result of the operation of the robot can be fed back to the database unit 130 and stored. It can be used to dynamically create interaction types corresponding to work orders.

The operation by the interaction method for human-robot interaction according to the above-described embodiments may be implemented at least partially as a computer program and recorded on a computer-readable recording medium. A program for implementing an operation by an interaction method for human-robot interaction according to the embodiments is recorded, and the computer-readable recording medium includes all types of recording devices in which data that can be read by a computer is stored. do. Examples of computer-readable recording media include ROM, RAM, CD-ROM, magnetic tapes, floppy disks, and optical data storage devices. In addition, the computer-readable recording medium may be distributed over a computer system connected through a network, and computer-readable codes may be stored and executed in a distributed manner. In addition, functional programs, codes, and code segments for implementing this embodiment may be easily understood by those of ordinary skill in the art to which this embodiment belongs.

Although described above with reference to examples, the present invention should not be construed as being limited by these examples or drawings, and those skilled in the art will be able to understand the spirit and scope of the present invention described in the following claims. It will be appreciated that various modifications and changes can be made to the present invention without departing from.

10: Interaction system for human-robot interaction
110: work command receiver
120: work order reasoning unit
130: database unit
140: robot motion command generation unit

Claims (7)

A database unit including an interaction type generation interface, robot information, object information, and human information;
A work command receiving unit for receiving a work command;
Generation of the interaction type for at least one interaction type for the robot to perform the work command by inferring the work command, the robot information read-out from the database unit, the object information, and the human information A work command inference unit dynamically determined by the interface; And
And a robot motion command generation unit that generates a robot motion command based on the determined one or more interaction types, and provides the generated robot motion command to the robot,
The interaction type is a command of a minimum semantic unit for causing an interaction between a human and a robot, and the type and number of interaction types determined according to at least one of the robot information, the object information, and the human information even if the same work command At least one of the parameters to be applied is different,
Current object information, current robot information, and current human information reflecting the result of the operation of the robot according to the robot motion command are fed back to the database and stored
The work command inference unit dynamically determines an interaction type corresponding to the next work command and a parameter applied to the interaction type based on the current object information, current robot information, and current human information stored by being fed back to the database unit,
The robot motion command generator considers the characteristics of the robot and the error generated by performing the interaction type, and estimates whether the current robot can perform the work command while changing a parameter applied to the interaction type. The interaction system for human-robot interaction, characterized in that to generate the robot motion command. The method of claim 1,
The work command inference unit first classifies the work command into at least one sub-task, classifies the classified at least one sub-task into the at least one or more interaction types, and determines a work structure of the work command. Interaction system for human-robot interaction. The method of claim 2,
The work command inference unit further considers the robot information, object information, and human information to determine a work structure of the work command. Receiving a work instruction;
From the database unit, the interaction type generation interface, robot information, object information, and human information are read out,
Inferring the work command, the robot information, the object information, and the human information, and dynamically determining at least one interaction type for the robot to perform the work command in the interaction type generation interface;
Generating a robot motion command based on the determined one or more interaction types, and providing the generated robot motion command to the robot; And
After the operation of the robot is performed according to the provided robot operation command, object information, robot information, and human information are fed back to and stored in the database unit,
The interaction type is a command of a minimum semantic unit for causing an interaction between a human and a robot, and the type and number of interaction types determined according to at least one of the robot information, the object information, and the human information even if the same work command At least one of the parameters to be applied is different,
The current object information, current robot information, and current human information, which are fed back and stored in the database unit, are used to dynamically determine an interaction type corresponding to the next work command and a parameter applied to the interaction type,
Generating a robot motion command based on the determined one or more interaction types, and providing the generated robot motion command to the robot,
The robot operation command is performed by considering the characteristics of the robot and the error generated by performing the interaction type, and estimating whether the current robot can perform the work command while changing a parameter applied to the interaction type. Interaction method for human-robot interaction comprising generating. The method of claim 4,
Dynamically determining at least one or more interaction types constituting the work command,
Human-robot interaction comprising first classifying the work command into at least one or more sub-tasks, classifying the classified at least one or more sub-tasks into the at least one or more interaction types, and determining a work structure of the work command Interaction method for The method of claim 5,
Dynamically determining at least one or more interaction types constituting the work command,
The database unit reads out the interaction type generation interface, robot information, object information, and human information, and further considers the robot information, the object information, and the human information to determine the work structure of the work command. Including determining the interaction method for human-robot interaction. A computer program stored in a medium to execute the interaction method for human-robot interaction according to any one of claims 4 to 6 in combination with hardware.

Images