KR101760881B1 - Remote apparatus and teaching method of robot thereof - Google Patents

Abstract

A remote terminal and a robot teaching method using the same are disclosed. A remote terminal and a robot teaching method using the same according to the present invention are a remote terminal connected to a robot for generating a map by recognizing a position and a camera section for photographing an environment, A communication section for receiving the image information photographed by the camera section of the robot and the generated map information, the image information received from the communication section, the map information, and the basic operation information stored in the memory section And a job generating unit for generating a new job by combining at least one basic operation selected by a user interface unit and a user interface unit, which are provided to a user and select a necessary basic operation.
Therefore, the present invention combines the basic operations already programmed to generate a high-level task and transmit it to the robot in real time, thereby enabling the robot to handle more various tasks in addition to the basic operation.

Description

TECHNICAL FIELD [0001] The present invention relates to a remote terminal and a robot teaching method using the remote terminal.

The present invention relates to a remote terminal and a robot teaching method using the same, and more particularly, to a remote terminal capable of teaching a robot a new task in real time and a robot teaching method using the remote terminal.

Robot technology has been developed in the 1990s with the transition from industrial robots to non-industrial robots, but there are still many problems to be commercialized.

In particular, in order for the service robot to provide the service desired by the user, the robot must be able to understand the given command and recognize the surrounding environment and to judge for itself.

However, there is a disadvantage in that the programmer has to predict and program all the cases that may occur in advance, and as a result, there is a problem that the programmer can not process a situation that is not predicted by the programmer.

Therefore, the importance of teaching the robot to work in real time emerges. In the past, Teach Pendant method used in industrial robots was used. However, this method has a problem in that a series of operations are performed by the unilateral instruction of the user by using buttons to which various commands of low level are mapped, and most of the movement of the robot arm by its behavior and motor control is also a problem.

In addition, there is a problem that the robot learns for a long time and constructs the task through instructions of the user's simple truth / falsehood, but there are also problems that are not suitable for teaching the robot in real time.

Therefore, there is a need for a method of instructing a robot to work effectively so as to perform complex high-level tasks with various purposes by moving away from industrial robots.

According to an aspect of the present invention, there is provided a remote terminal for generating a new job with a combination of basic operations stored in advance and transmitting the created job to a robot, and a robot teaching method using the remote terminal.

According to one aspect of the present invention, there is provided a remote terminal including a camera unit for photographing an environment, a robot for recognizing a position and generating a map, a remote controller connected to the robot through a wireless network, A terminal, comprising: a memory unit for storing a plurality of basic operations that can be performed; a communication unit for receiving image information photographed from a camera unit of the robot and generated map information; image information received from the communication unit; map information; And a job creation unit for creating a new job by combining at least one basic operation selected by a user interface unit and a user interface unit, which provide information to a user and select a necessary basic operation.

On the other hand, the user interface unit can receive detailed information on the basic operation selected by the user, which is information on the robot's operation position, operation object, and operation contents.

In addition, the communication unit transmits the command of the basic operation in which the detailed information is input to the robot, receives the image information of the robot performing the basic operation, and the user interface unit provides the image information of the robot performing the basic operation to the user, Feedback on the results can be obtained.

In addition, the job creation unit may store the basic operation if the execution result of the basic operation inputted from the user interface unit is 'successful', and may delete the basic operation if it is 'failure'.

In addition, the user interface unit receives the additional basic operation from the user, and the job creating unit additionally stores the additional basic operation in the pre-stored basic operation if the additional basic operation exists, and sets the pre-stored basic operation as the final operation And can be stored in the memory unit.

According to another aspect of the present invention, there is provided a robot teaching method using a robot for capturing an environment and generating a map by recognizing a position and a remote terminal connected through a wireless network, Receiving the image information of the surrounding environment and the generated map information from the robot, inputting to generate a new job based on the image information and the map information received from the user, and when a new job creation is inputted, And receives detailed information, which is information on the robot's operation position, operation target, and operation details, with respect to the selected basic operation, and transmits the basic operation in which the detailed information is inputted to the robot, Receives feedback on the performance result of the operation, and stores the basic operation according to the feedback result.

Meanwhile, if the result of the basic operation is 'successful', the basic operation is stored. If the result of the basic operation is 'successful', the basic operation can be deleted.

In addition, if an additional basic operation is input from the user, the additional basic operation is additionally stored in the preliminarily stored basic operation, and if not, the preliminarily stored basic operation is set as the final operation and stored.

In addition, if there is an additional basic action, the additional basic action is selected from a plurality of pre-stored basic actions, the detailed information about the selected additional basic action is received, and the additional basic action to which the detailed information is inputted is transmitted to the robot, And the process of storing the additional basic operation according to the feedback result can be repeatedly performed.

The present invention connects a remote terminal to a service robot requiring more various and precise behavior control, and combines basic operations already programmed through a remote terminal to generate a high-level task so that the robot performs more various operations So that it can be processed.

In addition, by inputting detailed information of the basic operation to the touch panel of the remote terminal by utilizing the real time information provided by the robot such as image and map information, the real time property, usability and convenience of the robot operation teaching can be improved.

Therefore, it is effective for the user to feel the intelligence of the robot close to the human being.

1 is a block diagram of a robot teaching system according to an embodiment of the present invention.
2 is a block diagram of the robot teaching system shown in Fig.
3 is a diagram illustrating a remote terminal according to an exemplary embodiment of the present invention.
4 is a flowchart illustrating a robot teaching method according to another embodiment of the present invention.
5A is a diagram illustrating a screen for creating a job in the user interface of the remote terminal shown in FIG.
5B is a diagram illustrating a screen for creating a job in the user interface of the remote terminal shown in FIG.
FIG. 5C is a diagram illustrating a screen for creating a job in the user interface of the remote terminal shown in FIG. 3. FIG.
5D is a diagram illustrating a screen for creating a job in the user interface of the remote terminal shown in FIG.
FIG. 5E is a diagram illustrating a screen for creating a job in the user interface of the remote terminal shown in FIG. 3. FIG.
FIG. 5F is a diagram illustrating a screen for creating a job in the user interface of the remote terminal shown in FIG. 3. FIG.

Hereinafter, a preferred embodiment of a remote terminal and a robot teaching method using the same according to the present invention will be described with reference to FIGS. 1 to 5F.

FIG. 1 is a block diagram of a robot teaching system according to an embodiment of the present invention. FIG. 2 is a block diagram of the robot teaching system shown in FIG. 1. Referring to FIG.

The robot teaching system according to the present embodiment includes a remote terminal 100 that receives information of the robot 200 and allows a user to instruct the robot 200 according to the instruction, And a communication network 300 for providing a data transmission / reception path between the remote terminal 100 and the robot 200.

The remote terminal 100 includes a user interface unit 110, a control unit 120, a memory unit 130, and a communication unit 140.

The user interface unit 110 is constituted by a touch panel as an input / output unit of the remote terminal 100, and outputs contents or command execution results input to the touch panel.

In particular, the user interface unit 110 of the present embodiment outputs image information and map information received from the robot 200, and allows the user to create a job based on the image information and the map information.

Meanwhile, the job is generated by a combination of the basic operations stored in the memory unit 130, and the user interface unit 110 provides necessary information to the user and receives information from the user to generate the job.

The control unit 120 includes a user interface processing unit 121 and a task generation unit 122. [

The user interface processing unit 121 controls signals input to / output from the touch panel of the user interface unit 110. [

The user interface processing unit 121 converts the analog signal input from the touch panel into touch panel data in a digital form, processes the touch panel data, converts the processed data back into analog signals, As shown in FIG. In addition, the user interface processing unit 121 transmits the contents inputted by the user to the touch panel so that the user can confirm the contents.

Specifically, the user interface processing unit 121 outputs the video information, the map information, and the audio information received from the communication unit 140 to the user interface unit 110, or the text information and audio information input from the user interface unit 110 And transmits it to the communication unit 140. Further, the user interface unit 110 outputs the user input to the user interface unit 110 so as to check the change of the screen to be output to the user interface unit 110 according to the input of the user.

When at least one basic operation among a plurality of basic operations stored in the memory unit 130 is selected by the user through the user, the job creation unit 122 generates a basic operation based on the selected basic operation, Save it as a subordinate action.

In addition, when the job is configured with a plurality of sub-basic operations, the job creating unit 122 sorts and stores the sub-basic operations in the order selected by the user. For example, if the user selects one of the basic operations stored in the memory unit 130 in the order of 'move', 'face recognition', and 'speech', the task creation unit 122 ' Quot ;, " face recognition ", and " talking ".

When all the lower basic operations are selected from the user, the task generating unit 122 stores information on the task composed of the selected lower basic operations in the memory unit 130. [

The memory unit 130 stores a plurality of preliminarily programmed basic operations and stores information on a new job generated from the job creation unit 122. [

The communication unit 140 includes a modem for communicating with the robot 200 through a wireless network such as a telephone line or an Internet-only line.

In particular, the communication unit 140 according to the present embodiment receives the image information and the map information from the robot 200, and transmits an operation command signal to the robot 200 to perform a basic operation or a generated operation.

The robot 200 includes a camera unit 210, a control unit 220, and a communication unit 230.

The camera unit 210 may be any one of a charge coupled device (CCD) camera, an infrared (IR) camera, a TOF camera, and a Z-Cam capable of simultaneously inputting 3D depth and two-dimensional pixel information .

Particularly, the camera unit 210 captures the state of the robot 200 and the situation around the robot 200, and sends the obtained video signal to the control unit 220.

The control unit 220 recognizes the position of the robot 200 and generates map information, and controls the robot 200 to be driven according to the command signal received through the communication unit 230. In addition, the control unit 220 may include a memory such as a ROM, a RAM, or a flash memory to store system data, map information, message information, and the like of the robot 200.

Specifically, the control unit 220 recognizes the position of the user and the position of the specific place, registers the same, and creates the map information. Accordingly, the robot 200 calculates a relative direction and a distance between a position of the robot 200 and a position to which the robot 200 moves, thereby enabling the robot 200 to move based on the calculated direction. Such map information can be created by marking itself in a space according to autonomous movement within a certain space.

The control unit 220 converts an operation command received from the communication unit 230 into an operation control signal and analyzes the operation control signal to generate an operation drive signal. Further, by transmitting the operation driving signal to the driving mechanism of the robot 200, the robot 200 is moved forward, backward, leftward and rightward, or the arms and legs of the robot 200 are moved in a predetermined direction.

In addition, the control unit 220 performs predetermined signal processing on the video signal from the camera unit 210, and sends the video signal thus obtained to the communication unit 230.

3 is a diagram illustrating a remote terminal according to an exemplary embodiment of the present invention.

The remote terminal 100 according to the present embodiment receives a video signal captured by the camera unit 210 of the robot 200 and displays the environment image of the surroundings of the robot and the current position of the robot 200 A map screen 20 for displaying the map information displayed on the map screen 20 is displayed. Therefore, the user creates a task to be taught to the robot based on the video screen 10 and the map screen 20. [

The user interface unit 110 displays the work list 30 indicating the basic operation of the robot that has already been programmed and executed at the bottom of the image screen 10 and the map screen 20 to the user so as to create a teaching job do. The user selects a necessary basic operation from the work list 30 based on the current state of the robot based on the image screen 10 and the map screen 20. [

In addition, when the basic operation is selected in the work list 30, a work information input window 40 for inputting detailed information of the selected basic operation is generated.

The user inputs detailed information of at least one of the work position, the work object, and the work contents of the robot 200 in the work information input window 40. [ For example, the user inputs 'move' (job content) to the 'first position' (job position) in the job information input window 40, or 'hello' ) Can be input.

On the other hand, in order for the robot 200 to effectively provide a desired service, it is necessary to perform complicated operations according to the surrounding environment. However, it is inefficient to pre-program various tasks in preparation for any situation that may occur in advance, and therefore a method is required for the user to teach complex high-level tasks in real time.

Accordingly, a method for selecting a basic operation stored in advance through the user interface unit 110 of the remote terminal 100 shown in FIG. 3 and combining the basic operations to teach a more complex task will be described with reference to FIG.

FIG. 4 is a flowchart illustrating a robot teaching method of a remote terminal according to another embodiment of the present invention, and FIGS. 5A to 5F are views illustrating a screen for creating a task in the user interface unit of the remote terminal shown in FIG. 3 .

First, the remote terminal 100 receives the image information about the environment of the robot captured by the camera unit 210 of the robot 200 and the map information showing the location of the robot 200 through the communication unit 140 (410 ).

The user interface unit 110 displays the received image information and map information and provides the user with information on the current state of the robot 200.

Accordingly, the user generates necessary work in real time according to the current state of the robot 200 provided to the remote terminal 100. [

Specifically, if there is no job to be performed (30) in the job list displayed on the user interface unit 110, the user inputs 420 to the user interface unit 110 to create a new job (420).

5A showing a state of the user interface unit 110 at the time of creating a new job, the user first selects 'Add' because there is no job to be performed in the work list 30 (1) Then, a work information input window 40 for entering the name of the work to be added appears. The user inputs the name of the job 'welcome' in the job information input window 40 (2) and selects 'save' to start the generation of the job (3).

When the generation of the job is started, the user selects at least one pre-stored basic operation (430) and gives detailed information to the selected basic operation (440), thereby creating a combined operation of the basic operation.

5B and FIG. 5C, the user selects the basic operation 'Move' (① and ② in FIG. 5B) in the work list 30, and transmits detailed information to be given to 'move' Is input in the information input window 40 (440).

The detailed information to be given to the 'move' corresponds to the movement position and the movement speed. The movement position (① in FIG. 5C) may be directly input by the user on the text, The coordinates corresponding to the selected destination are automatically input to the job information input window 40 (② in FIG. 5C) by selecting a destination to be moved on the screen 20 (② in FIG. 5C).

When the detailed information is inputted, the basic information is stored (step 3 in FIG. 5C), and the basic operation is transmitted to the robot 200 through the communication unit 140 to execute the basic operation (step 450 in FIG. 5C).
The remote terminal 100 receives the execution result of the basic operation again from the robot 200 and displays it on the user interface unit 110. The user feedbacks whether the basic operation has been successfully performed through the displayed execution result (460).

delete

5D, the user confirms the execution result of the basic operation through the image screen 10 of the user interface unit 110, and displays the operation result in the operation information input window 40 Enter whether the execution result is successful (①). If the execution result is successful, the user selects 'Yes' in the job information input window 40, and if it is determined that the basic operation is not the operation intended by the user, the user can select 'No' and receive feedback from the user in real time.

When the user receives a feedback indicating that the basic operation has been successfully performed (Yes at 460), the task creation unit 122 of the remote terminal 100 stores the basic operation according to the feedback result received from the user (470).

Specifically, if the result of the basic operation is successful, the task creation unit 122 stores the basic operation as a subordinate basic operation of the task. However, the failed surface task creation unit 122 deletes the detailed information inputted for the basic operation And then input the necessary basic operation from the user again.

That is, the basic operation is selected until the user receives feedback from the user about the result of performing the basic operation, the detailed information about the selected basic operation is input, and the robot 200 performs the process.

When the lower basic operation constituting the task is stored, the user interface unit 110 receives (480) whether the basic operation to be added is present from the user, and if there is an additional basic operation (Yes at 480) The job creation unit 122 additionally stores (430, 440, 450, 460, 470) additional basic operations as subordinate basic operations, and if there is not at least one subordinate basic operation Is set as the final operation and the final operation is stored in the memory unit 130 (490).

Meanwhile, the additional basic operation is performed in the same manner as described above (430, 440, 450, 460, 470).

5E, the user selects 'Face Recognition' from the work list 30, and assigns 'Face Recognition' to the 'Face Recognition' And inputs the detailed information to be input into the work information input window 40. [

On the other hand, the detailed information to be given to the 'face recognition' is a name, a face, and a position of a person to recognize a face. The position of a person is selected by selecting a place where a person is located on the map screen 20 Coordinates corresponding to the place are automatically input into the work information input window 40 and the name of the person 'Hong Kil-Dong' can be inputted into the text input window. In addition, the face to be recognized can be selected on the video screen 10 displayed on the user interface unit 110 (1, 2).

Accordingly, when the basic operation in which the detailed information is inputted is performed by the robot 200, 'Hong Kil-dong' located at the input location recognizes the face of a person (③, ④).

Referring to FIG. 5F, the steps 480 and 490 for completing job creation will be described. First, when the user selects 'view' for the 'welcome' job created in the work list 30, In the information input window 40, the previously stored lower basic operation of the 'welcome' job and the operation sequence of the basic operation are displayed (Move → Face Recognition → Speech).

In addition, the job information input window 40 is displayed to select whether or not to complete job creation. If there is a basic operation to be added, the user repeats the process of selecting 'No' to add the basic operation, If there is no default action, select 'Yes' to complete job creation (②).

Thus, the robot teaching method according to the present embodiment can generate complex and diverse tasks in real time with a combination of basic operations that can be performed according to the current situation. Therefore, the user has the effect of making the intelligence of the robot close to the human being by only the operation of the remote terminal.

100: remote terminal
200: Robot

Claims (9)

1. A remote terminal connected to a robot via a wireless network, the robot including a camera unit for photographing an environment and generating a map by recognizing a position of the robot,
A memory unit for storing a plurality of basic operations that can be performed;
A communication unit for receiving image information of the photographed surrounding environment from the robot and the generated map information;
A display unit for displaying the image information and the map information received through the communication unit and providing the user with information on the current state of the robot, receiving a new job creation command based on the current state of the robot from the user, Which is the information on the operation position, the operation target, and the operation details of the robot with respect to the selected basic operation, from the user A receiving user interface unit; And
And a control unit for transmitting the execution command of the basic operation to which the detailed information is input to the robot through the communication unit to perform the basic operation,
Wherein the user interface unit displays the result of performing the basic operation received from the robot through the communication unit to the user, receives feedback on the result of performing the basic operation from the user,
And the control unit includes a job generation unit for storing the basic operation according to a result of the feedback. delete delete The method according to claim 1,
Wherein the job creation unit stores the basic operation if the result of the basic operation is 'successful' and deletes the corresponding basic operation if the result of the basic operation is 'failure'. 5. The method of claim 4,
The user interface unit receives input from the user whether or not there is an additional basic operation necessary for performing a new operation,
Wherein the job creation unit further stores the additional basic operation in a preliminarily stored basic operation if the additional basic operation exists and sets the preliminarily stored basic operation as a final operation if the additional basic operation does not exist and stores it in the memory unit Lt; / RTI > A robot teaching method using a remote terminal connected through a wireless network with a robot for capturing an environment and generating a map by recognizing a position,
Receiving image information of the photographed surrounding environment and the generated map information from the robot;
Displaying the image information and the map information to provide a user with information on the current state of the robot;
Receiving a new job creation command based on the current state of the robot from the user;
Selecting a basic operation necessary for performing the new operation from among a plurality of basic operations stored in advance from the user;
Receiving details of the selected basic operation from the user, the detailed information being information on a work position, an operation target, and a work content of the robot;
The basic operation is performed by transmitting an execution command of the basic operation to which the detailed information is inputted to the robot;
Receiving a result of performing the basic operation from the robot and displaying it to the user;
Receiving feedback from the user on the result of performing the basic operation;
And the basic operation is stored according to a result of the feedback. The method according to claim 6,
Storing the basic operation according to a result of the feedback includes storing the basic operation if the result of the basic operation is 'successful', and deleting the corresponding basic operation if the result of the basic operation is 'failure' Robot teaching method. 8. The method of claim 7,
Receiving, from the user, whether there are additional basic operations required to perform a new task;
Further storing the additional basic operation in a pre-stored basic operation if the additional basic operation exists, and storing the pre-stored basic operation as a final operation if the additional basic operation does not exist, . 9. The method of claim 8,
If the additional basic operation exists,
Selecting the additional basic operation from among the plurality of basic operations stored in advance from the user;
Receiving the detailed information from the user for the selected additional basic operation;
Transmitting an additional basic operation execution command to which the detailed information is input to the robot so that the additional basic operation is performed;
Receiving the result of performing the additional basic operation from the robot and displaying it to the user;
Receiving feedback from the user about the result of performing the additional basic operation;
And the step of storing the additional basic operation is repeated according to a result of the feedback.

Images