KR100757500B1 - System and method for interfacing between human and service robot based on action - Google Patents

Abstract

The present invention relates to a motion-based interface system and method between a service robot and a person, and transmits a signal for a user to control a robot, etc. with a simple operation, and recognizes the transmitted signal as an individual motion suitable for a situation. By using the command interpretation table to interpret the command according to the situation and application to configure the robot to operate, there is an effect that the robot and human can quickly interface based on various movements.

Robot, interface, neural network, motion

Description

Motion-based interface system and method between service robot and man {SYSTEM AND METHOD FOR INTERFACING BETWEEN HUMAN AND SERVICE ROBOT BASED ON ACTION}

1 is a block diagram schematically illustrating an operation-based interface system according to the present invention;

2 is a block diagram showing a configuration of an operation signal input unit according to the present invention;

3 is a block diagram showing the configuration of an operation recognition unit, an instruction analysis unit and a situation management unit according to the present invention;

4 is a diagram showing the configuration of robot application software according to the present invention;

5 is a flowchart schematically illustrating an operation-based interface method according to the present invention.

<Description of the code for the main part of the figure>

100: operation signal input unit 200: operation recognition unit

230: neural network 240: neural network selector

300: command analysis unit 320: operation command analysis table

400: situation management unit 410: management unit

500: robot 600: video output device

700: voice output device 800: robot application software

The present invention relates to an interface between a service robot and a user. More particularly, the present invention relates to an interface between a service robot and a person, in which a user and a service robot can interact by performing a calculation operation on the basis of a motion recognized by the user. It relates to a motion-based interface system and method.

What distinguishes the robot from conventional information equipment is that it has an operating device such as a driving device that can move, a robot arm that can lift an object, and a mechanism that can adjust the angle of the built-in camera. These actuators have evolved from industrial and military robotics.

In general, industrial and military robots operate according to a pre-written program, so it is difficult to control the movement to interact with the user after the operation is started. However, service robots for the convenience of humans require intelligent work performance and interaction with users that can respond appropriately to various needs of users. In addition, recently, service robots have various functions similar to those of computers, and thus, a interaction method similar to GUIs provided by computer software is required. Thus, various user interface devices are devised and installed in the service robot to satisfy the needs and to perform interaction between the user and the service robot.

The interface device between the robot and the user can be broadly divided into an output device, an input device, and a bidirectional device.

The output device mainly outputs information using video and audio. The image output device requires a lot of space inside the robot to be mounted on the robot, and also consumes a lot of power and has a problem in that the distance from the user must be close to use the output image.

The audio output device has advantages in terms of space and power consumption compared to the video output device, and has recently been adopted because it has a function of delivering information to the user using voice synthesis technology.

Input interface devices existing inside the robot include various buttons or touch screens. However, in order to input information using a button and a touch screen, there is a problem that the user and the robot must be in a very close position.

A method of wirelessly giving a command to a robot by using a remote controller, a wireless mouse, and a wireless keyboard has a problem in that the device becomes large and heavy when it is trying to perform various functions, and thus the portability is poor.

In case of inputting the image through the camera recently, it is possible to recognize the user's face and motion by using image recognition software. However, due to the characteristics of the camera, there is spatial and optical limitation in the input range. There is a problem that there is a technical limitation in applying image recognition technology to dynamic environment.

In case of inputting voice using a microphone, voice recognition software and voice synthesizer (TTS) can be used to communicate commands or communicate between the robot and the user, but it is difficult to expect a natural conversation due to a misunderstanding problem. There is.

In recent years, mobile service robots tend to provide functions through a network by separating various functions to the outside except the minimum functions. Using this network has the effect of lowering the price and power consumption of the robot. However, since network-based robots cannot handle image recognition or voice recognition on their own, they must move to the limited input range of input devices such as cameras or microphones even in the inconvenience and inconvenience of having to request processing from a remote server whenever necessary. Even after inputting, there is a problem that the user must experience inconvenience due to delay and misrecognition.

Accordingly, the present invention has been made to solve the above problems of the prior art, an object of the present invention is to detect the user's motion as an input signal to recognize the individual operation is set according to the situation to output the command by rapid operation The present invention provides a motion-based interface system and method between a service robot and a person that can be quickly interfaced by a simple operation without limiting the distance between the input device and the user by allowing the user and the robot to interact.

On the other hand, another object of the present invention is that the user can directly mount the device that can detect the signal is not limited to the input range and can be carried lightly and also suitable for the situation according to the robot and robot applications Motion-based interface system and method between service robot and human being that can recognize various motions and process the corresponding commands quickly by using the registered analysis table to enable various interactions and increase the recognition rate of user's motions To provide.

An operation-based interface system between a service robot and a person of the present invention for achieving the above object is a motion signal input unit and a motion signal input unit for detecting a user's motion as a signal through a motion sensor and transmitting the detected signal. Received signal transmitted from the motion recognition unit for interpreting the signal to recognize the individual operation according to the situation and the command interpretation unit for interpreting the recognized individual operation as a command suitable for the situation and application and the command interpreter Characterized in that it comprises a robot and robot applications that operate according to the information contained in the command.

At this time, the operation recognition unit and the command analysis unit is characterized in that it is controlled by the situation management unit for managing and transmitting the situation according to the robot application.

On the other hand, the motion-based interface method between the service robot and the person of the present invention detects the user's motion as a signal through a motion sensor, and transmitting the detected signal and receiving the transmitted signal and interpret the signal Including the step of recognizing the individual motion in accordance with the situation, the step of interpreting the recognized individual motion as a command suitable for the situation and application, and the operation of the robot and the robot application according to the information included in the interpreted command. It features.

At this time, the step of recognizing the individual operation and the step of interpreting the command is characterized in that it is controlled by the situation management unit for managing and transmitting the situation according to the robot application.

Hereinafter, a motion-based interface system and a method between a service robot and a person of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically illustrating an operation-based interface system according to the present invention.

The motion-based interface system between a service robot and a person of the present invention detects a user's motion and transmits a detected signal to the motion signal input unit 100, and interprets the detected signal to recognize the motion as a separate motion 200. Command interpreting unit 300 for interpreting the recognized individual motions as appropriate commands for the situation, the situation to control the operation recognition unit 200 and the command interpreter 300 by managing and transmitting the situation according to the application of the robot ( context) management unit 400, and the robot application software 800 for operating the robot 500, the image output device 600, the voice output device 700 according to the command generated from the user's operation.

2 is a block diagram showing the configuration of an operation signal input unit according to the present invention.

The motion signal input unit 100 detects a user's motion and transmits the detected signal to the motion recognition unit 200.

The operation signal input unit 100 is a motion sensor device 101 for detecting a user's motion, a memory device 103 for temporarily storing the detected signal or storing the device and the identifier (ID) of the user, and appropriately processing the detected signal. And a control device 102 for providing the wireless transmitter / receiver 104 and a wireless transmitter / receiver 104 for transmitting the provided detection signal to the operation recognition unit 200.

In addition, the operation signal input unit 100 includes a power supply unit 106 for supplying power and a package and a wearing unit 105 formed to be worn by a user.

The motion sensor device 101 detects a user's motion and outputs a signal. The motion sensor device 101 may use a plurality of sensors capable of detecting a user's motion, and preferably a three-axis inertial sensor (acceleration sensor) is used.

The control device 102 collects the signal output from the motion sensor device 101 and processes the signal appropriately, and transmits the signal to the motion recognition unit 200 through the wireless transmission / reception unit 104.

The memory device 103 is embedded to temporarily store a signal processed by the control device 102 or to store information about the device and the user's identifier (ID) so that the information required by the control device 102 can be used.

The device and the user identifier (ID) stored in the memory device 103 may be used to determine the location of the user wearing the operation signal input unit 100 or to determine basic information about the current device or the user. Is used via 102.

The radio transmitter / receiver 104 transmits the signal processed by the control device 102 to the motion recognition unit 200. In addition, the radio transmitter / receiver 104 also functions to receive a signal including a request for information about the device and the user and to transmit information related thereto if necessary.

The package and the wearing portion 105 are formed in a form that can be worn by a user, and the shape and size of the wearing portion 105 are determined by the selection of the detail parts. The wearable form may be worn on various places of the body, and a typical wearable form such as a ring may be used.

3 is a block diagram showing the configuration of an operation recognition unit, an instruction analysis unit and a situation management unit according to the present invention.

The motion recognition unit 200 may include a wireless transmitter / receiver 210 that receives a signal transmitted from the motion signal input unit 100, a recognition device 220 that interprets the motion data of the received signal and recognizes the motion as a separate motion. The neural network 220 is used in the process of recognizing the signal as an individual operation, and the neural network required according to the control command of the situation management unit 400 among the various learned neural networks 330 and various learned neural networks 330. It consists of a neural network selector 240 for selecting (230).

The neural network 230 exists in various ways according to a user and a robot application, and each neural network 230 may be configured by a learning algorithm. For example, when there is a motion that moves from top to bottom, the direction recognition neural network recognizes the motion signal in the corresponding direction, but the number recognition neural network recognizes the number 1.

The neural network selector 240 selects the neural network 230 necessary according to the control command of the situation management unit 400. The neural network selector 240 selects and loads a neural network prepared in advance according to various situations such as robots and robot applications registered in the context management unit 400.

In the method of recognizing the signal received by the motion recognition unit 200 as an individual motion, the recognition device 220 may be configured using another soft computing technique (artificial intelligence technique) in addition to the method using a neural network. However, when recognizing various movements by inputting simple movements once, it is desirable to use a learned neural network. Since the motion recognition unit 200 using the neural network is composed of simple arithmetic operations of arithmetic level of the motion signal, it is simpler and faster than the image and voice recognition.

The command interpreter 300 operates or outputs the robot application software 800 through the robot 500, the image output device 600, and the voice output device 700 by using the recognition result of the motion recognition unit 200. Pass a command containing the information. The command interpreter 300 includes an operation including data information about a command corresponding to a situation corresponding to an individual operation, an operation under the control of the command analysis table 320 and the situation management unit 400, and a command analysis table 320. Based on the selected analysis table selection part 330 and the operation manager 400 selected by the operation manager, the command analysis table 320 generates a command suitable for the current situation and application, and finally the robot 500 or the robot application software ( It is composed of a command interpretation device 310 to deliver.

However, the operation of the command interpretation unit 300, the command analysis table 320 and the analysis table selection 330 is controlled by the operation registered in the situation management unit 400, the command analysis table 320 and the management unit 410 Analysis table selection 330 may be made.

The context management unit 400 controls the motion recognition unit 200 and the command interpreter 300 according to the currently available or selected application.

The context management unit 400 is largely divided into a management unit 410, a user situation unit 420, an application situation unit 430, an environment situation unit 440, and a robot situation. It is classified as part 450 to collect and manage the situation.

The user status unit 420 records a user ID, application preferences, and neural network features.

The application situation unit 430 and the robot situation unit 450 record the control commands and states that are disclosed from the outside.

In order to interpret a command transmitted to the robot 500 and the robot application software 800, a related operation and a command interpretation table may be registered in the application situation unit 430.

The situation management unit 400 selects an appropriate analysis table from the registered analysis table when the application situation is registered, and accordingly, the command interpretation device 310 generates a command.

The format of the command can vary depending on how the system is implemented. The message format is preferred. For example, a "forward push" action is interpreted as a push of a button output on the screen output to generate a "press" message and pass it to the robot application software 800. Alternatively, the "shake repeatedly up and down" operation is an operation for stopping the movement of the robot to generate a "stop" message and deliver it to the robot application software (800).

The environment situation unit 440 records detailed information of video and audio output devices that can be used by robots and applications.

Specific situation information is managed to be added as needed. In order to add situational information as necessary, it is desirable to use an XML method having a format and a content separated.

The robot application software 800 controls the operation of the robot 500 by using the command generated by the command interpreter 300 or outputs related information to the image output device 600 and the voice output device 700.

The robot 500 and the robot application software 800 perform a command based on the motion generated from the user's motion. The robot 500 may have a conventional interface device such as an image and voice recognition based interface. The robot application software 800 may also use the existing interface and the motion-based interface of the present invention.

The robot 500 and the robot application software 800 preferably use the image output apparatus 700 and the audio output apparatus 700 to provide the user with information necessary for interaction. Since it is not possible to memorize all of the currently available operations, it is desirable to inform this through the image output apparatus 600 and the audio output apparatus 700. In addition, it is preferable to check the contents of the input operation and allow the user to check the misrecognition result through the video and audio output devices 600 and 700.

4 is a diagram showing the configuration of the robot application software according to the present invention.

The robot application software 800 receives the command generated by the command interpreter 300 and analyzes the information included in the command by the application logic unit 810 to control the robot 820 and control information 820 and image output. The robot 500, the image output device 600, and the audio output device may be classified into the image information 830 output through the device 600 and the audio information 840 output through the audio output device 700. Relevant information is provided.

The robot 500 performs an operation by the control information 820. The robot 500 may receive control information through the robot application software 800, but may directly receive a command from the command interpreter 300 and perform a direct operation using the control information included therein.

The image output apparatus 600 and the audio output apparatus 700 output the image information 830 and the audio information 840 generated from the command, and the user checks the output contents and checks the result of the input operation. You can check.

5 is a flowchart schematically showing an operation-based interface method according to the present invention.

The motion-based interface method between a service robot and a person detects a user's motion as a signal through a motion sensor, receives an operation signal input step S100 for transmitting the detected signal, and receives a transmitted signal and interprets the received signal. Recognition step (S200) for recognizing the individual motion according to the situation and the command interpretation step (S300) for interpreting the recognized individual motion as a command suitable for the situation and application and the robot and the robot according to the information contained in the interpreted command The robot and the robot application operation step (S400) for operating the application is characterized in that it is made.

Individual motion recognition step (S200) and command interpretation step (S300) is controlled by the situation management unit for managing and transmitting the situation according to the robot application.

Individual motion recognition step (S200) recognizes the signal received from the motion signal input step (S100) as a separate motion through the neural network selected by the context manager based on a plurality of neural networks learned for the application and the user.

Command interpreting step (S300) is interpreted according to the result of the recognition by the individual motion recognition step (S200), the operation provided from the situation manager, command analysis table, and transmits the motion as a command suitable for the application of the robot apparatus.

Robot and robot application operation step (S400) provides the user with information necessary for interaction by the image output device and the audio output device.

The operation signal input step S100 is performed by a device mounted by a user.

A simple application scenario for a method of interfacing a robot and a person based on the motion by the motion-based interface system and method between the service robot and the person of the present invention will be described. The scenario is that a user sitting on a living room sofa calls the robot somewhere in the house while wearing the motion signal input unit 100.

The user performs a predefined operation with a hand wearing the operation signal input unit 100 (eg, a ring). For example, shake three times quickly. The operation signal generated at this time is sensed by the motion sensor device 101 and is transmitted to the motion recognition unit 200 through the wireless transmission / reception unit 104.

The situation management unit 400 loads the neural network 230 for the waiting situation prepared and learned in advance in the operation recognition unit 200 because the robot 500 is currently in the standby situation. The motion recognition unit 200 recognizes that the shake is recognized three times quickly through the neural network for the waiting situation, and transmits the result to the command interpreter 300.

The situation management unit 400 loads the operation corresponding to the waiting situation, the command analysis table 320 to the command analysis unit 300, and the command analysis unit 300 uses the operation, the command analysis table 320 in the waiting situation. The three quick shakes "action translates into a" call robot "command and generates a command called" robot call ".

The generated command is passed to the robot with the argument "who is where" provided by the robotic system or other application, and the robot approaches the calling user. At this time, by using the device and the user identification (ID) stored in the memory device 103 of the operation signal input unit 100 mounted on the user to determine the location of the user to approach the user.

If the image output apparatus 600 is being used, the current input operation and the result thereof may be shown to the user through the image output apparatus 600. For example, the living room TV of the user may output image output information in cooperation with the robot application software 800.

Although the present invention has been described in more detail with reference to some embodiments, the present invention is not necessarily limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention.

As described above, the motion-based interface system and method between a service robot and a person according to the present invention recognizes the user's motion and outputs a command suitable for a situation, thereby operating a robot, thereby eliminating a distance restriction between the input device and the user. The user can directly mount the operation signal input unit to detect the operation, thereby providing an effect that the input range is not limited.

In addition, the motion-based interface system and method between the service robot and the person according to the present invention can recognize the individual motion by the learned neural network, so that it is possible to quickly process arithmetic operations and use the motion and command interpretation table according to the situation. By recognizing a variety of commands and also by the output device there is an effect that the user can directly check whether or not.

Claims (14)

An operation signal input unit for detecting a user's motion as a signal through a motion sensor and transmitting the detected signal; An operation recognition unit for receiving a signal transmitted from the operation signal input unit, interpreting the signal, and recognizing it as an individual operation suitable for a situation; A command interpreter for interpreting the recognized individual motions into commands suitable for a situation and an application; It includes a robot that operates according to the information contained in the command transmitted through the command interpretation unit, The motion recognition unit and the command interpretation unit is a motion-based interface system between the service robot and the person, characterized in that controlled by the situation management unit for managing and transmitting the situation according to the robot application. delete The method of claim 1, The motion recognition unit between the service robot and the person, characterized in that the signal received from the motion signal input unit through the neural network selected by the situation manager based on a plurality of neural networks learned appropriately for the application and the user as an individual operation Motion-based interface system. The method of claim 3, wherein The command interpreter is an operation between the service robot and a person, characterized in that the operation is interpreted as a command suitable for the application of the robot device according to the recognition result of the operation recognition unit and the operation, command analysis table provided from the situation manager. Based interface system. The method of claim 1, The motion signal input unit is a motion-based interface system between a service robot and a person, characterized in that the user can wear a form. The method of claim 5, And the motion signal input unit stores an identifier (ID) of the user and the motion signal input unit. The method of claim 1, The robot and the robot application is a motion-based interface system between the service robot and the person, characterized in that to provide the user with information necessary for interaction by the image output device and the audio output device. The method of claim 1, The robot and the robot application is a motion-based interface system between a service robot and a person, characterized in that it has a function to publicly publish its own situation information or a situation manager. Detecting a user's motion as a signal through a motion sensor and transmitting the detected signal; Receiving the transmitted signal, interpreting the signal, and recognizing it as an individual operation suitable for a situation; Interpreting the recognized individual operation into a command suitable for a situation and an application; Including the step of operating the robot and the robot application according to the information contained in the interpreted command, Recognizing the individual operation and the step of interpreting the command is a motion-based interface between the service robot and the person, characterized in that controlled by the situation management unit for managing and transmitting the situation according to the robot application. delete The method of claim 9, Recognizing the individual operation is characterized in that the signal received from the operation signal input step through the neural network selected by the situation manager based on a plurality of neural networks learned for the application and the user as a separate operation characterized in that Behavior-based interface method between service robots and people. The method according to claim 11, The interpreting of the command may include interpreting and transmitting the motion as a command suitable for the application of the robot apparatus according to the recognition result by the step of recognizing the individual motion and the motion and command analysis table provided from the situation manager. An action based interface method between a service robot and a person. The method of claim 9, The operation of the robot and the robot application is a motion-based interface system between a service robot and a person, characterized in that to provide the user with information necessary for interaction by the image output device and the audio output device. The method of claim 9, The step of detecting the user's motion as a signal through the motion sensor is a motion-based interface method between the service robot and the person, characterized in that made by the user-mounted device.

Images