WO2016024849A1

WO2016024849A1 - Robot guardian module, and robot safety ensuring device and method using same

Info

Publication number: WO2016024849A1
Application number: PCT/KR2015/008533
Authority: WO
Inventors: 이동익; 양인석; 이동준
Original assignee: 경북대학교 산학협력단
Priority date: 2014-08-14
Filing date: 2015-08-13
Publication date: 2016-02-18
Also published as: KR20160020772A; KR101643812B1

Abstract

The present invention provides a robot guardian module, and a robot safety ensuring device and method using same and, more particularly, provides a robot guardian module, and a robot safety ensuring device and method using same, which: detect user state information including heartbeat information of the user, sound information, and image information, and robot state information indicating whether normal operation is underway; and analyze the detected user state information and the robot state information, deduce whether there is any danger to the user by the robot and whether there is danger to the robot, and determine a danger level and control the robot according to the determined danger level in order to nullify a dangerous situation.

Description

Robot guardian module and robot safety guarantee device and method using same

The present invention relates to a robot guardian module and a robot safety guarantee device and method using the same, and more particularly, to determine the risk of the robot to the user based on the multiple information detected through the sensor module and to determine the robot according to the determined risk The present invention relates to a robot guardian module which guarantees the safety of a user and the safety of a robot by controlling the same, and a robot safety guarantee device and a method using the same.

Robots, such as humanoids, have a human-friendly appearance, and the age of autonomous robots is being demonstrated by demonstrating smooth joint movements such as stair up and down, side steps, and curved walking, as well as dynamic bipedal walking (walking with two feet). As such, the robot industry has been developed with the ultimate goal of providing various services on behalf of or in cooperation with human beings.

Currently, Cog, developed by the MIT Institute of Artificial Intelligence, and Asimo, developed by Honda, Japan, have the most human-like functions. Robot technology, which looks, moves, and intelligence is no different from humans, is possible in the distant future.

However, due to the steady development of the robot industry, the demand for service robots for various purposes such as guide robots and educational robots is continuously increasing. In the same space, humans and service robots coexist in the same space. Therefore, human injuries caused by service robots that provide services to humans in a space where humans and service robots coexist together prevent human safety and It is required to present the technology to ensure the safety of the robot.

The present invention has been proposed to meet the demands of the development of robot technology as described above, an object of the present invention is to determine the risk of the robot to the user based on the multiple information detected through the sensor module and to determine the risk Accordingly, the present invention provides a robot guardian module and a robot safety guarantee device and method using the same, by removing a dangerous situation caused by the robot by controlling the robot, thereby ensuring the safety of the user and the safety of the robot.

In order to achieve the object of the present invention as described above, the robot guardian module according to the present invention, by analyzing the user's heartbeat information, voice information and image information detected from the heart rate sensor, voice sensor and image sensor respectively to the user's risk feeling A state recognition unit for recognizing related heartbeat change, screaming and facial expression information, and analyzing the state information of the robot detected from the robot state sensor to recognize the operation state of the robot; A risk inference unit that infers whether the robot has a danger to the user and whether the robot is in danger by analyzing the user's heartbeat change, screaming and facial expression information recognized from the state recognition unit, and the operation state of the robot; And a risk determination unit for classifying risks inferred from the risk inference unit by risk level to determine a risk corresponding to the inferred risk, and transmitting risk information to the robot control module to control the robot to remove the risk situation. It is done by

The risk determining unit determines the risk level by analyzing the magnitude of the heartbeat change of the user, the magnitude and frequency of occurrence of the user's scream, the degree of risk shown in the expression of the user, and the operation state of the robot.

In order to achieve the object of the present invention as described above, the robot safety guarantee device using a robot guardian module according to the present invention, including a heart rate sensor, a voice sensor, an image sensor and a robot state sensor, the user's heart rate information, voice information And a sensor module that detects the user's status information including the image information and the robot's status information, and infers whether the robot is dangerous to the user and whether the robot is dangerous based on the information detected from the sensor module. And a robot guardian module for transmitting the risk information, and a robot control module for controlling the robot according to the risk information transmitted from the robot guardian module.

In order to achieve the object of the present invention as described above, the robot safety determination method using the robot guardian module according to the present invention, by analyzing the user's heartbeat information detected from the heartbeat sensor to recognize the user's heartbeat change related to the risk feeling, voice Analyzes the user's voice information detected from the sensor to recognize the screams related to the risk emotion, analyzes the user image information detected from the image sensor to recognize the user's expression related to the risk emotion, and the state of the robot detected from the robot state sensor A first step of analyzing the information and recognizing the operation state of the robot; A second step of inferring whether the robot is dangerous to the user and whether the robot is dangerous by analyzing the recognized heartbeat change, screaming, facial expression, and operation state of the robot; And classifying the inferred risk by risk level to determine a risk corresponding to the inferred risk, and transmitting the risk information to the robot control module to control the robot to remove the risk situation.

In order to achieve the object of the present invention as described above, the robot safety method using the robot guardian module according to the present invention, the user's state information and normal operation of the sensor module including the user's heartbeat information, voice information and image information A first step of detecting state information indicating whether the robot is present; A second step of the robot guardian module analyzing the detected state information of the user and the state information of the robot to determine a risk by inferring whether the robot is dangerous to the user and whether the robot is dangerous; And a third step in which the robot control module controls the robot according to the determined risk to remove the dangerous situation.

The present invention is to determine the risk of the robot to the user based on the multiple information detected through the sensor module and to control the robot according to the determined risk by removing the risk situation caused by the robot to ensure the safety of the user and the safety of the robot There is.

The present invention has an effect that can detect the state information of the robot to diagnose and cope with the operation state of the robot.

1 is a block diagram showing a robot guardian module and a robot safety guarantee device using the same according to the present invention,

2 is a flow chart showing a robot safety method using a robot guardian module according to the present invention,

3 is a diagram illustrating an example of a method of inferring a risk based on voice information of a user,

4 is an exemplary diagram showing facial expression information pre-stored in a database in order to infer a risk based on facial information of a user.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing a robot guardian module and a robot safety guarantee device using the same according to the present invention. As shown in Figure 1, the robot safety guarantee device according to the present invention, the sensor module 100 for detecting the user's status information and the robot's status information, including the user's heartbeat information, voice information and image information, and Based on the information detected from the sensor module 100, the robot guardian module 200 for determining the risk and transmitting risk information by inferring whether the robot is dangerous to the user and whether the robot is dangerous, and the robot guardian module ( It is configured to include a robot control module 300 for controlling the robot according to the risk information transmitted from the 200.

The sensor module 100 detects image information including a heartbeat sensor 110 that detects user's heartbeat information, a voice sensor 120 that detects voice information of the user, and facial expression information of the user. It comprises an image sensor 130, and the robot state sensor 140 for detecting the robot state information indicating whether or not the normal operation of the robot.

The sensor for detecting the state information of the user such as the voice sensor 120 and the image sensor 130 may be installed outside the robot body. The robot state sensor 140 may be installed inside the robot body. In the case of the robot state sensor 140, the robot may receive a signal indicating whether the robot is in a normal state by utilizing a central processing unit (CPU) of the robot.

The robot guardian module 200 recognizes the emotional state felt by the user through the analysis of the state information of the user sensed by the sensor module 100, and transmits information on the state of the user, from the sensor module 100. State recognition unit 210 for transmitting the state information of the robot by recognizing the operation state of the robot through analysis of the detected state information of the robot, and whether the user state information and the robot transmitted from the state recognition unit 210 The risk inference unit 220 and the risk inference to infer whether or not the risk of the robot to the user by comprehensively analyzing the state or not information of the state and determine the risk corresponding to the risk inferred from the risk inference unit 220 It is configured to include a risk determination unit 230 for transmitting the risk information.

The state recognizer 210 analyzes the heartbeat of the user sensed by the heartbeat sensor 110 and recognizes the heartbeat change of the user with a sudden change over a predetermined size. Analyzing the voice of the user sensed from the 120 to recognize whether or not the voice recognition unit 212, and the expression of the facial expression to recognize the expression corresponding to the risk by analyzing the user's image detected from the image sensor 130 It includes a branch 213, and a robot state recognition unit 214 for analyzing the state information of the robot detected from the robot state sensor 140 to recognize whether the robot is in a normal state.

The risk inference unit 220 and the risk determination unit 230 may analyze the user's heartbeat information, voice information, and image information to determine whether the user is at risk and the degree of danger. Alternatively, the risk inference unit 220 and the risk determination unit 230 may use only one piece of information of a user's heartbeat information, voice information, and image information, or a combination of the two information, depending on the service environment of the robot. And risk.

In order to cope with a sudden dangerous situation, a safety mode switching switch 400 for converting the robot into a safe mode by an external manipulation is connected to the robot guardian module 200 and the robot control module 300.

The robot guardian module 200, the level of risk that the robot affects the user, the size of the heart rate change of the user related to the risk feeling, the size and frequency of the user's screaming, the degree of risk shown in the user's expression, and the robot The operating state is comprehensively analyzed and classified into first to third levels.

In addition, when the inferred danger is the first level, the robot guardian module 200 determines that the robot is in a safe state between the robot and the user, and transmits risk information to maintain the current operation of the robot to the robot control module 300. If the inferred risk is the second level, it is determined that the risk-related degree that the robot has on the user is a warning level, and the risk information for slowing or stopping the current speed of the robot is transmitted to the robot control module 300. When the inferred risk is the third level, it is determined that the risk-related degree that the robot has on the user is a risk level, and the robot control module 300 provides risk information for shutting off the robot power supply and switching the robot to a locked state. To send).

2 is a flow chart showing a robot safety method using a robot guardian module according to the present invention. As shown in Figure 2, the robot safety method using a robot guardian module according to the present invention, the user's state information including heartbeat information, voice information and image information of the user and the state information of the robot indicating whether or not normal operation The first step (S10) for detecting the second, and the second step (S12) to determine the risk by inferring whether the robot is dangerous to the user by analyzing the detected state information of the user and the state information of the robot (S12) S14 and S16, and a third step S18 of controlling the robot according to the determined risk level to eliminate the dangerous situation.

The second step (S12, S14, S16), through the analysis of the detected state information of the user to recognize the emotional state felt by the user to transmit the user's state information and to analyze the state information of the detected robot Recognizing the normal operation state of the robot through the step (S12) for transmitting the information whether the state of the robot, and whether or not the risk of the robot to the user by comprehensively analyzing the transmitted state information of the user and information of the state of the robot And inferring whether the robot is dangerous (S14), and classifying and determining the risk according to the inferred risk (S16).

Now, the operation of the robot guardian module and the robot safety guarantee device using the same according to the present invention will be described in detail.

The sensor module 100 detects information indicating that the user feels a danger and also detects whether the robot is operating normally. 1 illustrates a heartbeat sensor 110, a voice sensor 120, and an image sensor 130, but is not limited thereto.

The heartbeat sensor 110 of the sensor module 100 detects the user's heartbeat information, the voice sensor 120 detects the user's voice information, and the image sensor 130 detects the image information including the user's face information. , Respectively, the robot transmits to the heartbeat change recognition unit 211, the voice recognition unit 212, and the expression recognition unit 213 of the state recognition unit 210 in the Diane module 200. In addition, the robot state sensor 140 of the sensor module 100 detects the state information of the robot indicating whether the robot is in normal operation, and sends the robot to the robot state detection unit 214 of the state of the robot module 200. It transmits (S10).

The state recognition unit 210 of the robot guardian module 200 analyzes the state information of each user sensed by the sensor module 100 to recognize an emotional state felt by the user, and analyzes the state information of the detected robot. Recognizes the operating state indicating whether the normal operation of (S12).

That is, the heartbeat change recognition unit 211 of the state recognizer 210 receives a user's heartbeat detected from the heartbeat sensor 110 and analyzes whether there is a sudden change of a predetermined size or more that occurs when a feeling of danger occurs. When there is a sudden change (increase or decrease) over the size, the user's heartbeat change is recognized and the user's state recognition information is generated and transmitted.

The voice recognition unit 212 analyzes the user's voice detected by the voice sensor 120 to recognize whether the voice is screaming. As shown in FIG. 3, if a sound of a predetermined size band (for example, the 650 MHz to 2000 MHz band) lasts for a predetermined time (for example, 0.5 seconds), the sound is recognized as a scream generated when the user feels a dangerous feeling. . Alternatively, if the currently transmitted voice is higher than the previously transmitted voice, for example, 500 MHz, it may be recognized as a scream generated when a dangerous feeling is felt.

The facial expression recognition unit 213 analyzes the image information of the user detected by the image sensor 130 and recognizes the facial expression corresponding to the danger. That is, as shown in FIG. 4, the robot guardian module 200 stores various expressions of the user in a database and stores the combined expressions by combining the stored expressions. The robot guardian module 200 determines the emotion of the user by detecting the most similar expression by comparing the transmitted facial information of the user with the expression previously stored in the database. For example, among the facial expressions stored in the database of FIG. 4, the emotional expression of fear because the user feels a danger is determined by action unit 1 + 2 (Action Unit 1; AU 1) and action unit 2 (AU2). AU 1 + 2) can be said. If the facial expression information of the transmitted user is most similar to the pre-stored action unit 1 + 2 facial expression, the user may determine that he is feeling fearful emotion.

In this way, the perceived user's state recognition information, that is, cognitive information such as a user's heartbeat change, screaming, facial expression corresponding to the degree of danger is transmitted to the risk inference unit 220.

The robot state recognition unit 214 recognizes the operation state of the robot by determining whether the robot is operating normally or a failure state by checking the state information of the robot detected from the robot state sensor 140. The recognized operating state of the robot is transmitted to the risk inference unit 220.

The risk inference unit 220 infers a risk that the robot has on the user by analyzing the transmitted state information of the user and the state information of the robot (S14).

That is, the risk reasoning unit 220 is the size of the heart rate change of the user transmitted from the heart rate change unit 211, the frequency of occurrence of the user's scream transmitted from the voice recognition unit 212, duration of the scream, facial expression recognition unit By integrating analysis of the facial expression corresponding to the user's risk sent from 213 and the operation state of the robot transmitted from the robot state recognition unit 214, it is inferred that the robot is causing the user a danger.

The risk determination unit 230 classifies the degree of risk inferred by the risk inference unit 220 to determine a risk corresponding to the risk (S16). Risk classification can be classified into a number of stages depending on various factors such as the type of service provided by the robot or the user's health status. For example, if the risk is classified into three levels, it can be classified into blue (safe), yellow (warning), and red (danger).

The robot control module 300 controls the robot according to the risk transmitted from the risk determination unit 230 to remove the dangerous situation (S18).

For example, a method of inferring a risk and determining a risk by analyzing the image information of the user by the robot control module 300 will be described in more detail.

Upon receiving the user's image information from the image sensor, the robot guardian module 200 obtains the user's facial expression information related to the emotion from the user's image information, and obtains the user's facial expression information and the emotion-related facial expression information previously stored in the database as shown in FIG. 4. Compare with The most similar facial expression information is detected, and the risk-related emotion of the user is determined based on the emotional information corresponding to the detected facial expression information.

Each of the pre-stored facial expression information has a risk level set according to the degree of danger. For example, when the risk level is classified into three levels of "safety", "warning", and "danger", action unit 1 (AU1) is classified as "warning" level as the expression of the inner part of the eyebrows being raised, Action unit 1 + 2 (AU1 + 2) is an expression in which both the inner and outer portions of the eyebrows are raised and may be classified as a “dangerous” level.

If the detected facial expression information is the expression of the action unit AU1 + 2, the robot guardian module 200 determines that the user feels the danger at the "risk" level and determines the risk as "risk".

For example, when the risk is classified into three levels of safety, warning, and danger, the robot guardian module 200 displays the risk information corresponding to the "risk" when the inferred risk is the "risk" of the highest level. Transmission to the robot control module 300, the robot control module 300 quickly switches the robot to the safe mode. The safety mode means that the robot is in a locked state at the same time that power is supplied to the robot. In the situation where the service robot is on the slope, the robot guard module 200 determines that the risk feeling felt by the user is the highest, and the robot control module 300 switches the robot to the safe mode when the inferred risk level is the highest level. Therefore, the robot is prevented from sliding down the ramp along the ramp to prevent the possibility of secondary accidents at the bottom of the ramp.

When the risk level is a "warning" level, the robot control module 300 may control the robot in such a manner as to slow down or abruptly stop the robot.

In addition, when the risk level is a "safe" level, the robot control module 300 may determine that the user does not feel a danger to the robot providing the service, and maintain the operation without controlling the robot.

The operation of controlling the robot according to the risk level is not limited thereto, and may be variously implemented according to the type of service provided by the service robot or the state of the user.

The safe mode switching switch 400 is operated so that the robot can be switched to the safe mode by an external operation instead of the control of the robot guardian module 200 and the robot control module 300.

Safety mode switch for switching the robot to the safe mode according to the risk information transmitted from the robot guardian module 200 may be provided separately.

The present invention is to determine the risk of the robot to the user based on the multiple information detected through the sensor module and to control the robot according to the determined risk by removing the risk situation caused by the robot to ensure the safety of the user and the safety of the robot Provided is a guardian module and a robot safety guarantee device and method using the same.

Claims

Analyzing the user's heartbeat information, voice information and image information detected from the heart rate sensor, the voice sensor and the image sensor, respectively, recognizes the heartbeat change, screaming and facial expressions related to the user's risk feeling, and detects the robot A state recognition unit for recognizing state of operation of the robot by analyzing state information;

A risk inference unit that infers whether the robot has a danger to the user and whether the robot is in danger by analyzing the user's heartbeat change, screaming and facial expression information recognized from the state recognition unit, and the operation state of the robot; And

A risk determination unit for classifying risks inferred from the risk inference unit by risk level to determine a risk corresponding to the inferred risk, and transmitting risk information to the robot control module to control the robot to remove the risk situation. Robot Guardian Module.
The method of claim 1, wherein the risk determination unit,

Robot guardian module for determining the risk by analyzing the size of the heart rate change of the user, the size and frequency of occurrence of the user's screaming, the degree of risk shown in the expression of the user, and the operation state of the robot.
A sensor module including a heartbeat sensor, a voice sensor, an image sensor, and a robot state sensor, for detecting a user's state information including a user's heartbeat information, voice information, and image information and state information of a robot;

A robot guardian module for determining a risk by transmitting a risk information by inferring whether a robot has a danger to a user and a risk of the robot based on the information detected from the sensor module; And

Robot safety module using a robot guardian module, characterized in that it comprises a robot control module for controlling the robot according to the risk information transmitted from the guardian module.
According to claim 3, The robot guardian module,

Recognizing the emotional state felt by the user through the analysis of the user's state information detected from the sensor module and transmits the user's state information, and analyzes the robot's operation state through the analysis of the state information of the robot detected from the sensor module A state recognition unit for recognizing and transmitting state information of the robot;

A risk inference unit that infers whether the robot is dangerous to the user and whether the robot is in danger by comprehensively analyzing the user's state information and the state information of the robot transmitted from the state recognition unit; And

Robot safety module using a robot guardian module, characterized in that it comprises a risk determination unit for transmitting the risk information to determine the risk corresponding to the risk inferred from the risk inference unit.
The method of claim 4, wherein the state recognizer,

A heartbeat change recognition unit for analyzing a heartbeat of the user sensed by the heartbeat sensor and recognizing a change in the heartbeat of a user having a sudden change over a predetermined size;

A voice recognition unit for recognizing the scream by analyzing the voice of the user sensed by the voice sensor;

An expression recognizing unit that recognizes an expression corresponding to a risk by analyzing an image of the user sensed by the image sensor; And

Robot safety module using a robot guardian module, characterized in that it comprises a robot state recognition unit for analyzing the state information of the robot sensed from the robot state sensor to recognize whether the robot is in a normal state.
According to claim 3, The robot guardian module,

Analyzes the level of risk that the robot has on the user from the size of the user's heart rate change related to the risk emotion analyzed from the user's heartbeat information, the size and frequency of the user's scream analyzed from the user's voice information, and the user's video information. Robot safety module using a robot guardian module, characterized in that the classification of the first to the third level by analyzing the degree of risk and the operation state of the robot displayed on the expression of the user.
According to claim 3, The robot guardian module,

When the inferred danger is the first level, it is determined that the robot is safe between the user and the robot, and the risk information for maintaining the current operation of the robot is transmitted to the robot control module.

If the inferred risk is the second level, it is determined that the risk-related degree that the robot has on the user is a warning level, and the risk information for slowing or stopping the current speed of the robot is transmitted to the robot control module.

When the inferred risk is the third level, it is determined that the degree of risk related to the user is a risk level, and the risk information for shutting down the robot power supply and switching the robot to the locked state is transmitted to the robot control module. Robot safety device using a robot guardian module, characterized in that.
Analyzes user heartbeat information detected from heart rate sensor to recognize user's heartbeat change related to risk feeling, analyzes user voice information detected from voice sensor and recognizes screaming, analyzes user image information detected from image sensor A first step of recognizing a facial expression of a user and analyzing state information of the robot detected by the robot state sensor to recognize an operation state of the robot;

A second step of inferring whether the robot is dangerous to the user and whether the robot is dangerous by analyzing the recognized heartbeat change, screaming, facial expression, and operation state of the robot; And

The robot guardian includes a third step of classifying the inferred risk by risk level to determine a risk corresponding to the inferred risk, and transmitting risk information to the robot control module to control the robot to remove the risk situation. How to determine robot safety of modules.
The method of claim 8, wherein the third step,

Classifying the risks into first to third levels according to the magnitude of the heartbeat change of the user, the magnitude and frequency of screaming of the user, the degree of danger in the facial expression of the user, and whether the robot is normally operated;

If the inferred risk is the first level, determining that the robot and the user are in a safe state and transmitting risk information for maintaining the current operation of the robot to the robot control module;

If the inferred risk is the second level, determining that the risk-related degree that the robot has on the user is a warning level, and transmitting risk information for slowing or stopping the current speed of the robot to the robot control module; And

When the inferred risk is the third level, it is determined that the degree of risk related to the user is a risk level, and the risk information for shutting down the robot supply power and switching the robot to the locked state is transmitted to the robot control module. Robot safety determination method of the robot guardian module comprising the step of.
A first step of the sensor module detecting user state information including heart rate information, voice information, and image information of the user and robot state information indicating whether the robot is in normal operation;

A second step of the robot guardian module analyzing the detected state information of the user and the state information of the robot to determine a risk by inferring whether the robot is dangerous to the user and whether the robot is dangerous; And

And a third step of removing the dangerous situation by controlling the robot according to the determined risk level.
The method of claim 10, wherein the second step,

Analyzes detected user heartbeat information to recognize risk-related heartbeat changes, analyzes detected user voice information to recognize whether they are screaming, analyzes detected user image information to recognize facial expressions corresponding to risk, and detects robots A first sub-step of recognizing an operating state of the robot by analyzing state information of the robot;

A second sub-step of inferring whether the robot is dangerous to the user and whether the robot is in danger by comprehensively analyzing the recognized heartbeat change, scream, facial expression, and operation state of the robot; And

A third sub-step of comprehensively analyzing the risk of the inferred risk by analyzing the magnitude of the user's heartbeat change, the magnitude and frequency of the user's screaming, the degree of danger in the user's expression, and whether the robot is in normal operation; Robot safety method using a robot guardian module, characterized in that comprises a.
The method of claim 11, wherein the third sub-step,

Transmitting risk information to the robot control module to maintain a current operation of the robot when the inferred risk is a safe level;

If the inferred danger is a warning level, transmitting risk information to the robot control module to slow down or abruptly stop the current speed of the robot; And

When the inferred risk is a risk level, the robot guard module further comprises the step of transmitting the robot information to the robot control module and the risk information for switching off the robot power supply to a locked state. How to ensure robot safety.
The method of claim 10, wherein the second step,

According to the service environment provided by the robot to the user, by analyzing any one of the user heart rate information, the user voice information and the user image information, further comprising the step of inferring the risk to the user to determine the risk Robot safety method using a robot guardian module.