KR100813668B1 - Emotional expression equipment and method in android robot - Google Patents

Abstract

An emotional expression method for an android robot is provided to increase requests for new technologies related to the robot and communicate with human friendly by expressing emotion by way of vision, sound, and text. An emotional expression method for an android robot comprises a vision processing step(S1) of extracting the emotional information of human by using an image signal photographed by a camera; an intelligence processing step(S2) of editing the emotional information to adjust to a motor applied to each shaft by using a text signal input through the web, the messenger, or the keyboard, and a sound signal input through a micro phone, and the motion information extracted by the image signal photographed from the camera; and a motion controlling step(S3) of controlling the driving of each motor of the robot by corresponding to the signals output from the vision processing step and the intelligence processing step.

Description

Emotional expression equipment and method in android robot}

1 is a schematic block diagram of a emotion expression apparatus to which the present invention is applied.

Figure 2 is a detailed block diagram of an apparatus to which the present invention is applied.

3 is a main flowchart for explaining the method of the present invention.

4 is a detailed flowchart of a motion recognition step by color recognition in the vision processing step of FIG. 3.

FIG. 5 is a detailed flowchart of a face and object tracking step of the vision processing step of FIG. 3;

6 is a detailed flowchart of one embodiment according to an intelligent processing step of the method of the present invention.

7 is a detailed flowchart of another embodiment according to an intelligent processing step of the method of the present invention.

8 is a detailed flowchart of a motion control step of the method of the present invention.

9 is a block diagram showing a personality model of the robot applied in the present invention.

10 is a two-dimensional emotional space diagram showing the emotion model of the robot applied in the present invention.

Figure 11 is an illustration of four emotions shown on the face of the robot applied in the present invention.

Figure 12 is an illustration of the gestures performed by the robot applied in the present invention.

* Description of symbols on the main parts of the drawings *

1: Vision Division 2: Intelligence Division

3: motion control unit 4: camera

5: microphone 6: web or messenger or keyboard

7: speaker 8: face motor

9: Arm, Torso, Hand Motor 10: Eye, Neck Motor

11: face recognition unit 12: object recognition unit

13 color recognition unit 14 motion recognition unit

15: face tracking unit 16: object tracking unit

21: voice recognition unit 22: voice signal direction detection unit

23: dialogue engine unit 24: personality model unit

25: emotion model unit 26: scenario unit

27: sound signal synthesis unit 28: homeostasis generator

29,37 Database 31: Emotional Behavior

32: facial expression unit 33: gesture unit

34: Physiological Behaviors 35: Lip Sync

36: motor arbiter

The present invention relates to a method of expressing emotions of an android robot, and more particularly, to express an emotion of an android robot having a skin and a movement similar to a human with facial expressions and gestures to communicate more naturally and intimately with a person. It is invented to be.

In general, a robot is a machine that operates like human hands and feet, and is also called an artificial human being.

It puts a mechanism inside a doll that was originally a human figure, and refers to an automatic doll that works like the original person with hands and feet and other parts. The term robot means "working" in the Czech language. In 1920, Czechoslovak writer K. Chapec became interested in releasing the play "Rossum's Universal Robots".

Chapek symbolically expressed a very pessimistic view of the development of technology and its relationship with human society. All mental and physical labor can do the same as humans, but they do not have human emotions or spirits, and when worn out, they have introduced artificial humans that can be exchanged for new ones. Painted.

And these robots unfolded a story of the development of intelligence and rebellion through labor and the destruction of human beings, which attracted the attention as a typical view of the impact of modern automation on society.

Attempts to create artificial humans have been made since ancient times, and in the Greco-Roman period and BC, they were made as a tool of religious consciousness, and in the Middle Ages they made automatic dolls that opened the doors of a building or played musical instruments. It was also used to amaze people or to associate with Deity to show off the authority of the ruler.

On the one hand, it was also a joke of machine engineers, but the production of these automatic dolls did not directly contribute to the progress of science and technology, and even in the 20th century, the production of automatic dolls was attempted in various ways, and science and technology were advanced. It could have been made more sophisticated than before, either for merchandise display or for attracting audiences at the fair and not for practical use.

Notable among these robots were Televox by Knight Wensley of the Westinghouse Electric Company in 1927 and Eric by Richards of England, all of whom applied mechanical or electrical technology. It was an elaborate response.

In modern times, a combination of electronic tubes, photoelectric tubes, telephones, tape recorders, etc. can be used to answer people's questions, artfully move hands and feet, or move freely by remote control by wireless. This was made.

But this human-like appearance, however highly sophisticated in its movement, was limited in movement and not very practical.

Thus, a machine that does not resemble a human figure in practical terms but acts like a human's movement is called a robot, and the tendency to apply a robot in many aspects has become stronger.

Originally, the machine was placed in the industrial structure as a means of production, and in response to the multipolarization of the social environment, the robot appeared as a machine adapting to various abnormal environments, and also with the advance of automatic control technology or remote control technology. Work in dangerous environments such as space, the sea floor, high or low temperatures, or very monotonous work is all unsuitable for humans. Currently, robots are expanding their applications in such abnormal environments.

Robot applications are generally classified into industrial, medical, space, and submarine applications.

For example, in the machining industry, such as automobile production, many industrial robots that repeat the same movement for hours are already in operation if only one task is taught by the human arm. The meaning, that is, the prosthetic hand which drives the motor by taking out the command sent from the motor nerve at the near-potential, has been put into practical use. Remotely controlled robots such as Lunokhod have been developed.

Recently, robots are divided into pilot, autonomous, and autonomous, which is a method of manipulating a machine with a function corresponding to a human hand or foot from a distance, also called a teleoperator system. Manipulators (magic hands) used in nuclear reactors are controlled robots.

In relation to space development, the United States is experimenting with installing manipulators on space shuttles (Columbia, etc.) for technical services in space. Plans to remotely control the system. These technologies will be applied to marine development, and they will also be applied to hazardous work such as extinguishing fire and removing unexploded coal. In the medical field, the robot is expected to be used in all situations, including diagnosis, treatment, surgery or rehabilitation.

The automatic type is an industrial robot currently widely used in the industry. If you teach the order in advance, it's the type that you remember because you remember it. This type of robot is also a transitional to the next autonomous type. An autonomous type is a robot itself that recognizes its current state and environmental state and acts autonomously according to a command. This type is the original robot.

On the other hand, humans can know the condition of their hands and feet by closing their eyes because skin sensations, joint sensations, and senses from muscles and tendons return to the central body, The situation can be seen through eyes or ears, so autonomous robots need a sensory device comparable to this.

Some recent robots have humanoid limbs, television camera eyes, artificial ears, mouths, tactile sensations, and joint sensations. If you tell them to look for it, they look around the room, find it, answer it in words, walk on both feet and pick it up with their hands.

The fact that a machine judges and acts itself is that the machine has intelligence. Such a robot is called an intelligent robot, and a robot having only a visual function is called a visual robot.

For industrial use, development robots and inspection robots are being developed, and when assembling an object, it is highly advanced to recognize and select two objects from among a large number of objects with uneven dimensions or shapes, and to assemble them into required shapes. Belongs to.

Even with the accumulation of such intelligent robot research, the ideal robot is not yet realized, the function and structure of the human brain are not yet fully physiologically explained, and the computer used is different from the human brain in principle.

In addition, as the industrial robot developed in recent years has evolved from a rapid development of the industrial robot to a home robot capable of living like a human, there is a need for not only to recognize signals provided by humans but also to provide signals similar to those of humans. .

In cognitive science, brain science, neuroscience, etc., research has been conducted first, and gradually applied to robot fields. In the past, signals provided by humans using robots having similar joint structures with different virtual spaces or humans, but with different appearances It could be recognized, but there was a difficulty in providing it.

However, in the United States, K-bot at the University of Texas, David Hanson at NASA, and Kismet at MIT express their emotions with faces similar to humans.In Japan, the Actroid series developed by Osaka University and Kokorosa Co., Ltd. It is expressing.

As such, robots with appearances similar to those of humans (called "Android Robots") not only provide a feeling of familiarity but also facilitate communication with humans through expressions of emotions. The situation is required to be able to express more precisely through and gestures.

The present invention has been made to solve the above-mentioned conventional problems and requirements, Android robot having a skin and movement similar to humans can express the emotional state with a skin and joint structure similar to humans, vision and voice and Emotional expressions of Android robots that can express emotions to other parties through texts to communicate with humans more closely, thereby increasing the demand for new technologies related to robots and contributing to the robot market. The purpose is to provide a method.

Emotion expression apparatus to which the method of the present invention is applied to achieve the above object comprises: a vision unit for extracting human emotional information through facial expression, color and motion of the human being imaged through the camera; Emotional information is extracted from the personality, emotion model, and text detected through the voice of the person input through the microphone and the data provided by the vision unit, and data including facial expressions and gestures edited for the motor applied to each axis is stored. An intelligent unit for selecting and managing emotions to be expressed by the robot through the base; And a motion control unit transmitted from the vision unit and the intelligence unit to execute emotional actions using data, and to arbitrate a motor of each axis to perform a desired operation in the scenario unit.

The vision unit may include a face recognition unit, a color recognition unit, and an object recognition unit for recognizing a face, a color, and an object in an image captured by a camera mounted on two eyes of a robot; A motion recognition unit which receives the output signal of the color recognition unit and derives a positive / negative emotion signal by analyzing whether the skin is colored or the movement of the head and hands; And a face tracker and an object tracker for extracting a conversation distance by tracking the face and the object and tracking the motion range of the eyes, the neck, and the waist, respectively, by receiving the output signals of the face recognition unit and the object recognition unit, respectively. do.

The intelligent unit may further include: a voice recognition unit and a voice signal direction detection unit for detecting whether a voice signal is input through a microphone and a direction of the input voice signal; A dialogue engine unit for recognizing a text signal input through a web, a messenger or a keyboard, and a voice signal through a microphone input through the voice recognition unit, and extracting positive / negative emotion signals; A personality model unit which extracts two personalities of vigor / timidity by changing relative values by using emotion information extracted from the motion recognition unit of the vision unit and the dialogue engine unit; An emotion model unit which extracts an emotion to which personality is applied by applying a relative value provided by the personality model unit to an emotion model axis; A scenario unit configured to output a facial expression and gesture control signal of the robot to the motion control unit and output a speech synthesis signal by using the signals input from each unit and tags defined in a scenario previously stored in a database; An audio signal synthesizing unit for outputting a speech synthesizing signal output from the scenario unit through a speaker; And a homeostasis generator for generating physiological behavioral data using pre-input data.

The motion controller may further include: an emotional behavior unit configured to generate a driving signal necessary for expressing an emotion in response to a signal provided by the emotion model unit of the intelligent unit; A facial expression unit and a gesture unit generating driving signals necessary for facial expressions and gesture expressions in response to signals output from the scenario unit and the emotion model unit of the intelligent unit; A physiological action unit generating a driving signal necessary for expressing a physiological action in response to the signal output from the homeostasis generator; A lip sync unit configured to generate a driving signal necessary to change a shape of a mouth in response to an output signal of the sound signal synthesis unit; And a motor arbiter having a database and controlling driving of a face motor, an arm, a body, a hand motor, and an eye and neck motor in response to the driving signals output from the above-described parts.

On the other hand, the present invention method for achieving the above object, the vision processing step of extracting the emotional information of the person using the image signal captured by the camera; Emotional information is extracted by using motion information extracted by the image signal captured by the camera, a voice signal input through a microphone, and a text signal input through a web, a messenger or a keyboard, and edited according to a motor applied to each axis. An intelligent processing step; And a motion control step of controlling motor driving of each part of the robot in response to a signal extracted and edited in the vision processing step and the intelligent processing step.

In this case, the vision processing step may include a motion recognition step of recognizing a color from the image information coming from the camera to distinguish the head and the hand, and recognize the motion; Recognizing a face and an object from the image information from the camera to analyze the distance information between the robot and the human, and the face and object tracking step for selecting the tracking operation to be performed by the robot;

The intelligent processing may include: recognizing the motion information recognized in the vision processing step, a voice signal input through a microphone, and a text signal input through a web, a messenger, or a keyboard; Extracting the personality and emotion model to be expressed by the robot using the recognized signals; Generating scenarios according to facial expressions and gestures to be executed by the robot using the extracted information; And synthesizing various sound signals including the voice signal.

The motion control may include generating control signals necessary for lip syncing, facial expressions, gestures, and emotional behavior through signals inputted from the vision processing step and the intelligent processing step; And a motor arbitration step of outputting a final position value to all the motors of the robot in response to the signal generated above to control its driving.

Meanwhile, another embodiment of the intelligent processing may include a voice signal input through a microphone to express an emotion, a text signal input through a web, a messenger or a keyboard, and a face and a moving object detected by a camera. Recognizing signals; Extracting an emotional word input through the dialogue engine unit and a motion, a face, and an object coming through the camera; Matching the extracted signals with a database in a scenario unit including emotions and gestures to be expressed; Generating physiological behaviors such as blinking eyes, yawning, drowsiness, hiccups; And transmitting the position information to the motor coordinator of the motion control unit for distributing the position information to the motors on each axis.

Hereinafter, one preferred embodiment according to the present invention will be described in detail with reference to the accompanying drawings.

1 shows a schematic block diagram of an emotion expression apparatus to which the present invention method is applied, FIG. 2 shows a detailed block diagram of a device to which the present invention method is applied, and FIG. 3 is a main flow for explaining the present invention method. The chart is shown.

4 is a detailed flowchart of a motion recognition step by color recognition of the vision processing step of FIG. 3, and FIG. 5 is a detailed flowchart of a face and object tracking step of the vision processing step of FIG. 3. 6 illustrates a detailed flowchart of an embodiment according to an intelligent processing step of the present invention method, FIG. 7 illustrates a detailed flowchart of another embodiment according to an intelligent processing step of the present invention method, and FIG. 8 illustrates the present invention. The detailed flowchart of the motion control step of the method is shown.

9 is a block diagram showing a personality model of a robot applied in the present invention, FIG. 10 is a two-dimensional emotional space diagram showing an emotional model of the robot applied in the present invention, and FIG. Figure 4 shows an example of the four emotions shown on the face of the robot applied in the invention, Figure 12 shows an illustration of the gestures performed by the robot applied in the present invention.

Accordingly, as shown in FIG. 1, the emotion expression apparatus to which the method of the present invention is applied includes a vision unit for extracting human emotional information through facial expression, color, motion, and recognition of an object captured by the camera 4. (1);

Emotional information is extracted from the personality, the emotion model and the text detected through the voice of the person input through the microphone 5 and the data provided by the vision unit 1, and the facial expression edited for the motors applied to the respective axes and An intelligent unit 2 for selecting and managing emotions to be expressed by the robot through a database 29 in which gestures are stored;

A motion control unit (3) transmitted from the vision unit (1) and the intelligence unit (2) to execute emotional actions using data, and to arbitrate a motor of each axis to perform a desired operation in the scenario unit (26). It is characterized by.

In this case, as shown in FIG. 2, the vision unit 1 recognizes a face, a color, and an object in an image captured by a camera 4 mounted on two eyeballs of a robot, respectively. A color recognizer 13 and an object recognizer 12;

A motion recognition unit 14 which receives the output signal of the color recognition unit 13 and derives positive / negative emotion signals by analyzing whether it is flesh color and movements of the head and hands;

The face tracking unit 15 which receives the output signals of the face recognition unit 11 and the object recognition unit 12, respectively, tracks the face and the object, and extracts the conversation distance by tracking the motion range of the eyes, the neck, and the waist, respectively. And an object tracker 16.

Also, as shown in FIG. 2, the intelligent unit 2 also includes a voice recognition unit 21 and a voice signal direction detection unit 22 for detecting whether a voice signal is input through a microphone and a direction of the input voice signal. )Wow;

The dialogue engine unit 23 which recognizes a text signal input through the web or a messenger or the keyboard 6 and a voice signal through the microphone 5 input through the voice recognition unit 21 and extracts positive / negative emotion signals. )Wow;

The personality model unit 24 which extracts two personalities of liveness and timidity by changing relative values using emotion information extracted from the motion recognition unit 14 and the dialogue engine unit 23 of the vision unit 1, respectively. )Wow;

An emotion model unit 25 for extracting an emotion to which personality is applied by applying a relative value provided by the personality model unit 24 to an emotion model axis;

Outputs the facial expression and gesture control signal of the robot to the motion controller 3 by using the signals input from each unit and the tags defined in the scenarios previously stored in the database 29, and simultaneously outputs a speech synthesis signal. A scenario unit 26 for making it;

An audio signal synthesizing unit 27 for outputting a speech synthesizing signal output from the scenario unit 26 through a spur 7;

It is characterized in that it comprises a; homeostasis generator 28 for generating physiological behavioral data using the input data.

In addition, as shown in FIG. 2, the motion controller 3 also includes an emotional behavior unit that generates a driving signal necessary for expressing an emotion in response to a signal provided by the emotion model unit 25 of the intelligent unit 2. (31);

In response to the signals output from the scenario unit 26 and the emotion model unit 25 of the intelligence unit 2, the facial expression unit 32 and the gesture unit 33 generating driving signals necessary for facial expression and gesture expression. )Wow;

A physiological behavior part 34 generating a driving signal necessary for expressing physiological behavior in response to the signal output from the homeostasis generator 28;

A lip sink unit 35 generating a driving signal necessary for changing a mouth shape in response to an output signal of the sound signal synthesizing unit 27;

A motor arbiter having a database 37 and controlling driving of the face motor 8, the arm, the body, the hand motor 9, and the eye and neck motor 10 in response to the drive signals output from the above-described parts ( 36); characterized by consisting of.

On the other hand, the present invention method for achieving the above object is, as shown in Figure 3, the vision processing step (S1) for extracting the emotional information of the person using the image signal captured by the camera (4);

Emotional information is extracted by using the motion information extracted by the image signal captured by the camera 4 and the audio signal input through the microphone 5 and the text signal input through the web, messenger, or keyboard 6. And intelligent processing step (S2) for editing according to the motor applied to each axis;

And a motion control step (S3) of controlling motor driving of each part of the robot in response to the signals extracted and edited in the vision processing step (S1) and the intelligent processing step (S2).

At this time, the vision processing step (S1) is a motion recognition step (S11) to distinguish the head and hands by recognizing the color from the image information coming from the camera 4, as shown in Figures 4 and 5 (S11) )Wow;

A face and object tracking step (S12) of recognizing a face and an object from image information received from the camera 4 to analyze distance information between the robot and a human, and selecting a tracking operation to be performed by the robot. It features.

In addition, as shown in FIG. 6, the intelligent processing step S2 may include a motion signal recognized in the vision processing step S1 and a voice signal input through the microphone 5, a web or a messenger, or the like. Recognizing a text signal input through the keyboard 6 (S21);

Extracting the personality and emotion model to be expressed by the robot using the recognized signals (S22);

Generating scenarios according to facial expressions and gestures to be executed by the robot by using the extracted information (S23);

And synthesizing the various sound signals including the voice signal (24).

In addition, as shown in FIG. 8, the motion control step S3 is required for lip synch, facial expressions, gestures, and emotional actions through signals input from the vision processing step S1 and the intelligent processing step S2. Generating a control signal (S31);

And a motor mediation step (S32) for outputting a final position value to all the motors of the robot in response to the generated signal to control its driving.

On the other hand, another embodiment of the intelligent processing step (S2) is a voice signal input through the microphone 5 to express the emotion by the robot, as shown in Figure 7, the web, messenger or keyboard 6 Recognizing text signals input through the camera and signals of a face and a moving object detected by the camera 4 (S25);

Extracting an emotional word input through the dialogue engine unit 23 and a motion and a face and an object coming through the camera 4 (S26);

Matching the extracted signals with a database (29) in the scenario unit (26) containing emotions and gestures to be expressed (S27);

Generating physiological behaviors such as blinking eyes, yawning, drowsiness, and hiccups (S28);

And transmitting (S29) the motor arbiter 36 of the motion control unit 3 to distribute the position information to the motors on each axis.

Referring to the operation and effect of the present invention made of such a configuration and steps as follows.

First, the emotion expression apparatus of the android robot to which the present invention is applied is mainly composed of the vision unit 1, the intelligence unit 2, and the motion control unit 3 as main technical components.

In this case, the vision unit 1 includes a face recognition unit 11, a color recognition unit 13, an object recognition unit 12, a motion recognition unit 14, a face tracking unit 15, and an object tracking unit 16. And a person's emotional information is extracted through facial expression, color and motion, and recognition of an object captured by the camera 4.

In the vision unit 1, the face recognizing unit 11, the color recognizing unit 13, and the object recognizing unit 12 are face and color in an image captured by a camera 4 mounted on two eyes of a robot. And objects are respectively recognized, and the motion recognition unit 14 receives the output signal of the color recognition unit 13 to derive a positive / negative emotion signal by analyzing whether the skin is colored or the movement of the head and hands.
In this case, the color recognized by the color recognizer 13 means extracting emotional information through the color of a face and a hand of a person, and the motion recognized by the motion recognizer 14 cuts off the head with a negative meaning. Shaking in unison, nodding in a positive sense, shaking hands to show welcome, or making an "O" or "X" shape with your arms in a positive sense, the color of the hand perceives motion As the information is used to inform the position of the hand, the color of the hand as well as the face is used to extract emotional information.

In addition, the face tracking unit 15 and the object tracking unit 16 receive the output signals of the face recognition unit 11 and the object recognition unit 12 to track the face and the object, respectively, Each range of motion is tracked to extract the conversation distance.
In this case, the recognition of the object does not mean extracting the emotional information only by the recognition of the object, and the object recognition unit 12 provides the necessary information to the object tracking unit 16, and for a specific object. Ongoing tracking can extract emotional information that increases curiosity about the object, provide a cause to focus attention on a specific object, and lead to a story about the object. Emotional information extraction through the curiosity is used to extract the emotion.

In addition, the intelligent unit 2 includes a voice recognition unit 21 and a voice signal direction detection unit 22, a dialogue engine unit 23, a personality model unit 24, an emotion model unit 25, and a scenario unit 26. Emotional from the personality and emotion model and text detected by the human voice input through the microphone 5 and the data provided by the vision unit 1 and the sound signal synthesis unit 27 and homeostasis generator 28 It extracts information and selects and manages emotions to be expressed by the robot through a database 29 in which facial expressions and gestures edited for the motors applied to the axes are stored.

In this case, the voice recognition unit 21 and the voice signal direction detection unit 22 detect whether the voice signal is input through the microphone 5 and the direction of the input voice signal, and the dialogue engine unit 23 Recognizes a text signal input through the web, a messenger or the keyboard 6 and a voice signal through the microphone 5 input through the voice recognition unit 21 and extracts a positive / negative emotion signal.

In addition, the personality model unit 24 changes the relative value by using the emotion information extracted from the motion recognition unit 14 and the dialogue engine unit 23 of the vision unit 1, respectively, so that both the lively and timid The personality is extracted, and the emotion model unit 25 applies a relative value provided by the personality model unit 24 to the emotion model axis to extract the emotion to which the personality is applied.

In addition, the scenario controller 26 uses the signals input from the respective units and the tags defined in the scenarios previously stored in the database 29 to control the facial expression and gesture control signals of the robot. The sound synthesizer 27 outputs the voice synthesized signal, and outputs the synthesized voice signal output from the scenario unit 26 through the sputter 7 at the same time. In (28), physiological behavioral data (ie, data such as blinking eyes, yawning, drowsiness, and hiccups) are generated using previously input data.

On the other hand, the motion control unit 3 is the emotional behavior unit 31, the facial expression unit 32 and the gesture unit 33, the physiological behavior unit 34, the lip sink unit 35 and the motor moderator 36 And transmits from the vision unit 1 and the intelligence unit 2 to execute emotional actions using data, and to mediate the motor of each axis to perform a desired operation in the scenario unit 26. .

In this case, the emotional behavior unit 31 generates a driving signal necessary for expressing emotion in response to the signal provided by the emotion model unit 25 of the intelligent unit 2, and the facial expression unit 32 and the gesture. The unit 33 generates driving signals required for facial expressions and gestures in response to signals output from the scenario unit 26 and the emotion model unit 25 of the intelligent unit 2, respectively.

In addition, the physiological behavior part 34 generates driving signals necessary for expressing physiological behaviors (ie, blinking, yawning, drowsiness, hiccups, etc.) in response to the signals output from the homeostasis generator 28. The lip sync unit 35 generates a driving signal necessary for changing the shape of the mouth in response to the output signal of the sound signal synthesizing unit 27, and the motor arbiter 36 includes a database 37. In response to the driving signal output from each part, the direct control of the driving of the face motor 8, the arm, the body, the hand motor 9, and the eye and neck motor 10 allows the Android robot to express emotions like humans. To make it possible.

On the other hand, the method of the present invention mainly consists of the vision processing step (S1), the intelligent processing step (S2) and the motion control step (S3) as the main technical components.

At this time, the vision processing step (S1) is a step of extracting the emotional information of the person by using the image signal captured by the camera (4), by recognizing the color from the image information coming from the camera (4) to the head and hands Distinguishing and recognizing a motion (S11), recognizing a face and an object from image information received from the camera 4, analyzing distance information between the robot and a human, and selecting a tracking operation to be performed by the robot. Step S12 is made.

In addition, the intelligent processing step (S2) is basically through the motion information extracted by the video signal captured from the camera 4, the audio signal input through the microphone 5 and the web, messenger or keyboard 6 Extracting the emotional information by using the input text signal and editing according to the motor applied to each axis, the embodiment is input through the motion information and the microphone 5 recognized in the vision processing step (S1) Recognizing the voice signal and the text signal input through the web or the messenger or the keyboard 6 (S21), and then extracting the personality and emotion model to be expressed by the robot using the recognized signals ( S22), and then generating a scenario according to a facial expression and gesture that the robot should execute using the extracted information (S23) and then generating a voice signal. And a step (S24) of synthesizing various sound signals.

Further, in another embodiment of the intelligent processing step (S2), the voice signal input through the microphone 5, the text signal input via the web, messenger or keyboard 6 and the camera to express the emotion by the robot Recognizing the signals of the face and the moving object detected in (4) (S25), and then the emotional words input through the dialogue engine 23 and the motion and the face and the object coming through the camera 4 are extracted. After performing step S26 and matching the database 29 in the scenario unit 26 containing emotions and gestures to express the extracted signals (S27), blinking and yawning are performed. Generating a physiological behavior such as drowsiness and hiccups (S28), and then transmitting to the motor arbiter 36 of the motion control unit 3 which distributes the position information to the motors on each axis (S2). 9) is carried out.

In addition, the motion control step (S3) is a step of controlling the driving of each motor of the robot in response to a signal extracted and edited in the vision processing step (S1) and the intelligent processing step (S2). After generating the control signals necessary for lip synch, facial expressions, gestures, and emotional behaviors through the signals input from S1) and the intelligent processing step S2, all motors of the robot in response to the signals generated above are generated. And a motor arbitration step S32 for outputting a final position value to control its driving.

In other words, the emotion expression method of the robot according to the present invention includes a three-step vision processing step (S1), an intelligent processing step (S2) and a motion control step (S3), in detail the input signal recognition step, Extracting personality and emotions or motions, faces, objects, and generating actions.

At this time, the camera 4 of the vision unit 1 is mounted on the eyeball of the robot to provide a visual field and position similar to that of a human. In the recognition step, information of a face, an object, and color is recognized, and the signals are motion. , To track the face, objects are provided.

The signals extracted through the above steps are compared and evaluated with the contents of the database 29 in the scenario unit 26 of the intelligence unit 2 in which the registered facial expression and gesture data are stored. to provide.

In addition, the motion control step S3 performed by the motor arbiter 36 of the motion control unit 3 is the lowest step of the emotion expression method, without interfering with the motors constituting the robot according to an embodiment of the present invention. In addition, the position value is provided so that the movement is natural.

In addition, the intelligent processing step (S2) is the highest level, the signal input to perform a specific task is combined to provide a signal in a lower step, as well as defining the input signals that can be used in the present invention as a tag as a scenario It also provides the necessary functions by tags.

Input signal tags provide lip sync, text, voice files, voice tone and speed, facial expressions, gestures, etc., and tags created as needed provide annotation, end, stop, start, and the like.

On the other hand, the vision unit 1 recognizes the flesh color from the signals provided through the camera (4). In order to extract the emotional signals from the human motion through the vision unit 1, a signal can be obtained from the movement of the head and hands which are related to the emotional signals.

Since a large amount of data processing is required to obtain detailed signals, the robot used in the present invention extracts two positive / negative signals from head and hand movements requiring a relatively small amount of data processing. The same head and hand separation can be obtained from the general human structural data.

Head and hand can be distinguished by color recognition because they have the same color but are different in area, range of motion, and location, and analyze the movements from the extracted head and hand to provide the positive and negative signals to the personality model. The state values calculated in the personality model are applied to the emotion model to provide emotional signals to the facial expressions and gestures executed in the scenario unit 26.

Normally, humans keep a certain distance from each other while talking to each other, which can vary according to cultural differences.

By recognizing the face and the object through the camera 4, the conversation distance can be determined, and if it enters into the conversation distance, it will show rejection, and if it is too far, it will try to reduce the conversation distance.

In addition, in order to communicate with the other party, a function of tracking a face and an object by looking at the other's face and grasping the emotions of the other party is required.

As you enter the conversation distance, you provide a signal to the motor moderator 36 to maintain the conversation distance.To track your face and objects, first determine if you can track within your pupil's range of motion. Tracks faces and objects.

Meanwhile, the intelligent unit 2 recognizes a voice signal input through the microphone 5 and a text signal input from the web, a messenger, or the keyboard 6, and the recognized signals are sent to the emotional engine through the dialogue engine 23. Extracted.

At this time, the dialogue engine of the dialogue engine unit 23 stores 10,000 text information and 100 gestures. If the extracted signals are not emotional signals, the text and gestures registered in the dialogue engine unit 23 are executed.

If emotional signals have been extracted, a positive / negative signal is provided to the personality model unit 24 to change an internal state value, and the changed state value is provided to the emotion model unit 25.

In addition, the state value of the personality model unit 24 changes the axis of the emotion model so that the emotion model has four tests of smile, sadness, surprise, and anger. FIG. 9 shows the personality model of such a robot. It is a block diagram.

As described above, the signals input by voice, vision, and text change the internal state value and the current state value of the personality model unit 24. In this case, the personality model has two personalities: vigorous and timid. Has

The initial internal state value is 0.5 and has a minimum of 0 and a maximum of 1. Active personality affects the smile and surprise axes of the emotion model, timid personality affects the sadness and anger axes of the emotion model, and the current state value is either added or added as the input signal is positive or negative. Erased.

에 입력된다. 수식으로 나타내면,Which is a step function

Is entered. In terms of a formula,

Same as

On the other hand, if positive signals continue to be provided, the personality of the robot to which the present invention is applied increases the state value of vigor, the state value of timidity decreases, and the current state value of the personality model affects the axis of the emotion model and thus Make your model personal.

10 is a two-dimensional emotional space diagram illustrating the emotion model of the robot applied in the present invention.

The expression of emotions in a continuous dimension of space is MIT, Kismet's emotion space method in Breazeal and Miwa's WE-3 method in Nagoya University.

Breazeal expressed nine emotions in three emotion axes, Arousal, Valence, and Stance. Miwa expressed seven emotions in three emotion axes: Pleasantness, Activation, and Certainty. Generally, happiness, sadness, anger, surprise, fear, and disgust are classified into six emotions.

In the robot to which the present invention is applied, the four emotions of happiness (smile), sadness, anger, and surprise, except for fear and disgust, which have a low frequency of emotions, are assigned to one axis (E ₀ -E ₃ ) based on the two-dimensional origin. do.

At one point in time, the inputs of the emotion model appear as a pair of four emotion values, with each emotion value represented as a value between 0 and 1, with index 0 being happy, 1 being angry, 2 being sad, and 3 being surprised. Indicates.

For example, the input of emotion shown in FIG. 11 is [e0 e1 e2 e3] = [0.8 0.3 0.4 0.4]. S ₀ to S ₄ are the areas of the area created by the line bisecting each emotion axis and are used to calculate the output of the emotion. At this time, the output of the emotion (Wi),

It is defined as

When the axis of emotion changes, the output value of the emotion for the same input changes, resulting in the response characteristics of different emotions.

Accordingly, each individual has a variety of reaction patterns, i.e. personality, according to his or her own experience. The axis of emotion becomes longer as the frequency of occurrence occurs and gets closer as the probability of simultaneous occurrence of emotion increases.

FIG. 12 shows the result of increasing the frequency of happiness (smile), increasing the E ₀ axis, increasing the probability of simultaneous sadness and surprise, and feelings of sadness and anger, resulting in closer axes of E ₂ , E _1, and E ₃ . This increases the responsiveness to the emotions that occur frequently and makes similar responses to the emotions that occur at the same time.

11 is an illustration of four emotions shown on the face of the robot applied in the present invention.

12 is an exemplary view of gestures performed by the robot applied in the present invention.

This invention can be implemented in other various forms, without deviating from the mind or main characteristic. For this reason, the above-described embodiments are merely examples in all respects and should not be interpreted limitedly.

The scope of the present invention is shown by the scope of the claims, which are not limited by the specification text, and all modifications and changes belonging to the equivalent scope of the claims are within the scope of the present invention.

As described above, according to the present invention, since the Android robot can express the emotional state with the skin and joint structure similar to the human, and can express the emotion to the other party through vision, voice, and text, it can communicate more friendly with the human. In addition to this, it is a very useful invention that can increase the demand for new technologies related to robots and contribute to the promotion of the robot market.

Claims (9)

delete delete delete delete A vision processing step of extracting emotional information of a person using an image signal captured by a camera; Emotional information is extracted by using motion information extracted by the image signal captured by the camera, a voice signal input through a microphone, and a text signal input through a web, a messenger or a keyboard, and edited according to a motor applied to each axis. An intelligent processing step; Motion control step of controlling the motor drive of each part of the robot in response to the signal extracted and edited in the vision processing step and the intelligent processing step; The vision processing step may include: a motion recognition step of recognizing color among image information received from a camera to distinguish a head and a hand, and recognizing a motion; A face and object tracking step of recognizing a face and an object from image information received from a camera, analyzing distance information between a robot and a human, and selecting a tracking operation to be performed by the robot; The intelligent processing step may include: recognizing motion information recognized in the vision processing step, a voice signal input through a microphone, and a text signal input through a web, a messenger, or a keyboard; Extracting the personality and emotion model to be expressed by the robot using the recognized signals; Generating scenarios according to facial expressions and gestures to be executed by the robot using the extracted information; Synthesizing various sound signals including a voice signal; The intelligent processing may include: recognizing a voice signal input through a microphone to express emotions, a text signal input through a web, a messenger or a keyboard, and signals of a face and a moving object detected by a camera; Extracting an emotional word input through the dialogue engine unit and a motion and a face and an object coming through the camera; Matching the extracted signals with a database in a scenario unit including emotions and gestures to be expressed; Generating physiological behaviors such as blinking eyes, yawning, drowsiness, hiccups; And transmitting the position information to the motor arbiter of the motion control unit for distributing the position information to the motors on each axis. The motion control step may include generating control signals necessary for lip syncing, facial expressions, gestures, and emotional behavior through signals input from the vision processing step and the intelligent processing step; And a motor mediation step of controlling a driving thereof by outputting a final position value to all motors of the robot in response to the generated signal. delete delete delete delete

Images