JP2020037155A - Gesture control device and gesture control program - Google Patents

Abstract

To provide a gesture control device which can express change in emotion of a robot, and a gesture control program.SOLUTION: A gesture control device 1 comprises: a gesture database 22 which stores emotion vectors of a plurality of emotion words in a space defined by a plurality of axes which express emotion, and allocates gesture of a robot to each of a plurality of areas into which the space is divided; an emotion word extraction part 12 which extracts emotion words from an input sentence; an emotion determination part 13 which determines emotion vectors at a transition destination in the space corresponding to the emotion words; and a gesture determination part 14 which gradually transits emotion vectors from emotion vectors at a transition source, which is currently set to the robot, to emotion vectors at the transition destination, and determines gestures allocated to areas corresponding to emotion vectors at respective steps in order.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an apparatus and a program for controlling a gesture of a robot.

  In order for the robot in the home to watch TV with the user and express the behavior of the robot responding to the viewing video, the robot must estimate the emotional state received from the viewing video and determine the gesture. Therefore, it is important to express emotional behavior as if it were reacting to the viewing video.

For example, Patent Literature 1 proposes a device that determines an emotional state of a robot using video impression information and user emotion information, and causes the robot to execute a synchronous reaction that makes the user feel that empathy has been obtained.
Patent Document 2 proposes a device that evaluates content based on reference data detected by a robot and controls the behavior or state of the robot based on the evaluation result.

JP 2016-36883 A International Publication No. 2012/172721

However, in Patent Literature 1, since the operation of the robot is limited to the synchronous operation for obtaining sympathy with the user, the emotion of the robot itself, particularly the change of the emotion, cannot be expressed.
Further, even in Patent Literature 2, even if the behavior or state of the robot is controlled based on the evaluation result of the content, it is not possible to express a change in the response of the changing content to the scene.
Therefore, the emotion expressed by the robot's gesture may suddenly change, and the user may feel uncomfortable with the robot's gesture.

  An object of the present invention is to provide a gesture control device and a gesture control program capable of expressing a change in the emotion of a robot.

  A gesture control device according to the present invention includes a first storage unit that stores emotion vectors of a plurality of emotion words in a space defined by a plurality of axes expressing emotions, and a plurality of areas that divide the space. A second storage unit to which the gesture of the robot is assigned; an emotion word extraction unit that extracts an emotion word from the input sentence; an emotion determination unit that determines a transition destination emotion vector in the space corresponding to the emotion word; From the emotion vector of the transition source currently set to the robot, the emotion vector is transitioned stepwise to the emotion vector of the transition destination, and the gesture assigned to the area corresponding to the emotion vector of each stage is determined in order. And a gesture determining unit.

  The gesture determining unit may determine a time required for a transition based on a distance between the emotion vector of the transition source and the emotion vector of the transition destination.

  The gesture determining unit may determine the number of steps for transitioning the emotion vector based on a distance between the transition source emotion vector and the transition destination emotion vector.

  The space may be a Russell ring model composed of two-dimensional axes using “pleasure-discomfort” and “awake-drowsiness” as indices.

  A gesture control program according to the present invention causes a computer to function as the gesture control device.

  ADVANTAGE OF THE INVENTION According to this invention, the change of the emotion of a robot can be expressed.

It is a block diagram showing the functional composition of the gesture control device concerning an embodiment. FIG. 3 is a block diagram illustrating a detailed functional configuration of a video information acquisition unit according to the embodiment. It is a figure which illustrates the emotion word dictionary which concerns on embodiment. It is a figure which illustrates the emotion vector memorize | stored in the gesture database which concerns on embodiment. It is a figure which illustrates the relationship between the emotion vector and the gesture stored in the gesture database which concern on embodiment. It is a figure which illustrates the step-by-step transition method of the emotion vector which concerns on embodiment, and the determination method of the corresponding gesture. It is a figure which illustrates the motor control data of the robot corresponding to the gesture concerning embodiment.

Hereinafter, an example of an embodiment of the present invention will be described.
FIG. 1 is a block diagram illustrating a functional configuration of a gesture control device 1 according to the present embodiment.
The gesture control device 1 is an information processing device (computer) including a control unit 10 and a storage unit 20, and further includes an input / output and communication interface. The gesture control device 1 realizes various functions of the present embodiment by causing the control unit 10 to execute software (gesture control program) stored in the storage unit 20.
Note that the gesture control device 1 may be built in a robot to be controlled, or may be configured to control the robot externally through communication.

  The gesture control device 1 controls the gesture of the robot by various functions of the present embodiment. The robot to be controlled in the present embodiment is a home robot such as a communication robot or a smart speaker. In addition, the gesture includes, in addition to the spatial operation caused by the operation control unit 4 of the robot operating the motor 5, the light emission of the light emitting device provided in the robot, or the change in the speed or intensity of the utterance by the robot. Refers to general operation.

  At this time, when the gesture control device 1 acquires the video and audio being watched by the robot from the camera 2 and the microphone 3, the gesture control device 1 extracts an emotion word expressing an emotion, and extracts the emotion state of the robot from the current state. The behavior of the robot is determined stepwise by gradually transitioning over time to the emotional state of the emotional word. The initial state of the emotion of the robot may be mapped in advance to any location of the Russell ring model.

The control unit 10 includes a video information acquisition unit 11, an emotion word extraction unit 12, an emotion determination unit 13, and a gesture determination unit 14.
The storage unit 20 includes an emotion word dictionary 21, a gesture database 22, a robot emotion unit 23, and a direction database 24.

The video information acquisition unit 11 analyzes video or audio information when watching a television or the like, and generates an input sentence for determining a gesture of the robot.
FIG. 2 is a block diagram illustrating a detailed functional configuration of the video information acquisition unit 11 according to the present embodiment.
The video information acquisition unit 11 includes a video acquisition unit 111, an audio acquisition unit 112, a video image analysis unit 113, an audio analysis unit 114, and a video information integration unit 115.

The video information acquisition unit 11 acquires video information from the camera 2 mounted on the robot by the video acquisition unit 111 and audio information from the microphone 3 by the audio acquisition unit 112.
The acquired video information is analyzed by the video image analysis unit 113 such as object recognition, scene detection, and character recognition, and the voice information is subjected to voice recognition by the voice analysis unit 114.
The video information integration unit 115 integrates the analysis results obtained from the video image analysis unit 113 and the audio analysis unit 114, and outputs video / audio analysis information. At this time, the video information integration unit 115 may perform filtering of the corresponding word and text of the video / audio analysis information in order not to use an inappropriate specific word or text.

The video / audio analysis information is provided to the emotion word extraction unit 12 as an input sentence for gesture control.
Note that the video information acquisition unit 11 may use a publicly available image recognition API or the like using a cloud network for analyzing video information or analyzing voice information.

The emotion word extraction unit 12 extracts an emotion word from the video / audio analysis information provided by the video information acquisition unit 11.
At this time, the emotion word extraction unit 12 extracts an emotion word using the emotion word dictionary 21. The emotion word extraction unit 12 acquires a sentence structure by performing syntax analysis such as morphological analysis and dependency analysis on the character string, and extracts an emotion word defined in the emotion word dictionary. For the morphological analysis, an open source morphological analysis engine (for example, “MeCab”, <http://takaku910.gifhub.io/mecab/>) or the like can be used.
The emotion word dictionary 21 may be collected from a web page, for example, using “Emotion Expression Dictionary, Akira Nakamura, Tokyodo Shuppan, 1993” or the like.

FIG. 3 is a diagram illustrating the emotion word dictionary 21 according to the present embodiment.
For example, a plurality of words showing the same emotion but different expressions are grouped and defined as a main word, word 1, word 2,...
The emotion word extraction unit 12 may extract a main word of the same group as an emotion word from words included in the video / audio analysis information.

Here, for each of the plurality of emotion words defined in the emotion word dictionary 21, an emotion vector in a space defined by a plurality of axes expressing emotions is stored in the gesture database 22.
This space may be, for example, a Russell ring model composed of two-dimensional axes of “pleasant-displeased” and “awake-drowsiness”. A plurality of emotion words are mapped in advance in such a space, and the coordinate value of the emotion vector for each emotion word is stored in the gesture database 22 (first storage unit).
Further, in the gesture database 22 (second storage unit), a gesture of the robot is assigned to each of the plurality of areas obtained by dividing the space.

FIG. 4 is a diagram illustrating an example of emotion vectors stored in the gesture database 22 according to the present embodiment.
For example, in the Russell ring model, the x-axis is provided with an emotional value representing "pleasure-discomfort", and the y-axis is provided with a degree of arousal representing "wake-up drowsiness". Thus, each of the four quadrants expresses emotions such as joy, anger, sadness, and relaxation. The emotion words included in the emotion word dictionary 21 are each mapped to unique coordinates and stored as emotion vectors.

FIG. 5 is a diagram exemplifying the relationship between the emotion vector and the gesture stored in the gesture database 22 according to the present embodiment.
The gesture of the robot is defined for each area obtained by dividing the space to which the emotion words are mapped into a plurality.
For example, the gesture “sad 1” is in the area “a _sad ≦ x <0, b _sad ≦ y <0”, and “sad 2” is in the area “x <a _sad , b _sad ≦ y <0”. Is assigned.
In this example, the areas are provided in a grid pattern at equal intervals, but the method of dividing the space is not limited to this.

  The emotion determination unit 13 extracts an emotion vector corresponding to the extracted emotion word from the gesture database 22 as a transition destination of the emotion of the robot, and determines an emotion vector of the transition destination in the space.

Gesture determination section 14, the transition from the transition source that is currently set in the robot emotion vector Er (a, b), emotion vector (a _{m, b m)} of a transition destination based on the view image emotional vector stepwise until Then, the gesture assigned to the area corresponding to the emotion vector at each stage is determined in order.
At this time, the gesture determining unit 14 may determine the time required for the transition based on the distance between the emotion vector of the transition source and the emotion vector of the transition destination, or may specify a predetermined time in advance.
In addition, the gesture determining unit 14 may determine the number of stages for transitioning the emotion vector based on the distance between the emotion vector of the transition source and the emotion vector of the transition destination, or may determine a predetermined number in advance. Good.

For example, when the emotion vector transitions in four stages from Er (-1, -1) to Em (1, 1) between time t0 and time t4, time t0: Er (-1, -1) and time t1: Er (−0.5, −0.5), time t2: Er (0, 0), time t3: Er (0.5, 0.5), time t4: Er (1, 1) Then, three emotion vectors are interpolated from the emotion vector Er of the transition source to the emotion vector Em of the transition destination.
Thereby, the gesture determining unit 14 gradually transitions the emotion vector Er of the transition source to the emotion vector Em of the transition destination, and determines the gesture of the robot according to each emotion vector at each stage during the transition.

FIG. 6 is a diagram exemplifying a method of stepwise transition of an emotion vector and a method of determining a corresponding gesture according to the present embodiment.
In this example, the gesture determining unit 14 changes the emotion vector Er () in four stages from the emotion vector Er (t0) of the transition source at time t0 to the emotion vector Em = Er (t4) of the transition destination at time t4. Let me.

For example, when the emotion vector Er (a, b) of the transition source corresponding to the emotion word “boring” is set in the robot, if the emotion word “happy” is newly detected from the live sound of television or the like. emotion vector Em transition destination corresponding to the detected emotional word (a m, _{b m)} is determined.
The distance between the emotion vector ^{_{R = √ {(a m -a}} ) 2 + (b m -b) 2} and the angle θ is obtained. When the distance R is divided into four equal parts and the emotion vectors are interpolated, the emotion vectors Er (t1), Er () at the respective stages in the direction of the angle θ from (a, b) for each change amount D = R / 4. t2), Er (t3), Er (t4) = Em are determined.

  Next, gestures defined for each area are extracted from the determined emotion vectors Er (t1), Er (t2), Er (t3), and Er (t4) at each stage, and the times t1, t2, At t3 and t4, the gestures "sad 1", "relax 1", "joy 1", and "joy 3" are executed, respectively.

The method of interpolation of the emotion vector by the gesture determining unit 14 is not limited to this, and for example, the following variations (1) to (3) can be adopted.
(1) The gesture determining unit 14 determines the emotion vector of each stage by dividing a line segment from the emotion vector of the transition source to the emotion vector of the transition destination into equal intervals D having a width equal to or larger than the threshold.
(2) The gesture determining unit 14 divides the time required for the transition, which is set in advance or determined according to the distance R or the coordinates of the transition source, into equal intervals having a width equal to or greater than the threshold value, thereby obtaining the emotion vector at each stage. To determine.
(3) The gesture determining unit 14 determines the emotion vector of each stage so that the total operation time of each gesture falls within the time required for transition.
The trajectory of the transition may be a curve, and the division of the distance or time may not be equal.

  When the gesture control device 1 changes the emotion vector as described above, the updated emotion vector is stored in the robot emotion unit 23. The stored emotion vector defines the current emotion of the robot, and becomes the transition source emotion vector when an emotion word is extracted next.

FIG. 7 is a diagram illustrating motor control data of the robot corresponding to the gesture according to the present embodiment.
For example, when the robot expresses the gesture of “joy 1”, the operations of operation 1 to operation 3 are sequentially performed with an operation time of 3000 msec. In the operation control data of each operation, a control value corresponding to each motor provided in the robot is described as an array. In this example, an array having five elements is used for a robot having five degrees of freedom. For example, the amount of rotation of the head of three axes and the amount of movement of the left and right arms are specified.
As a result, from the operation 1 to the operation 3, the operation of the robot raising the arm, which is the gesture of “joy 1”, is realized.

  In addition, the gesture of the robot may include an operation of pointing to the television or the viewer who is watching. The direction database 24 holds the directions of the television and the viewer set in advance or detected by the robot. The direction is represented, for example, by horizontal and vertical angles with respect to the front of the robot.

  The operation time of each gesture (for example, 3000 msec) may be adjustable within a predetermined range. Thereby, the robot can complete the gesture of each stage within the transition time of the emotion vector.

According to the present embodiment, the gesture control device 1 determines an emotion vector in a space in which an emotion state is defined by extracting an emotion word from a video viewed by the robot. Then, the gesture control device 1 changes the emotion vector of the transition source indicating the emotional state of the current robot to the emotion vector of the transition destination indicating the emotional state determined based on the viewing video in a stepwise manner. Of the robot according to the area to which the emotion vector belongs.
Thereby, the gesture control device 1 enables operation control for expressing a state in which the emotional state of the robot gradually changes with respect to the viewing video. In addition, the gesture control device 1 does not communicate with a television display or the like, and enables an operation of a robot that gradually makes a transition from a sad emotional state to a happy emotional state, for example. Therefore, the user can view the video together with the robot without feeling uncomfortable due to the robot expressing a sudden emotional change.

  The gesture control device 1 determines the time required for the transition based on the distance between the emotion vector of the transition source and the emotion vector of the transition destination, and takes appropriate time in accordance with the degree of change of the emotion to determine the emotion. Can express change.

  The gesture control device 1 determines the number of stages for transitioning the emotion vector based on the distance between the emotion vector of the transition source and the emotion vector of the transition destination. Emotional change can be expressed by number.

  The gesture control device 1 divides a line segment from the emotion vector of the transition source to the emotion vector of the transition destination into equal intervals having a width equal to or larger than the threshold value, thereby determining the emotion vector at each stage, and thereby the appropriate change amount. The emotion vector can be changed by using to change the emotion vector easily.

  The gesture control device 1 divides the time required for the transition into equal intervals having a width equal to or larger than the threshold value, determines the emotion vector at each stage, changes the gesture at appropriate time intervals, and easily changes the emotion. Can express change.

  The gesture control device 1 determines the emotion vector of each stage so that the total operation time of each gesture falls within the time required for the transition of the emotion vector, so that the operation time of the gesture for each stage does not interfere with each other. , And can appropriately express changes in the emotions of the robot.

  The gesture control device 1 uses a Russell ring model composed of two-dimensional axes of “pleasure-discomfort” and “awake-sleepiness” as a space for defining an emotion vector. Thereby, the gesture control device 1 can easily determine the emotion vector corresponding to various emotion words, the emotion vector in the process of changing the emotion, and the gesture for each area in the space. As a result, the gesture control device 1 can easily express a change in the emotional state based on the emotional word by determining a plurality of gestures in stages.

  The embodiments of the present invention have been described above, but the present invention is not limited to the above-described embodiments. Further, the effects described in the present embodiment merely enumerate the most preferable effects resulting from the present invention, and the effects according to the present invention are not limited to those described in the present embodiment.

  In the above-described embodiment, a spatial operation mainly controlled by a motor has been exemplified as a gesture of the robot, but is not limited thereto. For example, in the case of a smart speaker, the light emission of the light emitting device, or the speed or strength of the utterance, such as slowly blinking the LED or slowly uttering during relaxation, can be defined as a gesture corresponding to the emotional word. Also in this case, the gesture control device 1 can express the emotion change by gradually changing the light emission or the utterance according to the transition of the emotion vector.

  Further, in the above-described embodiment, the Russell ring model is exemplified as the space for defining the emotion. However, the space is not limited to two dimensions, and may be a tertiary or higher multidimensional space.

  In the present embodiment, the configuration and operation of the gesture control device 1 have mainly been described. However, the present invention is not limited to this, and a gesture control method or a gesture control program including each component and controlling the operation of the robot is provided. It may be constituted as.

  Furthermore, a program for realizing the function of the gesture control device 1 may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read and executed by a computer system to be realized. Good.

  Here, the “computer system” includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in a computer system.

  In addition, a "computer-readable recording medium" means that a program is dynamically stored for a short time, such as a communication line for transmitting a program via a network such as the Internet or a communication line such as a telephone line. For example, a program holding a program at a certain time, such as a volatile memory in a computer system serving as a server or a client in that case, may be included. Further, the above-mentioned program may be for realizing a part of the functions described above, or may be for realizing the above-mentioned functions in combination with a program already recorded in a computer system. .

Reference Signs List 1 gesture control device 10 control unit 11 video information acquisition unit 12 emotion word extraction unit 13 emotion determination unit 14 gesture determination unit 20 storage unit 21 emotion word dictionary 22 gesture database (first storage unit, second storage unit)
23 Robot Emotion Department 24 Direction Database

Claims (5)

A first storage unit that stores emotion vectors of a plurality of emotion words in a space defined by a plurality of axes expressing emotions;
A second storage unit to which a gesture of a robot is assigned to each of the plurality of areas obtained by dividing the space;
An emotion word extraction unit for extracting an emotion word from the input sentence,
An emotion determination unit that determines an emotion vector of a transition destination in the space corresponding to the emotion word,
From the emotion vector of the transition source currently set to the robot, the emotion vector is stepwise transitioned to the emotion vector of the transition destination, and the gesture assigned to the area corresponding to the emotion vector of each stage is determined in order. A gesture control unit comprising:   The gesture control device according to claim 1, wherein the gesture determining unit determines a time required for the transition based on a distance between the emotion vector of the transition source and the emotion vector of the transition destination.   3. The gesture control device according to claim 1, wherein the gesture determining unit determines the number of steps for transitioning the emotion vector based on a distance between the transition source emotion vector and the transition destination emotion vector. 4. .   The gesture control device according to any one of claims 1 to 3, wherein the space is a Russell ring model composed of two-dimensional axes using "pleasure-discomfort" and "awake-sleepiness" as indices.   A gesture control program for causing a computer to function as the gesture control device according to any one of claims 1 to 4.

Images