JP2013099823A - Robot device, robot control method, robot control program and robot system - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a robot system that improves the ability of expressing information to a user and also changes contents of an expression according to the involvement of the user.SOLUTION: The robot system uses a combination of a robot and an image display terminal and expresses a real service in a skit to facilitate the finding of a function and the emotional involvement of a user, thereby achieving improvement in the ability of expressing information to the user, and also changes contents of the expression in response to obtained user attribution and environmental attribution to prevent the user from being tired of the expression.

Description

  The present invention relates to a robot apparatus, a robot control method, a robot control program, and a robot system that operate in conjunction with a video display terminal.

  In recent years, home robots have been released by domestic and foreign manufacturers, and there are many opportunities to find them in ordinary homes. Among them, communication robots that respond to calls from people and provide information such as content while communicating with people are emerging. For example, there has been proposed one that selects and reproduces content that matches the emotion of a robot (see Patent Document 1).

  In the content reproduction system described in Patent Document 1, the robot changes its emotion according to the work from the outside and transmits the emotion information to the content reproduction device, so that the content reproduction device matches the emotion of the robot. By selecting and playing the content to be played, the emotion of the robot is expressed.

  In addition, as a way to keep users from getting bored, stories that are printed and recorded on print media and recording media such as books are dynamically changed according to changes in time, seasonal changes, and emotions detected by the robot. There has been proposed a robot that reads the different contents each time by modifying (see Patent Document 2).

JP 2005-169567 A JP 2002-205291 A

  However, in the configuration disclosed in Patent Document 1, the relationship between the external work and the emotional change of the robot is fixed, such as when the user hits the robot, the robot starts to get angry. Since the content cannot be changed, the user will eventually get bored with a robot that only repeats the same output.

  In the configuration disclosed in Patent Document 2, when the robot itself dynamically changes the expression content, the robot is inferior in expression of facial expressions and gestures compared to humans, so that the intention of the robot is less likely to be transmitted to the user. Since the medium itself does not reflect the user attribute and the environment attribute and is fixed, the information provided by the robot and the medium does not match each other.

  SUMMARY OF THE INVENTION In view of the above-described problems of the conventional technology, an object of the present invention is to provide a robot system that improves the ability to express information to a user and changes the content of expression with the user's involvement.

  The present invention is a video display terminal cooperation type robot system composed of a video display terminal and a robot, and a stage space having a robot as a performer is constructed by placing a video display terminal as a stage device beside the robot. is there.

  Since the video display terminal is equipped with display means such as a display, it functions as a visual aid to give persuasive power to the robot's performance, and is a tool or prop used to express the scene, season, time, atmosphere, etc. By displaying these objects, it is possible to increase the empathy of the user with respect to the contents conveyed by the robot.

  Since the video display terminal includes input detection means such as a touch panel, when the user touches an object of a big tool or a prop, an operation suitable for the feature of the object is executed. For example, if you touch a picture book object, which means the role of the picture book in the “Picture Book Storytelling” service, an animation that opens the cover of the picture book will be executed, and then the robot will speak the picture book story while gesturing. Traces the same behavior as reading aloud, making it easier for the user to communicate the robot's intention to move.

  Since the robot has user detection means such as face recognition by a camera, it is determined whether or not the person in front matches the user information stored in the storage means in advance. Referring to the title of the picture book stored and the last reading position, the story can be read from the previous continuation. In addition, when the user information of the person in front of the person is not stored in the storage means, the age of the person is determined by face recognition. For example, if the person is determined to be an infant, the robot can easily hear the voice quality and speed・ Select a way to speak, and the video display terminal can change the text of the picture book displayed on the display to a plain text for infants and a text expression that eliminates kanji, and then read aloud from the first page of the picture book.

  When the video display terminal conveys the object state to the robot, it gives and conveys a plurality of behavior patterns as robot behavior branches, and the robot determines the gender of the person in front by face recognition, A corresponding action pattern can be selected and executed. The behavior patterns of the plurality of utterances described above may be stored in advance in storage means provided in the robot as being linked to the object state, in addition to being transmitted from the video display terminal to the robot.

  The video display terminal, for example, arranges an object meaning the role of the clock in the “wake-up call” service on the display, and the robot wakes up to the user according to the elapsed time from the preset time and the user's awakening condition. It is possible to change the volume, pitch, inflection, speed, conversation, and the size, speed, and strength of body movement when making a call.

  The video display device gives priority to the object state to be transmitted to the robot when the robot operates an object different from the above immediately after the robot starts an action pattern such as speech and body movement by operating the object displayed on the display. Based on the degree information, the action pattern of the robot being executed can be canceled, and the action pattern associated with the subsequent object operation can be interrupted and executed.

  Since the robot has voice recognition means using a microphone or environmental sound recognition means, for example, in a “video content playback” service, the user can receive video content to be played back on the video display terminal by detecting the cheering or applause sound of the user. As the robot is clapping in synchronization with the user's clapping, the video display terminal changes to a visually flashy production.

  As described above, the present invention can provide a robot system that improves the ability to express information to the user and changes the content of the expression according to the user's involvement.

Functional block diagram showing a configuration example of a system including a robot in the first embodiment The figure which shows the state in which the robot in Embodiment 1-4 is used with a video display terminal Functional block diagram showing a configuration example of a system including a robot in Embodiments 2 to 4 Functional block diagram showing a configuration example of a system including a robot in the fifth embodiment The figure which shows the state in which the robot in Embodiment 5 is used with a video display terminal

  Embodiments of the present invention will be described below with reference to the drawings.

(Embodiment 1)
FIG. 1 is a functional block diagram showing a configuration of a system including a robot in the first embodiment. The system shown in FIG. 1 includes a video display terminal 200 and a robot 100.

  FIG. 2 is a diagram illustrating a state where the robot 100 is used with the video display terminal 200. By placing the video display terminal 200 as a stage device beside the robot 100, a stage space with the robot as a performer is constructed. The robot 100 and the video display terminal 200 are not limited to the horizontal arrangement of the left and right sides, but also include the arrangements before and after the vertical arrangement and the upper and lower arrangements of different levels depending on the contents to be expressed in conjunction with each other.

  The video display terminal 200 can be configured by, for example, a television device, a tablet, a PC, a PDA, a game machine, a mobile phone, an interphone, or the like.

  The video display terminal 200 includes a display unit 201, an input detection unit 202, an object state determination unit 203, and an object state transmission unit 204.

  The display unit 201 functions as a visual auxiliary unit for giving persuasive power to the performance of the robot, and displays objects of large tools and props for expressing scenes, seasons, times, atmospheres, and the like. For example, when the “picture book reading” service as shown in FIG. 2 is realized, a library object 201a with bookshelves arranged as a large tool showing a scene is displayed in the background, and a plurality of picture book objects 201b are displayed as “props”. “Picture book is closed”. Further, in order to indicate the season according to the current date and time, a seasonal object 201c such as a fan is displayed in summer and a stove is displayed in winter, or in addition to the time display of the clock object 201d to indicate time, the window object 201e. The background to be displayed is adjusted according to the brightness of the light emitted from the indoor lighting object 201f to adjust the color of the entire object to be displayed imitating the change in the wavelength of sunlight pointing from the outside, or to indicate the atmosphere of the place And adjust the color shade of the whole object.

  Since the input detection unit 202 is configured as a touch panel display in which the display and the touch panel are overlapped, the position coordinates touched by the user's finger can be detected. Therefore, among the multiple tools and props displayed on the display, the user can detect the position coordinates. An operation input such as which object touched is detected.

  In response to the operation input detected by the input detection unit 202, the object state determination unit 203 transitions the current object state to another state, and instructs the display unit 201 to display the apparent display form (shape, Color, size, position and orientation) can be changed.

  The object state transmitting unit 204 transmits to the robot 100 that the object state has been changed by the object state determining unit 203.

  The robot 100 and the video display terminal 200 are connected by a wired or wireless communication module, and the object state reception unit 105 of the robot 100 receives the data including the object state transmitted from the object state transmission unit 204 of the video display terminal 200. To do. For example, Wi-Fi (registered trademark), Bluetooth (registered trademark), or the like can be used as a communication form at this time.

  Each means constituting the video display means 200 is realized by a video display device operation program.

  The robot 100 includes a storage unit 101, an output unit 102, a user detection unit 103, a robot state determination unit 104, an object state reception unit 105, and a robot state transmission unit 106.

  The user detection unit 103 includes a camera for capturing an image including the user's face and body, and a microphone for collecting the sound emitted by the user. The user's face and voice feature information, age and gender estimation, facial expression recognition including the user's line of sight and voice frequency components and voice rhythm analysis technology. And emotion information such as concentration or excitement can be estimated. For example, the robot 100 includes an external sensor and an internal sensor. For example, as an external sensor, a touch sensor, a pressure sensor, a pulse sensor, or a thermal sensor that detects that a user touches the body surface of the robot 100. Examples of the internal sensor include an acceleration sensor, a gyro sensor, an electronic compass, and the like that detect movement and rotation operations when the user holds the robot 100. The information obtained by the user detection means 103 is handled as “current user information”.

Table 1 shows a configuration of “current user information” obtained by the user detection unit 103. One ID includes basic information such as age, gender, and preference from a robot to a user, facial feature information, voice feature information, emotion information, external sensor information, and internal sensor information.

  The output unit 102 includes a speaker for speaking and a mechanism having a plurality of degrees of freedom, and applies a driving force to the movable part to cause the robot to perform a body motion. As a driving force generation source (actuator) at this time, a technique such as a servo motor, artificial muscle, magnetic force, electrostatic force, or the like can be used. The robot 100 according to the present embodiment is a bear-type stuffed toy robot designed to imitate the shape and structure of a bear, which is a typical stuffed toy that is easy to blend in with people's daily life as shown in FIG. Because the exterior uses the same flexible material as the stuffed toy, you can directly touch and hold it with confidence. Bear-shaped stuffed animals are easy to accept as anthropomorphic characters of animals that perform human-like utterances and body movements, as have traditionally existed as children's play partners. Of course, this embodiment can also be applied to characters other than bears.

  The robot 100 is divided into a body part 100a, a head part 100b, an arm part 100c, and a leg part 100d, imitating an animal. The head portion 100b is arranged at the upper end position of the body portion 100a with degrees of freedom in the axial directions of roll, pitch and yaw at the neck position. The arm portion 100c is arranged at the left and right end positions of the upper half of the body portion 100a with degrees of freedom in the axial directions of the roll and pitch at the shoulder position and the roll and pitch at the elbow position. The leg part 100d is arranged at the left and right end positions of the lower half of the body part 100a with the degree of freedom of the pitch and yaw axial directions at the hip joint position.

  With the above configuration, it is possible to express some body movements (gestures) as non-linguistic information for prompting dialogue so that non-linguistic information sometimes conveys information more than language in communication between people. Specifically, nodding to mean consent, swinging to mean denial, head-to-head to question, eye contact to look at or disagree with the person you are talking to, raised hand to present spontaneous speech, invitation to ask for speech from the other party It is possible to perform gestures such as pointing at the subject of the topic and letting go of the feet to convey discomfort while synchronizing with the utterance.

  The storage unit 101 includes a storage medium for storing various information. Specifically, in addition to the “past user information” stored in advance, “a plurality of behavior patterns associated with the object state” can also be stored.

Table 2 shows a configuration of “past user information” stored in the storage unit 101. One ID includes basic information such as name, age, sex, and preference from the robot to the user, facial feature information, voice feature information, and a plurality of application information. In the application information, among the actions that the user has performed in the past, for example, the name of the picture book read so far, the reading position and the number of readings as application information related to the “picture book reading story” service are linked to the basic information. It can also be attached.

  The user detection unit 103 compares the facial feature information, in particular, with “current user information” obtained from the user in front of the robot 100 and “past user information” stored in the storage unit 101. By doing so, the user can be personally identified.

Table 3 shows the configuration of the “behavior pattern” stored in the storage unit 101. One ID includes a plurality of pieces of behavior pattern information associated with an object name and an object state. The behavior pattern has information related to utterances and body movements to be output by the robot.

  For example, regarding the “telebook storytelling” service, action patterns such as “speak aloud”, “promote a book”, “wait” are prepared. In addition, when “action probability” is set for each action pattern, the value of user preference included in the “previous user information” that increases according to the number of interactions between the user and the robot, The higher the concentration value included in the “current user information” obtained from the user, the higher the action probability is given to the action. When “behavior condition” is described in each action pattern, the robot selects an action pattern according to the condition.

  The robot state determination unit 104 receives the object received from the video display terminal 200 based on at least one of “past user information” stored in the storage unit 101 and “current user information” acquired by the user detection unit 103. By determining one from a plurality of behavior patterns associated with the state and transitioning the current robot state to another state, the robot outputs utterances or body movements by the function of the output means 102.

  The robot state transmission unit 106 transmits to the video display terminal 200 that the robot state has been changed by the robot state determination unit 104.

  Each means constituting the robot 100 is realized by a robot operation program.

  As a specific example of operation, an example of the “Picture Book Storytelling” service is shown below.

  A video display terminal 200 shown in FIG. 2 includes a touch panel display in which the display unit 201 and the input detection unit 202 are overlapped, and a library in which bookshelves are arranged as a large tool for showing a scene when the “telebook reading” service is realized. The object 201a is displayed in the background, and a plurality of picture book objects 201b are displayed in a “picture book closed state” as props. Further, the time display of the season object 201c for impressing the current season (in this case summer = electric fan) and the clock object 201d indicating the time, and the window object 201e in which the color outside the window is blue because it is daytime, The indoor lighting object 201f is displayed to show the atmosphere, and the shades of surrounding colors are corrected to lighten to show that the lighting is on. The picture book object 201b has three states of "a picture book is closed", "a picture book is opened", and "a picture book is being read".

  When the user touches the picture book object 201 a displayed on the touch panel display with a finger, it is determined that the picture book object has been operated by the operation of the input detection unit 202.

  Further, the object state determination means 203 changes the object state of the picture book from “the picture book is closed” to “the picture book is opened”, and at the same time, instructs the display means 201 to perform an animation for opening the picture book cover. After execution, the background showing the symbolic scene in the story of the picture book is displayed on the display, and the object state of the picture book is changed to “the picture book is opened” via the object state transmission means 204. To 100.

  The robot 100 includes facial feature information of “current user information” obtained from the person in front of the user detection unit 103 and facial feature information of “past user information” stored in the storage unit 102 in advance. Is matched.

  The object state receiving unit 105 receives “the state where the picture book is opened”, and the robot state determination unit 104 selects an action pattern of “read a picture book” with an action condition of 90% with reference to IDc001 in Table 3 When the facial feature information matches, the “picture book name”, “reading position until the previous time”, and “the last reading position” of the application information stored in association with “past user information” of IDb000 in Table 2 are stored. With reference to the “telephone setting”, it is possible to execute reading from the previous continuation by instructing the output means 102.

  Further, when the facial feature information does not match, the age of the person is determined by the face recognition of the user detection unit 103. For example, when the person is determined to be an infant, the robot state determination unit 105 selects " “Read a picture book” is selected, and the output means 102 is instructed to read from the first page of the picture book with a voice quality, speed, and way of speaking that is easy for an infant to hear.

  As described above, the picture book is opened by the user's action to open the cover of the picture book object 201b, and then the robot 100 talks while gesturing the story of the picture book according to the characteristics of the person in front. Since the same series of actions as reading a picture book to children can be traced, it is easier for the user to communicate the intention of the robot to read the picture book.

  When the user operates the object B immediately after the user operates the object A in a state where there are a plurality of objects to be displayed on the display in the user's touch display operation, the utterance or the body movement as a result of the robot 100 operating the object A The object state transmitting unit 204 provides a mechanism for giving priority to the object state and transmitting it to the robot 100 so that the user's operation after the execution is continued and the robot is not ignored. Specifically, when the object A is operated first, priority (normal) is given and transmitted to the robot 100, and the object B is operated before reaching the time of the robot's utterance and body movement that is assumed in advance. If it is, the priority (high) is given and transmitted to the robot 100. When the robot 100 receives information having a higher priority, the robot 100 can cancel the action pattern of the robot being executed and execute the action pattern associated with the subsequent object operation. To do.

  The above embodiment is an example, and the configuration example, operation example, use example, and external shape example of the robot 100 and the video display terminal 200 are not limited to the above example.

(Embodiment 2)
FIG. 3 is a functional block diagram illustrating a configuration example of a system including the robot according to the second embodiment. The system shown in FIG. 3 includes a display storage unit 205 and a robot state reception unit 206 in the video display terminal 200.

  Since the robot 100 and the video display terminal 200 are connected by wired or wireless communication, the robot state transmission unit 106 acquires the “past user information” stored in the storage unit 101 and the user detection unit 103. At least one of “current user information” can be given to the robot state and transmitted.

  The display storage unit 205 includes a storage medium for storing various information. Specifically, a “display pattern” can be stored for each object.

Table 4 shows the configuration of the “display pattern” stored in the display storage unit 205. One ID includes an object name, an object state, a plurality of display pattern information linked to the robot state and user information, and an object attribute. In the object attribute, the type indicating the role group of the object, the target age of using the object, and the effective period of using the object can be associated with the ID.

  The object state determination unit 203 selects a display pattern associated with the robot state received from the robot 100. If user information is given, the object state determination unit 203 corresponds to the information included in the user information and the object attribute. The display state of the display object is determined and displayed by the function of the display means 101.

  As a specific example of operation, an example of the “Picture Book Storytelling” service is shown below.

  On the display unit 201 of the video display terminal 200 shown in FIG. 2, a library object 201 a in which bookshelves reminiscent of a “telephone” service scene and a picture book object 201 b in a “picture book closed” state are displayed.

  Here, when a user (infant) who expects to read the “telephone service” stands in front of the robot 100, the robot state determination unit 104 changes the robot state from “standby state” to “state for searching for a picture book”. At the same time, it instructs the output means 101 to drive the arm portion 100c to execute a body motion that points to the library object 201a with the bookshelf by hand. Next, the “current user information” includes a robot state that means that the robot has transited to “a state of searching for a picture book” via the robot state transmission unit 106, and the user detection unit 103 includes information that the user is an infant. Is transmitted to the video display terminal 200.

  The object state determination unit 203 refers to the information “book and target age is infant” of the object attribute in Table 4 in order to arrange the picture books whose target age is an infant on the bookshelf according to the display pattern of IDd050 in Table 4. Furthermore, when receiving the robot status of “Reading a picture book”, according to the display pattern of IDd001 in Table 4, the text of the picture book displayed on the display can be changed to a plain text for infants and a character expression without kanji. .

  As described above, when the robot 100 alone dynamically modifies the utterance and body motion expression contents based on the user information by the combination of the first embodiment and the second embodiment, the medium that is the modification source Since the display content of the video display terminal 200 is also modified in conjunction with the user attributes, the information provided by the robot 100 and the video display terminal 200 always matches, so that the function discovery and emotion transfer by the user can be performed. Easily improve the ability to express information to the user, and change the content of the expression to reflect the user attributes and environment attributes that are obtained, thereby preventing the user from getting bored.

  The user information received by the video display terminal 200 is an example. Besides, basic information such as the user's name, age, and gender, facial feature information, voice feature information, emotions, concentration and excitement levels. There is emotion information.

(Embodiment 3)
Here, a mode will be described in which the robot 100 determines and outputs one action among “a plurality of action patterns expected from the robot” received from the video display device 200.

Table 5 shows the configuration of “behavior expected pattern” stored in the display storage unit 205 of the video display terminal 200. The format of this information is the same as the “behavior pattern” stored in the storage unit 101 of the robot 100 described above, and the content is changed.

  For example, a plurality of behavior pattern information such as “send a story”, “promote a book”, “ask to read a picture book”, and “wait” are prepared as behavior pattern information related to the “telebook storytelling” service.

  In response to the operation input detected by the input detection unit 202, the object state determination unit 203 is associated with the object state of the object in Table 5 in addition to changing the current object state to another state. A plurality of behavior patterns expected from the robot are transmitted to the robot 100.

  The robot state determination unit 104 receives the object received from the video display terminal 200 based on at least one of “past user information” stored in the storage unit 101 and “current user information” acquired by the user detection unit 103. By determining one from a plurality of behavior patterns associated with the state and transitioning the current robot state to another state, the robot outputs utterances or body movements by the function of the output means 102.

  The robot state transmission unit 106 transmits to the video display terminal 200 that the robot state has been changed by the robot state determination unit 104.

  The robot state determination unit 104 receives the object received from the video display terminal 200 based on at least one of “past user information” stored in the storage unit 101 and “current user information” acquired by the user detection unit 103. By determining one from the state and a plurality of behavior patterns expected of the robot and transitioning the current robot state to another state, the robot outputs utterances or body movements by the function of the output means 102.

  As a specific example of operation, an example of the “Picture Book Storytelling” service is shown below.

  When the user touches the picture book object 201 a displayed on the touch panel display with a finger, it is determined that the picture book object has been operated by the operation of the input detection unit 202.

  Further, the object state determination means 203 changes the object state of the picture book from “the picture book is closed” to “the picture book is opened”, and at the same time, instructs the display means 201 to perform an animation for opening the picture book cover. After execution, the background showing a scene from the story of the picture book is displayed on the display. Subsequently, the object state of the picture book is changed to “the state where the picture book is opened” via the object state transmission unit 204, and a plurality of robots associated with the object state of IDe001 in Table 5 are expected. Two action patterns, “Ask to read the picture book” and “Close the picture book” are assigned and transmitted to the robot 100.

  When the object state receiving unit 105 receives “the state that the picture book is opened”, the robot state determining unit 104 uses the user detecting unit 103 according to the “behavioral condition” and the facial feature information is “past user information”. Whether or not is matched.

  When the facial feature information matches, the action pattern of “Ask to read a picture book” is selected, and “Picture Book Name” of application information stored in association with “Past User Information” of IDc001 in Table 3 is stored. And “Number of readings”. If the corresponding picture book has been read five times or more, the robot 100 outputs an utterance asking whether or not to read the picture book and a body motion, and then the face recognition of the user detecting means 103 means “yes” of the user. When recognizing the vertical movement of the eye or nose organ as an action, the reading is executed from the previous continuation with reference to “Reading setting”. Further, the face detection of the user detection means 103 recognizes the left and right movements of the eyes and nose organs as the swing motion that means “no” of the user, or recognizes neither the nodding motion nor the swing motion. If more than one second has elapsed, an action of closing the picture book is executed. If the number of readings is less than 5, the reading is executed with reference to the “reading position until the previous time” and “story setting”.

  Further, when the facial feature information does not match, the robot state determination unit 104 selects an action pattern of “close picture book” and executes an action to close the picture book.

  As described above, by transmitting a plurality of expected behavior patterns to the robot 100 from the video display terminal 200, the capacity for storing the behavior patterns in the storage means 101 on the robot side can be saved. Also, unlike the old behavior patterns stored in the robot 100, for example, by giving a new behavior pattern that changes according to the scene, season, time, atmosphere, etc. from the video display terminal 200, the interaction of the robot 100 becomes diverse. The user can interact with the robot 100 for a long time without getting bored.

(Embodiment 4)
A mode in which the robot 100 determines and outputs an action according to the “arranging means” received from the video display device 200 will be described.

  For example, regarding the “wake-up call” service, action patterns such as “notify”, “stop clock”, and “wait” are prepared in correspondence with the clock as shown in IDs d040 and d041 in Table 4 and stored in the storage unit 101. . In addition, the behavior pattern includes information about the utterance and the body movement for the robot to output according to the story of the picture book to be read.

  Since the type of the clock object 201d is a timer, the object state determination unit 203 changes the object state from “time display state” to “alarm state” when the preset alarm time is reached, and the state changes. And the arrangement means are transmitted to the robot 100.

  As a specific operation example, an example of a “wake-up call” service is shown below.

  When the object state determination unit 203 refers to IDd040 in Table 4 and determines that the wake-up time set at the clock object 201d has reached 7:00, the object state of the clock in Table 4 is changed from “time display state” to “ At the same time as the transition to the “alarm state”, the display unit 201 is instructed to execute an animation that the clock rings, and then the object state of the clock transitions to the “alarm state” via the object state transmission unit 204; The parameter “small voice” as the arrangement means is transmitted to the robot 100.

  When the object state receiving unit 105 of the robot 100 receives an “alarm state” at 7:00, the robot state determining unit 104 refers to the IDc 041 in Table 3 and the person who is in front of the user by the user detecting unit 103 If the degree of concentration obtained from the movement or utterance of the line of sight / expression is less than the threshold value, it is determined that the consciousness is not awake (sleeping), and the output means 101 is instructed to “notify the wake-up call” Is executed at a low volume corresponding to the arrangement means “low voice”. When the robot state means 206 of the video display terminal 200 receives the “notification state”, the object state determination means 203 refers to IDd041 in Table 4 and sounds the alarm again after 1 minute.

  When the above flow is repeated every 1 minute until the user's concentration level exceeds the threshold, the parameters as the arrangement means such as “loud voice”, “big movement” and “sound loud” Program to change.

  Corresponding to the arrangement means received by the robot, speak with a louder volume corresponding to the “loud voice”, and move the body and limbs away from the torso center in response to the “big movement”. It is possible to prompt the user to wake up (wake up) by changing and acting with a large sound effect such as a plosive sound or a hitting sound in response to “making a loud sound”. Finally, if the degree of concentration obtained from the person in front of the user by the user detection means 103 is equal to or greater than the threshold value, it is determined that the consciousness has been awakened (wakes up), and the output means 101 is instructed to “ Stop the watch "and perform the physical action. When the robot state means 206 of the video display terminal 200 receives the “state to stop the clock”, the object state determination means 203 refers to IDd041 in Table 4 and transitions the clock to the time display state.

  As described above, the capacity of the storage unit 101 can be saved by changing the utterance or body movement stored in the storage unit 101 of the robot 100 by providing the arrangement unit from the video display terminal 200. Also, the emotional expression of the robot 100 becomes diverse, and the user can interact with the robot 100 without getting tired.

  The above utterance and body movement arrangement methods are only examples, and in addition, the size of the voice, the inflection of the voice, the tempo of the voice, the nature of the voice (men and women), the magnitude of the movement, the speed of movement, the power of movement There is strength.

(Embodiment 5)
FIG. 4 is a functional block diagram illustrating a configuration example of a system including the robot according to the fifth embodiment. The system shown in FIG. 4 includes voice recognition means 107 and environmental sound recognition means 108 in the robot 100.

  As a specific operation example, an example of a “video content playback” service of a concert is shown below.

  FIG. 5 is a diagram illustrating a state where the robot 100 is used together with the video display terminal 200. The display unit 201 arranges a moving image content object 201g as an area for reproducing a moving image, a spotlight object 201h for attracting the attention of the audience as a production device, and a mist generator object 201i for creating a fantastic atmosphere. In FIG. 5, these effect devices are arranged in the display area of the display so that they can be seen. However, in the actual usage scene, they may be arranged outside the display area of the display so that they cannot be seen.

Table 6 shows the configuration of the “display pattern” stored in the display storage unit 205. The format of this information is the same as in Table 4 above, and the contents are changed.

  The object state determination unit 203 changes the moving image content object 201g from the “stop state” to the “playback state”, thereby playing the concert music and video on the display, and changing the spotlight object 201h from the “light off state” to “ By making the transition to the “lighting state”, a yellow color meaning spotlight light and a triangle having a transmittance of 50% and having the spotlight as a vertex are superimposed and displayed on the moving image content object 201g. The mist generator object 201i remains in the stopped state. The robot 100 placed beside the user monitors the music and video of the concert reproduced on the video display terminal 200 with the camera and the microphone of the user detection unit 103.

  When detecting a change in the opening / closing state of the user's mouth by the face recognition of the user detection unit 103, the sound information input to the microphone is transferred to the voice recognition unit 107, and the presence or absence of the utterance is detected by analyzing the content of the user's utterance. . In this way, by synchronizing the mouth open / closed state and voice recognition, adverse effects on recognition due to ambient noise are reduced. As an example of the analysis of utterance content, the image of the words that have been converted to text is included in a “good” group (such as “good”, “great”, “interesting”, etc.), and the frequency components of the speech and the rhythm of the voice are analyzed. Based on the fact that elements of joy and comfort are included, the level of “cheers” can be determined.

  Further, when detecting a change in the user's hand movement by the image recognition of the user detection means 103, the sound information input to the microphone is transferred to the environmental sound recognition means 108, the environmental sound is analyzed, and the applause sound and the hitting sound are detected. Detect the presence or absence. In this way, by synchronizing hand movement and environmental sound recognition, adverse effects on recognition caused by ambient noise are reduced. As an example of the analysis of the environmental sound, the level of the “applause sound” can be determined by analyzing the frequency components of the sound and including many sound features similar to applause.

  Here, when the environmental sound recognition unit 108 determines that the level of the applause sound is larger than a preset threshold value, the robot state determination unit 104 changes the robot state from “standby state” to “applause state”. At the same time, the output unit 101 may be instructed to drive the arm portion 100c to applaud in synchronization with the timing of the user's applause sound.

  The robot state determination unit 104 changes the robot state from the “standby state” to the “user detection state”, and changes the robot state to the “user detection state” via the robot state transmission unit 106. The voice recognition result recognized by the voice recognition unit 107 or the environmental sound recognition result recognized by the environmental sound recognition unit 108 is added and transmitted.

  When the received speech recognition result given to the robot state of the “user detection state” includes “cheer”, the object state determination unit 203 follows the display pattern of IDe021 in Table 6 and the cheer level is equal to or greater than the threshold value. If so, the color of the spotlight object 201h is changed in the order of yellow → red → blue → yellow, and a triangle representing the light of the spotlight is animated like a pendulum around the spotlight. If the cheering level is less than the threshold value, the spotlight object 201h is kept lit in a single color.

  When the received environmental sound recognition result given to the robot state of the “user detection state” includes “applause sound”, the object state determination unit 203 follows the display pattern of IDe030 in Table 6 to determine the level of applause sound. Is greater than or equal to the threshold value, the mist generator object 201i is transitioned from the “stop state” to the “execution state”, and a white cloud shape having a transparency of 50%, which means a mist effect, is superimposed on the lower part of the moving image content object 201g. Display an animation that moves left and right. If the level of the applause sound is less than the threshold value, the mist object 201i maintains the stopped state.

  As described above, since the robot has “voice recognition means” or “environmental sound recognition means” using a microphone, by detecting the cheers or applause sounds of the user, the user can view the video content to be played back on the video display terminal. The robot is clapping in sync with the user's clapping, and the video display terminal can be transformed into a visually flashy presentation to increase the emotional transfer of the user's video content. .

  In the above, examples of the embodiment of the present invention have been described as the first to fifth embodiments, particularly the “picture book reading”, “wake-up call”, and “video content playback” services, but the present invention has been described above. It is not limited to form.

  In recent years, many real-time information services have been built on the Internet by IT technology, and it has become possible to output information on a browser and input information using a mouse or keyboard, but natural interactions that interact with humans are impaired. Therefore, it is difficult for a child, an elderly person, or a person unaccustomed to handling an information terminal to enjoy the information service. In the present invention, a combination of “robot = actor” and “video display terminal = stage apparatus” is used, for example, in the category “sales”, the robot caster is replaced with a store clerk, and the video display terminal has “pharmacy” “food department”. ”“ Clothing department ”, etc. By switching and displaying the actual service, the information service system can output information in a more understandable way by inputting the information through natural interaction with the user. Can be provided.

  The present invention is a combination of a robot and a video display terminal, which makes it easy for users to discover functions and transfer emotions by expressing skits of actual services, and outputs information in a way that is easier to understand for users. It is useful as an information service system for inputting information.

DESCRIPTION OF SYMBOLS 100 Robot 100a Body part 100b Head part 100c Arm part 100d Leg part 101 Memory | storage means 102 Output means 103 User detection means 104 Robot state determination means 105 Object state reception means 106 Robot state transmission means 107 Voice recognition means 108 Environmental sound recognition means 200 Image | video Display terminal 201 Display means 201a Library object 201b Picture book object 201c Seasonal object 201d Clock object 201e Window object 201f Indoor lighting object 201g Movie content object 201h Spotlight object 201i Mist generator object 202 Input detection means 203 Object state determination means 204 Object state transmission Means 205 Display storage means 206 Robot state Shin means

Claims (16)

A “video display terminal cooperative robot system” composed of a “video display terminal” and a “robot”, wherein the video display terminal detects “input means” for displaying an object and “input” for detecting an operation input to the display means. "Detection means", "object state determination means" for determining the state of the object that changes based on the input to the object detected by the input detection means, and displaying it by the display means, and the determination result of the object state determination means Is transmitted to the robot, and the robot transmits "output means" capable of at least one of speech and physical movement, "user detection means" for detecting user behavior, and user information. "Storage means" for storing and "Object state" for receiving "Object state" from the video display terminal One of a plurality of action patterns associated with the object state, based on at least one of past user information stored in the storage unit and current user information acquired by the user detection unit. And a “robot state transmission unit” for transmitting a determination result of the robot state determination unit to the video display terminal. The video display terminal receives simultaneously a “display storage unit” that stores a plurality of display patterns associated with a robot state, and a “robot state reception unit” that receives the “robot state” from the robot. A video display terminal cooperation robot, comprising: an “object state determination unit” that determines one of a plurality of display patterns stored in the display storage unit based on current user information and displays the display pattern by the display unit system. The video display terminal according to claim 1, further comprising a “video display terminal state transmission unit” that adds a plurality of behavior patterns expected of the robot to the determination result of the object state determination unit and transmits the behavior pattern to the robot. Is based on at least one of past user information stored in the storage means and current user information acquired by the user detection means, and is one of a plurality of behavior patterns expected from the robot received from the video display terminal. An image display terminal cooperation robot system comprising “robot state determination means” for determining and outputting an action. The robot according to claim 1 stores a plurality of behavior patterns linked to an object state in the storage unit, past user information stored in the storage unit, and current user information acquired by the user detection unit. And a robot state determining means for determining and outputting one action among a plurality of action patterns stored in the storage means based on at least one of the above. The video display terminal includes an “video display terminal state transmission unit” that adds an arrangement unit to the determination result of the object state determination unit and transmits the result to the robot. The robot receives the arrangement received from the video display terminal. A video display terminal cooperation robot system, comprising: a “robot state determination unit” that arranges at least one of an output of speech and body movement based on the unit. The video display terminal includes a “video display terminal state transmission unit” that adds a behavior pattern having a priority attribute to the determination result of the object state determination unit, and transmits the behavior pattern to the robot. The robot is currently being executed. A behavior pattern of a high priority attribute by comparing the priority attribute received from the video display terminal with the priority attribute of the action currently being executed. An image display terminal cooperation robot system comprising “robot state determination means” for determining and outputting the image. The robot includes a “voice recognition unit” for recognizing a voice uttered by a user, and a robot that adds the voice recognition result recognized by the voice recognition unit to the determination result of the robot state determination unit and transmits the result to the video display terminal. The video display terminal includes a “display storage unit” for storing a plurality of display patterns associated with the voice recognition result, and the display storage unit based on the voice recognition result received at the same time. A video display terminal cooperation robot system comprising “object state determination means” for determining one from a plurality of stored display patterns and displaying the display pattern by the display means. The robot displays an image by adding an “environmental sound recognition unit” for recognizing a transmission source from the voice of the user's surrounding environment, and an environmental sound recognition result recognized by the environmental recognition unit to the determination result of the robot state determination unit. A "robot state transmission means" for transmitting to the terminal, wherein the video display terminal includes a "display storage means" for storing a plurality of display patterns associated with the environmental sound recognition result, and an environmental sound recognition result received simultaneously. An image display terminal cooperation robot system comprising: an “object state determination unit” that determines one from a plurality of display patterns stored in the display storage unit and displays the display pattern on the display unit. A video display terminal capable of communicating with a robot, wherein a “display unit” for displaying an object, an “input detection unit” for detecting an operation input to the display unit, and an input to the object detected by the input detection unit An image display terminal, comprising: an “object state determining unit” that determines a state of an object that changes based on an object, and an “object state transmitting unit” that transmits a determination result of the object state determining unit to a robot. A robot capable of communicating with the video display terminal according to claim X, wherein an “output means” capable of at least one of speech and physical movement, a “user detection means” for detecting user behavior, and user information "Storing means" for storing, "Object state receiving means" for receiving "Object state" from the video display terminal, Past user information stored in the storing means, and current user information acquired by the user detecting means Based on at least one of the above, a “robot state determination unit” that determines and outputs one of a plurality of behavior patterns associated with the object state, and transmits the determination result of the robot state determination unit to the video display terminal A robot comprising “robot state transmission means”. A video display terminal capable of communicating with a robot having a “user detection means” for detecting a user's behavior, and storing a “display means” for displaying an object and a plurality of display patterns associated with the state of the robot. Display storage means ”,“ robot state reception means ”for receiving“ robot state ”including the robot action result from the robot, and a plurality of the stored in the display storage means based on the current user information received simultaneously. An image display terminal comprising “object state determination means” for determining one from display patterns and displaying the display pattern by the display means. A video display terminal operation program capable of executing processing on a video display terminal that is communicable with a robot and includes a “display unit” that displays an object and an “input detection unit” that detects an operation input to the display unit, “Object state determination process” that determines the state of the object that changes based on the input to the object detected by the input detection unit and displays it by the display unit, and transmits the determination result of the object state determination process to the robot “ An image display terminal operation program that executes an “object state transmission process”. 13. A robot operation program that allows a robot to execute processing by being able to communicate with the video display terminal according to claim 12 and comprising a “user detection unit” that detects a user's behavior and a “storage unit” that stores user information. Then, based on at least one of past user information stored in the storage means and current user information acquired by the user detection means, one is determined from a plurality of behavior patterns linked to the object state and output. A robot operation program that executes a "robot state determination process" and a "robot state transmission process" for transmitting a determination result of the robot state determination means to a video display terminal. It is possible to communicate with a robot having “user detection means” for detecting user behavior, and includes “display means” for displaying an object and “display storage means” for storing a plurality of display patterns linked to the state of the robot. A robot operation program for causing the video display terminal to execute processing, based on “robot state reception processing” for receiving “robot state” including the robot action result from the robot, and the current user information received at the same time, An image display terminal operation program for executing an “object state determination process” for determining one of a plurality of display patterns associated with the state of the robot and displaying it by the display means. A video display terminal cooperation type robot system including a video display terminal and a robot,
The video display terminal is
Display means for displaying the object;
Input detection means for detecting an input to the display means;
Based on an input to the object detected by the input detection means, a transmission means for transmitting a plurality of behavior information to the robot;
With
The robot is
Storage means for storing information about the user;
Movable parts capable of multiple actions;
Receiving means for receiving the plurality of behavior information from the transmitting means;
Control means for controlling the movable part to perform one action among the plurality of action information received by the receiving means based on information on the user stored in the storage means;
Comprising
The video display terminal cooperation type robot system. The robot is
Sensing means;
Transmitting means for transmitting the sensing information obtained by the sensing means to the video display terminal after controlling the movable part to perform one of the plurality of behavior information received by the receiving means;
Further comprising
The video display terminal cooperation type robot system according to claim 15.