JP2019018336A - Device, method, program, and robot - Google Patents

Abstract

To ensure that an involved state is continued relative to a target person.SOLUTION: A processing section causes a robot: to execute any one of a first action, a second action, and a third action, as an initial action, in order for the robot to communicate with a target person according to an acquired image and acquired sound; to execute a one-level higher action after executing a current action containing the initial action, when sound is acquired by a microphone; to execute, when no sound is acquired by the microphone following execution of the current action, determination of whether elapsed time after the execution of the current action is less than a threshold; to continue the current action if the elapsed time is less than the threshold; and to execute a one-level lower action of the current action if the elapsed time is equal to or more than the threshold.SELECTED DRAWING: Figure 1

Description

  The present disclosure relates to a robot or the like that communicates with a person.

  2. Description of the Related Art Conventionally, robots whose main purpose is to engage with a person through communication with the person are known. In such a robot, it is important to keep the user engaged with the robot as much as possible.

  Patent Document 1 includes an autonomous task state in which a robot executes a task independent of user input, and a participating state in which the robot interacts with the user. Based on the current situation, the autonomous task state changes from the autonomous task state to the participating state. A technique for determining the transition timing and the transition timing from the participation state to the autonomous task state is disclosed.

Special table 2014-502565 gazette

  However, the above-mentioned prior art does not take into account the characteristics of infants who cannot see the surroundings when they are concentrated and are easily bored, so there is a problem that the state of involvement cannot be continued, and further improvement is necessary. There is.

An apparatus according to an aspect of the present disclosure is an apparatus that communicates with a target person by executing a predetermined action,
A camera for acquiring video around the device;
A microphone for acquiring sound around the device;
A processing unit;
Speakers,
A drive unit for moving the device,
The processor is
According to the acquired video and the acquired sound, the apparatus is caused to execute any one of a first action, a second action, and a third action as an initial action for communicating with the target person. , The second action is one action higher than the third action, the first action is one action higher than the second action,
If there is a sound acquired by the microphone after the current action including the initial action is executed, the apparatus executes the action one level higher than the current action,
If there is no sound acquired by the microphone since the current action was executed, determine whether the elapsed time since the current action was executed is less than a threshold,
If it is determined that the elapsed time is less than the threshold, the device continues the current action,
If it is determined that the elapsed time is greater than or equal to the threshold value, the device performs an action one level lower than the current action,
Let the device execute a predetermined task as the first action,
As the second action, the speaker is made to output a voice to be spoken to the target person, and the drive unit is controlled to cause the apparatus to make a movement synchronized with the movement of the target person as the third action.

  According to the above aspect, further improvement can be realized.

It is a block diagram showing an example of the whole robot composition in an embodiment of this indication. It is a figure which shows an example of an initial participating stage table. It is a figure which shows an example of a transfer table. It is a flowchart which shows an example of the determination process of a participating stage. FIG. 5 is a flowchart continued from FIG. 4. It is a flowchart which shows an example of the detailed process of S205 of FIG. It is a figure which shows an example of an interruption prohibition condition table. It is a figure which shows an example of a dictionary table. It is a figure which shows an example of the external appearance of a robot.

(Background to inventing one aspect of the present disclosure)
As described above, Patent Document 1 discloses a method for determining a transition timing between a participating state and a non-participating state other than the participating state (such as an autonomous task state and a resting state). Specifically, in Patent Document 1, it is disclosed that when a user pays attention to a robot and the user responds to a participation offer (for example, an inquiry) from the robot, the state shifts from a non-participation state to a participation state. Yes. Moreover, in patent document 1, there is no response from a user with respect to a plurality of questions after the conversation is not established in the engaged state, and when the standby state continues for a certain time, the state may shift to the non-engaged state. It is disclosed.

  By the way, the present inventor educates and trains the infant through some kind of work (task), such as quizing with the infant, encouraging the infant to clean up, or encouraging the infant to learn. We are researching robots that improve their ability. Here, infants have a psychological characteristic that they have strong self-centeredness, and while they are very curious, they tend to get bored and have difficulty in maintaining their concentration.

  Therefore, when an infant is targeted in Patent Document 1, it cannot be expected to gaze at the robot or respond to the question of the robot with certainty, and there is little opportunity for transition from the non-participating state to the participating state. Challenges arise. In addition, even if the child can move to an engaged state, the infant tends to immediately take an action that does not respond to the question from the robot or moves away from the robot. The problem of returning to the engagement state occurs. Thus, patent document 1 has the subject that a participating state cannot be maintained for a long time.

  Such a problem is not mentioned in the above-mentioned Patent Document 1, and is recognized as not existing in the past.

  Therefore, when the present inventor performs a certain task on the infant, imposing the task suddenly in a state where the infant is low is counterproductive, and the task is executed after sufficiently increasing the infant's awareness. It was found that this is effective. To that end, it was found that it is effective to gradually increase children's interest in robots while increasing or decreasing the degree of involvement with children.

  In order to solve the above problems, the present inventor examined the following improvement measures based on the above knowledge.

An apparatus according to an aspect of the present disclosure is an apparatus that communicates with a target person by executing a predetermined action,
A camera for acquiring video around the device;
A microphone for acquiring sound around the device;
A processing unit;
Speakers,
A drive unit for moving the device,
The processor is
According to the acquired video and the acquired sound, the apparatus is caused to execute any one of a first action, a second action, and a third action as an initial action for communicating with the target person. , The second action is one action higher than the third action, the first action is one action higher than the second action,
If there is a sound acquired by the microphone after the current action including the initial action is executed, the apparatus executes the action one level higher than the current action,
If there is no sound acquired by the microphone since the current action was executed, determine whether the elapsed time since the current action was executed is less than a threshold,
If it is determined that the elapsed time is less than the threshold, the device continues the current action,
If it is determined that the elapsed time is greater than or equal to the threshold value, the device performs an action one level lower than the current action,
Let the device execute a predetermined task as the first action,
As the second action, the speaker is made to output a voice to be spoken to the target person, and the drive unit is controlled to cause the apparatus to make a movement synchronized with the movement of the target person as the third action.

  In this aspect, the first, second, and third actions are prepared in descending order of the degree of involvement. First, any one of the first to third actions is determined from the surrounding video and sound conditions. The action is determined as the initial action. Here, the first action is an action for causing the target person to execute the task, the second action is an action to speak to the target person, and the third action is an action in synchronization with the movement of the target person. It can be said that the involvement in the target person is high in the order of the third action.

  When the microphone detects ambient sounds, the initial action is appropriate for the target person, and the initial action is shifted to one higher level action because the target person is highly interested in the device. On the other hand, if no sound is detected by the microphone until the elapsed time from the execution of the initial action reaches the threshold, the initial action is not appropriate for the target person's consciousness, and the target person's interest may be increased. It is determined that the action could not be performed, and the initial action is shifted to a lower action. Thereafter, in this manner, the interest of the target person is increased while the current action is gradually shifted, and the task is imposed on the target person in a state where the interest of the target person is sufficiently increased.

  Thus, since this aspect can impose a task on the target person after sufficiently increasing the interest of the target person in the apparatus, even if the target person is a person having psychological characteristics such as an infant, for example. The state of involvement of the device with the target person can be continued for a long time. As a result, the target person can be tasked for a long time, and the ability of the target person can be enhanced effectively.

In the above aspect, the one action lower than the third action is a fourth action,
The first action lower than the fourth action is the fifth action,
The processor is
If there is a sound acquired by the microphone since the current action was executed, and the voice of the target person included in the acquired sound was included in the dictionary provided in the device If the phrase is included, the device performs the fifth action,
Controlling the drive unit to cause the device to perform a predetermined movement at the current position of the device as the fourth action,
The apparatus may stop communication with the target person as the fifth action.

  In this aspect, a fourth action for causing the device to perform a predetermined movement at the current position and a fifth action having a lower degree of involvement than the fourth action and stopping communication with the target person are provided. Yes. When the target person speaks a phrase (for example, go there) included in the dictionary, the fifth action is executed, so the target person actively refuses to communicate with the apparatus, and In the situation where the interest is not expected to be improved, the apparatus is prevented from unnecessarily stimulating the target person, and the apparatus can be prevented from becoming annoying for the target person.

  In the above aspect, when the processing unit recognizes the target person from the acquired video and recognizes the voice of the target person from the acquired sound, the processing unit performs the initial action as to the device. The first action may be executed.

  In this aspect, for example, when the target person is speaking toward the device, the first action is executed on the assumption that the target person is sufficiently interested. Can be imposed.

  In the above aspect, when the processing unit does not recognize the target person from the acquired video and recognizes the voice of the target person from the acquired sound, the processing unit performs the initial action on the device. The second action may be executed as follows.

  In this aspect, for example, when the target person is uttering something without going to the device, the second action is executed by the device speaking to the target person. , Can increase the interest of the target person.

  In the above aspect, when the processing unit recognizes the target person from the acquired video and does not recognize the target person's voice from the acquired sound, the processing unit performs the initial action for the device. The third action may be executed.

  In this aspect, for example, when the target person does not go to the apparatus and does not speak, the third action that is synchronized with the movement of the target person is executed. Therefore, the target person is appropriately stimulated and interested in the target person. Can be increased.

  The said aspect WHEREIN: The said process part may output the audio | voice which proposes starting the communication with the said target person with respect to the said speaker as said 1st action.

  In this aspect, when performing the first action, a task can be requested through communication with the target person, and the task can be requested from the target person without a sense of incongruity.

In the above aspect, the processing unit includes:
When the inclination of the head of the target person is recognized from the acquired video,
The drive unit may be controlled to tilt the upper part of the apparatus at the same direction and angle as the head tilt as the third action.

  In this aspect, when performing the third action, the posture of the device is changed in conjunction with the change in the posture of the target person's head, so that the device is operating in synchronization with the movement of the target person. A person can be easily understood, and the interest of the target person in the device can be increased.

In the above aspect, the processing unit includes:
When recognizing an operation in accordance with a predetermined rhythm of the target person from the acquired video,
You may control the said drive part and can move the said apparatus according to the said rhythm as said 3rd action.

  In this aspect, when performing the third action, the apparatus moves in accordance with the movement rhythm of the target person, so that the target person can easily know that the apparatus is operating in synchronization with the movement of the target person. It is possible to increase the interest of the target person in the device.

In the above aspect, the processing unit includes:
As the second action, a sound including a name corresponding to the target person may be output to the speaker.

  In this aspect, when performing the second action, the apparatus talks to the target person using the name of the target person, so that the interest of the target person with respect to the apparatus can be increased.

In the above aspect, the processing unit includes:
As the fourth action, the device may be swung left and right.

  In this aspect, when performing the fourth action, the device is shaken to the left and right, so that the subject person's interest in the device can be drawn with a relatively small amount of stimulation in a situation where the subject person's interest is not so high.

In the above aspect, the processing unit includes:
As the fourth action, the device may be turned about the direction of gravity.

  In this aspect, when performing the fourth action, the device rotates on the spot, so that the subject person's interest in the device can be drawn with a relatively small amount of stimulation in a situation where the subject person's interest is not so high.

In the above aspect, the processing unit includes:
As the fifth action, the apparatus may be moved away from the target person.

  In this aspect, when the fifth action is performed, the apparatus moves away from the target person, and therefore it is possible to prevent unnecessary stimulation from being applied to the target person in a situation where improvement in the interest of the target person is not expected.

In the above aspect, the processing unit includes:
As the fifth action, the device may be rotated 180 degrees around the direction of gravity.

  In this aspect, since the apparatus is directed in the opposite direction with respect to the target person, it is possible to prevent unnecessary stimulation from being applied to the target person in a situation where the interest of the target person is not expected to improve.

In the above aspect, the processing unit includes:
When there is no sound acquired by the microphone since the current action is executed, and when a predetermined interrupt prohibition condition is set in the apparatus, the apparatus is caused to execute the fifth action,
The predetermined interrupt prohibition condition is
A condition regarding a predetermined time zone and a condition regarding the location of the target person may be included.

  It may be customary to perform a predetermined action (for example, meal and sleep) that makes it difficult for a target person to be involved in a robot at a predetermined place at a predetermined time. In this case, it is not preferable to involve the robot in the target person because it disturbs the life pattern. Therefore, in this aspect, an interrupt prohibition condition is provided, and the robot performs a fifth action in which the apparatus leaves the involvement of the target person in the time zone and place where the interrupt prohibition condition is set. This can prevent the robot from disturbing the life pattern of the target person.

  The present disclosure can also be realized as a computer program that causes a computer to execute each characteristic step included in such an apparatus. Needless to say, such a computer program can be distributed via a computer-readable non-transitory recording medium such as a CD-ROM or a communication network such as the Internet.

  Note that each of the embodiments described below shows a specific example of the present disclosure. Numerical values, shapes, components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present disclosure. In addition, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims indicating the highest concept are described as optional constituent elements. In all the embodiments, the contents can be combined.

(Embodiment)
(overall structure)
Hereinafter, embodiments of the present disclosure will be described. In the following description, a case where the apparatus of the present disclosure is applied to a robot will be described as an example.

  FIG. 1 is a block diagram illustrating an example of the overall configuration of the robot 1 according to the embodiment of the present disclosure. The robot 1 is a robot whose main purpose is to support the growth of an infant through communication with a user who has a psychological characteristic that the self-centeredness of the infant is strong, for example. The communication includes not only direct communication in which the infant and the robot 1 have a conversation by voice, but also indirect communication in which the robot 1 and the infant dance together.

  The robot 1 includes a sensor unit 100, an action execution unit 200, and an output unit 300. The sensor unit 100 includes a microphone 101 and a camera 102. The microphone 101 converts sounds around the robot 1 into audio signals. Further, the microphone 101 performs A / D conversion on the converted audio signal at a predetermined sampling rate, converts it to digital audio data, and outputs the digital audio data to the processor 210. The camera 102 acquires a video around the robot 1 and acquires image data. Here, the camera 102 is composed of, for example, a CCD or a CMOS image sensor, captures an image around the robot 1 at a predetermined frame rate (for example, 60 frames per second), and converts it into digital image data. The camera 102 may be configured with a normal camera that captures the front front of the robot 1 or may be configured with an omnidirectional camera that captures all directions of the robot 1. Image data captured by the camera 102 at a predetermined frame rate is input to the processor 210 at a predetermined frame rate. Here, the camera 102 may be configured by a stereo camera or an infrared camera. In this case, the image data captured by the camera 102 includes a distance component indicating the distance to the surrounding object.

  The action execution unit 200 includes a processor 210 and a memory 208. The processor 210 is composed of a processor such as an ASIC, DSP, or CPU, for example, and includes a voice recognition unit 201, an image recognition unit 202, an initial participation stage determination unit 203, a participation execution unit 204, a transition determination unit 205, and a command generation unit. 207. Here, each component included in the processor 210 may be realized by the processor executing a program that causes the computer stored in the memory 208 to function as the action execution unit 200, or may be configured by a dedicated hardware circuit. Also good. All the components constituting the action execution unit 200 may be mounted on the same terminal. Alternatively, all or some of the components constituting the action execution unit 200 are individually mounted on another terminal or server connected via an arbitrary network such as an optical fiber, a radio, or a public telephone line. Also good. In this case, the action execution unit 200 is realized by communicating with another terminal or server.

  The voice recognition unit 201 performs a predetermined voice recognition process on the voice data input from the microphone 101, recognizes the utterance content of the user existing around the robot 1 from the voice data, and determines the recognized utterance content. Convert to character string and generate utterance data. As the predetermined speech recognition processing, for example, a known method such as a hidden Markov model, a statistical method, or a dynamic time expansion / contraction method can be employed.

  In addition, the voice recognition unit 201 identifies the user who spoke by comparing the voice print of the voice data input from the microphone 101 with the voice print of a predetermined user stored in the memory 208. Then, the voice recognition unit 201 outputs data including the identified user identification information and utterance data as voice recognition data. The predetermined user includes, for example, an infant to be supported by the robot 1 as an educational support and a family of the infant. In the following, an explanation will be given by taking an infant whose robot 1 is a target of education support as an example of a target person.

  The image recognition unit 202 recognizes a user located around the robot 1 by applying face recognition processing to the image data input from the camera 102, and outputs the identification information of the recognized user as image recognition data. To do. In addition, the image recognition unit 202 detects the user's eye line, face direction, user movement amount, and the like recognized by the face recognition process, and outputs the detection result as image recognition data. Here, the image recognition process includes, for example, a process of extracting a feature amount of a person's face included in image data, and a feature amount of a predetermined user's face stored in the memory 208 in advance. Processing to compare is included.

  The initial participation stage determination unit 203 indicates an initial participation stage indicating an initial participation stage for the infant of the robot 1 based on the voice recognition data output from the voice recognition unit 201 and the image recognition data output from the image recognition unit 202. Determine the stage. FIG. 2 is a diagram illustrating an example of the initial participation stage table T1 used by the initial participation stage determination unit 203 to determine the initial participation stage. Note that the initial participation stage table T1 is stored in the memory 208 in advance.

  The initial participation stage table T1 is a database in which a plurality of initial participation stages and conditions for each initial participation stage are associated with each other. The fields of “recognition item”, “sensor”, “condition”, and “initial participation stage” Is provided.

  In the “initial participation stage” field, three participation stages of “task execution”, “imitation”, and “question” and “N / A” indicating that there is no corresponding participation stage are registered. The degree of involvement of the robot 1 with the infant is high in the order of “task execution”, “question”, and “imitation”. The “participation stage” refers to an index for determining an action when the robot 1 is involved with an infant.

  When the participation stage is “task execution”, the robot 1 executes an action that imposes a predetermined task on the infant (an example of the first action). The predetermined task refers to a work or task to be tackled by the infant, such as riddle play, tidying up, studying, homework, reading, reading a picture book by the robot 1, and helping. The task is preliminarily executed by the processor 210 prior to the determination of the initial participation stage based on at least one of the speech recognition data output from the speech recognition unit 201 and the image recognition data output from the image recognition unit 202. It is determined. In other words, the process of determining the initial participation stage is executed every time a new task is determined.

  For example, when the utterance data includes a message uttered by a mother who prompts an infant to clean up, “clean up” is determined as a task. For example, if the speech recognition data includes utterance data by a mother urging the user to clean up, such as “Let ’s clean up,” “cleanup” is determined as a predetermined task.

  In the “recognition item” field, recognition items of the robot 1 for each of the initial participation stages are registered. In the example of FIG. 2, “person recognition” and “voice recognition” are included as “recognition items” for all initial participation stages. “Human recognition” means that the robot 1 recognizes an infant by video. “Voice recognition” means that the robot 1 recognizes an infant by sound.

  In the “sensor” field, the type of sensor for realizing the recognition listed in the recognition item is registered. In the example of FIG. 2, “camera” is registered in “person recognition” and “microphone” is registered in “voice recognition” in all initial participation stages. That is, “person recognition” is executed using image data captured by the camera 102, and “voice recognition” is executed using sound data collected by the microphone 101.

  In the “condition” field, a determination condition for each initial participation stage is registered. For example, “task execution” is determined when both “person recognition” and “voice recognition” are “recognition”. “Imitation” is determined when “person recognition” is “recognition” and “speech recognition” is “none” (= non-recognition). “Question” is determined when “person recognition” is “none” and “voice recognition” is “recognition”. When both “person recognition” and “voice recognition” are “none”, the initial participation stage is not determined.

  Here, the determination of the initial participation stage is based on the idea that the degree of involvement in the robot 1 increases as the infant's interest in the robot 1 increases. The infant's interest in the robot 1 is that the infant is talking to the robot 1, the infant is talking without going to the robot 1, and the infant is going to the robot 1. It is considered higher in the order of the situation that has not.

  Therefore, in the initial participation stage table T1, the situation where both “person recognition” and “speech recognition” are “recognition” is considered to be in a stage where the degree of interest in the robot 1 is the highest, and the robot 1 “Task execution”, which is the participation stage with the highest degree of involvement with infants, is assigned. In addition, the situation where “person recognition” is “none” and “voice recognition” is “recognition” is considered that the infant's interest in the robot 1 is at the second highest stage, and the robot 1 is involved in the infant. “Question”, which is the second highest involved stage, is assigned. The situation where “person recognition” is “recognition” and “speech recognition” is “none” is considered that the infant's interest in the robot 1 is in the third highest stage, and the robot 1 is involved in the infant. “Imitation”, which is the third highest involved stage, is assigned.

  Returning to FIG. The participation execution unit 204 outputs a command output request for causing the robot 1 to execute an action corresponding to the initial participation stage determined by the initial participation stage determination unit 203 to the command generation unit 207. In addition, when the transition determination unit 205, which will be described later, determines the transition of the participating stage, the involvement executing unit 204 sends a command output request for causing the robot 1 to execute an action corresponding to the participating stage of the transition destination. It outputs to 207.

  Here, as shown in FIG. 3 to be described later, the participation stage is composed of five items of “task execution”, “question”, “imitation”, “waiting”, and “leaving”. Therefore, the involvement executing unit 204 outputs a command output request for causing the robot 1 to execute the five actions corresponding to these five participating stages to the command generating unit 207. The actions corresponding to “task execution”, “question”, “imitation”, “waiting”, and “leaving” are the first action, the second action, the third action, the fourth action, and the fifth action, respectively. It corresponds to an example of action.

  For example, if the determined participation stage is “task execution”, an output request for a “task execution” command is output to the command generation unit 207, and if the determined participation stage is “imitation”, “imitation”. The command output request is output to the command generation unit 207.

  The transition determination unit 205 determines whether or not to transfer the initial participation stage determined by the initial participation stage determination unit 203 to another participation stage. In addition, the transition determination unit 205 determines whether or not to shift the participation stage after shifting the initial participation stage to another participation stage. Hereinafter, the participation stage currently set in the robot 1 including the initial participation stage and the participation stage after the transition from the initial participation stage is described as a “current participation stage”.

  Here, the transition determination unit 205 shifts the current participation stage to a higher participation stage when the voice recognition unit 201 recognizes the voice of the infant after the action corresponding to the current participation stage is executed. Judge that. That is, as a result of causing the robot 1 to execute the current participation stage, the transition determination unit 205 determines that the infant's interest in the robot 1 tends to increase when the infant responds by utterance. Is moved to a higher participation stage.

  On the other hand, if the voice recognition unit 201 does not recognize an infant's voice after the action corresponding to the current participation stage (current action) is executed, the transition determination unit 205 executes the current action. It is determined whether the elapsed time is less than the threshold value. If the transition determination unit 205 determines that the elapsed time is less than the threshold, the transition determination unit 205 determines to continue the current participation stage. On the other hand, if the infant does not speak even when the elapsed time is equal to or greater than the threshold, the transition determination unit 205 determines that the current participation stage is shifted to a lower participation stage. That is, if the transition determination unit 205 waits until the elapsed time reaches the threshold value and there is no response from the utterance from the infant, the current participation stage is not suitable for the infant, and instead the infant's robot 1 Judge that it will reduce interest, and move on to the next stage of involvement.

  FIG. 3 is a diagram illustrating an example of the transition table T2 that determines the order of the participating stages of the robot 1. In the transition table T2 shown in FIG. 3, five participation stages of “task execution”, “question”, “imitation”, “waiting”, and “leaving” are registered in descending order of the degree of involvement of the robot 1 with the infant. ing. For example, the transition determination unit 205 sets the participation stage step by step according to the order registered in the transition table T2, and raises or lowers the degree of involvement of the robot 1 with the infant. Note that the migration table T2 is stored in the memory 208 in advance.

  When the command generation unit 207 receives a command output request from the participation execution unit 204, the command generation unit 207 outputs the command indicated by the output request to the output unit 300.

  The output unit 300 is a component that causes the robot 1 to realize an action corresponding to the current participation stage in accordance with a command from the command generation unit 207, and includes a speaker 301 and a drive unit 302.

  The speaker 301 converts voice data necessary for executing an action corresponding to the current participation stage into voice according to the command from the command generation unit 207 and outputs the voice to the outside.

  The drive unit 302 includes, for example, an actuator such as a motor and a mechanism unit that is operated by the actuator, and causes the robot 1 to realize an action corresponding to the current participating stage in accordance with a command from the command generation unit 207. The mechanism unit includes a member that moves the robot 1 forward or backward, a member that changes the posture of the robot 1, a member that changes the orientation of the display unit that displays the facial expression of the robot 1, and the like. The actuator includes a motor that drives a member that moves the robot 1 forward or backward, a motor that drives a member that changes the posture of the robot 1, a motor that changes the orientation of the display unit, and the like.

  When the action execution unit 200 is mounted on a terminal or server different from the main body of the robot 1, the speaker 301 and the drive unit 302 are connected to the terminal on which the action execution unit 200 is mounted via a wired or wireless connection. What is necessary is just to connect with a server.

(flowchart)
Next, the involved stage determination process in the robot 1 will be described with reference to FIGS. FIG. 4 is a flowchart illustrating an example of the involved stage determination process. FIG. 5 is a flowchart subsequent to FIG.

  First, the processor 210 activates the robot 1 when the voice recognition unit is powered on (S101). Next, the initial participation stage determination unit 203 detects the presence / absence of a sensor input based on the voice recognition data by the voice recognition unit 201 and the image recognition data by the image recognition unit 202 (S102). Here, the initial participation stage determination unit 203 determines that there is no sensor input when the speech recognition unit 201 does not output the speech recognition data about the infant and the image recognition unit 202 does not output the image recognition data about the infant. When at least one of the voice recognition data regarding the infant and the image recognition data regarding the infant is output, it may be determined that there is a sensor input.

  If it is determined in S102 that there is a sensor input (YES in S102), the process proceeds to S103. If it is determined that there is no sensor input (NO in S102), the process returns to S102.

  In S103, the initial participation stage determination unit 203 refers to the initial participation stage table T1 stored in the memory 208, and determines the initial participation stage of the robot 1 (S103). Here, the initial participation stage determination unit 203 determines the participation stage that matches the condition item as the initial participation stage of the robot 1.

  For example, when the speech recognition unit 201 recognizes the speech of the infant and the image recognition unit 202 recognizes the image of the infant, “task execution” is determined as the initial participation stage. Further, when the speech recognition unit 201 does not recognize the speech of the infant and the image recognition unit 202 recognizes the image of the infant, “imitation” is determined as the initial participation stage. In addition, when the speech recognition unit 201 recognizes the speech of the infant and the image recognition unit 202 does not recognize the infant image, “question” is determined as the initial participation stage.

  Next, referring to FIG. 5, in S202, when the current participation stage is not “leave” (NO in 202), the participation execution unit 204 outputs a command for executing an action corresponding to the current participation stage. The request is output to the command generation unit 207 (S203). Here, since there are three initial participation stages, “task execution”, “question”, and “imitation”, first, an action corresponding to one of these three participation stages is executed.

  As a task, for example, when riddle play is executed, the processor 210 causes the speaker 301 to output a voice suggesting the start of communication such as “Let's play riddle play” to the infant, and a riddle problem. A command for outputting a voice to read out from the speaker 301 may be output to the output unit 300. In addition, the processor 210 reads the riddle problem, and if the speech recognition unit 201 recognizes the speech of the answer from the infant, the processor 210 compares the content of the speech with the riddle answer. A command for outputting the indicated voice from the speaker 301 may be output to the output unit 300.

  Further, when cleanup is executed as a task, the processor 210 outputs, to the output unit 300, a command for outputting a voice asking “let's clean up the room” from the speaker 301, and then the recognition result of the image recognition unit 202. From this, it is determined whether or not the infant is cleaning up. If it is determined that the baby is cleaning up, a command for outputting a voice that compliments the infant's action from the speaker 301 may be output to the output unit 300. On the other hand, if the processor 210 determines that the infant has not been cleared, the processor 210 may output a command to the output unit 300 to output a sound prompting the infant to be cleaned from the speaker 301.

  The action corresponding to the “question” participation stage corresponds to an action in which the robot 1 speaks to the infant by calling the infant's name, for example.

  The action corresponding to the “imitation” participation stage corresponds to an action in which the robot 1 moves in synchronization with the infant, that is, an action that imitates the movement of the infant. When the robot 1 performs an action corresponding to the “imitation” participation stage, the processor 210 indicates that, for example, from the recognition result of the image recognition unit 202, the infant is concentrating and performing some work. If detected, a command to tilt the upper part of the robot 1 in the same direction as the tilt of the infant's head may be output to the output unit 300. Here, the same direction assumes that the robot 1 and the infant are facing each other. For example, if the infant tilts the head to the left, the robot 1 tilts the upper part to the right. If the infant tilts his / her head to the right, the robot 1 tilts the upper part to the left.

  Further, for example, when the processor 210 detects from the recognition result of the image recognition unit 202 that the infant is performing an operation in accordance with a predetermined rhythm like a dance, the processor 210 performs the dance in accordance with the predetermined rhythm. The command to be output may be output to the output unit 300.

  On the other hand, when the current participation stage is “leave” in S202 (YES in S202), the participation execution unit 204 issues a command output request for executing a withdrawal action that causes the robot 1 to leave the infant. The data is output to the generation unit 207, and the robot 1 is caused to perform a withdrawal action (S207), and the process is terminated.

  Here, the withdrawal action refers to an action in which the robot 1 leaves the infant. For example, as the separation action, the robot 1 is moved to a position away from the infant by a predetermined distance, or the robot 1 is turned 180 degrees around the direction of gravity as the axis of gravity to move the front of the robot 1 in the direction opposite to the infant's face. An autonomous action in which the robot 1 stops communication with the infant, such as pointing it, corresponds. Thereby, in the state where the infant's consciousness is so low that the interest in the robot 1 is not expected to increase, the infant can be prevented from unnecessarily stimulating the robot 1 from feeling disgusted.

  In S204, if the current participation stage is “task execution” and the task is completed (NO in S204), the participation execution unit 204 sends a command output request to the robot 1 to execute a withdrawal action. In step S207, the robot 1 is caused to execute a withdrawal action (S207), and the process ends. Thereby, even after the task is completed, it is possible to prevent the robot 1 from staying on the infant and causing the infant 1 to feel disgusted with the robot 1.

  On the other hand, when the current participation stage is “task execution” and the task has not ended (NO in S204), or when the current participation stage is “question”, “imitation”, or “waiting” ( In step S204, NO is determined, and the transition determination unit 205 performs determination processing for transition of the involved stage (S205).

  Note that a task is terminated when a termination condition set for each task is satisfied. For example, if the task is a task for answering an infant's question, the task is terminated if there is no question from the infant until a predetermined time elapses after the robot 1 answers the infant's question. If the task is riddle play, the process is terminated when the infant answers the riddle that the robot 1 has given a question a predetermined number of times or when there is no utterance from the infant until a certain time has passed since the question was given.

  Next, the participation executing unit 204 outputs a command output request for causing the robot 1 to execute an action corresponding to the participating stage determined by the processing of S205 to the command generating unit 207 (S206). As a result, the robot 1 executes an action corresponding to the participation stage determined by the process of S205. The participation stage determined by the process of S205 is temporarily stored in the memory 208.

  Next, the migration determination process shown in S205 of FIG. 5 will be described. FIG. 6 is a flowchart showing an example of detailed processing in S205 of FIG.

  First, when it is determined that there is no voice response of the infant (NO in S301), the transition determination unit 205 checks whether or not the interrupt is prohibited (S302). FIG. 7 is a diagram illustrating an example of an interrupt prohibition condition table T3 in which interrupt prohibition conditions are set.

  If it is customary to perform an action that would be troublesome if the infant is involved in the robot 1 at a predetermined place at a predetermined time, if the robot 1 is involved with an infant at the predetermined place at the predetermined time, Will disturb the life pattern. For example, when it is customary for an infant to have breakfast in dining in the morning time zone, if the robot 1 is involved with the infant in this morning time zone, breakfast will be hindered. Therefore, in the present embodiment, an interrupt prohibition condition is provided, and in the time zone and place where the interrupt prohibition condition is set, the robot 1 performs a withdrawal action to prevent the robot 1 from disturbing the infant's life pattern. ing.

  The interrupt prohibition condition table T3 is a two-dimensional table in which a vertical axis indicates a time zone and a horizontal axis indicates a location. In the “time zone” field, a time zone in which the daily time zone is divided into a plurality of times is registered. Here, the time zone of the day is divided into seven time zones, such as 7 o'clock to 9 o'clock, 9 o'clock to 12 o'clock, ..., 21 o'clock to 7 o'clock. In the “place” field, the room name of the infant's house is registered. In each cell of the interrupt prohibition condition table T3, data “ON” indicating that the interrupt prohibition condition is set and data “OFF” indicating that the interrupt prohibition condition is not set are registered.

  For example, “ON” is registered in the dining cell in the time zone from 7:00 to 9:00. Accordingly, when the transition determination unit 205 detects that the infant is in the dining room during this time period, the transition determination unit 205 determines the participation stage to “leave”. This is because, in this family, for example, it is customary for an infant to have breakfast at a dining time from 7:00 to 9:00, and if the robot 1 is involved in the infant, it will interfere with breakfast. .

  On the other hand, for example, “OFF” is registered in the cells other than the dining in the time zone from 7:00 to 9:00. Therefore, the transition determination unit 205 performs the normal operation in which the robot 1 is allowed to perform actions other than the withdrawal action if the infant is in a place other than the dining room even during this time period.

  The interrupt prohibition condition is set in advance based on data input by the user to a mobile terminal such as a smartphone, for example. For example, the user may set the interrupt prohibition condition by speaking to the mobile terminal. Thereby, when it is in the interruption prohibition state, the robot 1 can be prevented from participating in the infant.

  Returning to FIG. In S302, the transition determination unit 205 refers to the interrupt prohibition condition table T3, and determines that the interrupt is disabled (YES in S302), shifts the participating stage to “leave” (S305). On the other hand, if it is determined that the interrupt is not prohibited (NO in S302), the process proceeds to S303. Here, the transition determination unit 205 determines the room where the infant is present from the recognition result of the image recognition unit 202, and “ON” is registered in the cell corresponding to the determined room and the time zone to which the current time belongs. Then, it is determined that the interrupt is prohibited (YES in S302). If “OFF” is registered in the cell, it is determined that the interrupt is not disabled (NO in S302).

  In S303, the transition determination unit 205 determines whether or not the elapsed time since the execution of the action corresponding to the current participation stage is less than the threshold value. If the elapsed time is less than the threshold (YES in S303), the transition determination unit 205 maintains the current participation stage (S307). On the other hand, if the elapsed time is equal to or greater than the threshold value (NO in S303), the transition determination unit 205 refers to the transition table T2 and shifts the current participation stage to a lower participation stage (S308). As the threshold value, for example, a predetermined time in which it is expected that the interest of the infant will not increase even if the robot 1 is caused to perform an action corresponding to the same participating stage is adopted. Values such as 2 minutes, 3 minutes, 5 minutes, and 10 minutes can be employed.

  In FIG. 3, for example, when the current participation stage is “imitation” and it is determined that the participation stage moves down one level in this state, the participation stage is set to “standby”. Note that the action corresponding to the “standby” participation stage corresponds to an action of swinging the robot 1 left and right on the spot, or an action of turning (spinning) the robot 1 on the spot around the gravity direction. Thereby, in the situation where the infant's interest is low, the infant's interest in the robot 1 can be drawn with relatively little stimulation.

  In S301, when it is determined that the infant has a voice response (YES in S301), the transition determination unit 205 determines whether or not the utterance content of the infant has hit the rejection term dictionary (S304). FIG. 8 is a diagram illustrating an example of a dictionary table T4 in which rejection term dictionaries are registered. In the dictionary table T4, terms that reject the robot 1, such as “get it right”, “don't talk”, and “noisy” are registered. When the utterance content of the infant includes any term registered in the dictionary table T4 (YES in S304), the transition determination unit 205 determines that the infant is actively refusing to involve the robot 1. The current participation stage is shifted to “leave” (S305).

  On the other hand, when the utterance content of the infant is registered in the dictionary table T4 and does not include any term (NO in S304), the transition determination unit 205 shifts the current participation stage to a higher participation stage (S306). . Referring to FIG. 3, for example, if the current participation stage is “imitation” and it is determined that the participation stage is to be raised one level in this state, the participation stage is shifted to “question”.

  When the processes of S305, S306, S307, and S308 are completed, the process returns to S206 in FIG. 5, and after the participation stage is shifted, the process proceeds to S202 in FIG. The processes of S202 to S206 are repeated until.

(robot)
Next, the mechanism of the robot 1 will be described. FIG. 9 is a diagram illustrating an example of the appearance of the robot 1. The robot 1 includes a spherical main casing 401 and crown portions 402 and 403, and the main casing 401 and the crown portions 402 and 403 constitute a sphere as a whole. That is, the robot 1 has a spherical shape. In addition, the robot 1 includes a microphone 101, a camera 102, and a speaker 301 in a spherical crown 402 (or a spherical crown 403). The robot 1 includes a control circuit (not shown). The action execution unit 200 shown in FIG. 1 is implemented in this control circuit. In the example of FIG. 9, the camera 102 is configured by a stereo camera including two cameras provided in each of the spherical crown portion 402 and the spherical crown portion 403, and acquires the surrounding environment image and distance data.

  The center of the spherical crown portion 402 and the center of the spherical crown portion 403 are fixedly connected by a shaft (not shown) provided inside the main housing 401. The main casing 401 is rotatably attached to the shaft. A frame (not shown) and a display unit (not shown) are attached to the shaft. A first motor (not shown) for rotating the main casing 401 is attached to the frame. As the first motor (not shown) rotates, the main casing 401 rotates with respect to the crown portions 402 and 403, and the robot 1 moves forward or backward. When the robot 1 moves forward or backward, the crown portions 402 and 403 are in a stopped state, so that the speaker 301 and the camera 102 are maintained facing the front of the robot 1. Further, an image showing the eyes and mouth of the robot 1 is displayed on the display unit. The display unit is attached so that the angle with respect to the shaft can be adjusted by power from a second motor (not shown). Therefore, the direction of the eyes and mouth of the robot is adjusted by adjusting the angle of the display unit with respect to the shaft. Since the display unit is attached to the shaft independently of the main casing 401, the angle with respect to the shaft does not change even when the main casing 401 rotates. Therefore, the robot 1 can move forward or backward with the direction of eyes and mouth fixed.

  Further, a weight (not shown) is lowered on the shaft. This weight is attached so as to be swingable about the front direction of the robot 1 by the power of a third motor (not shown). Therefore, by swinging the weight, the robot 1 can be operated to swing on the spot. Further, the robot 1 can be turned leftward or rightward by moving the robot 1 forward in a state where the weight is tilted leftward or rightward when viewed from the rear to the front. For example, the robot 1 can change the direction of gravity by repeating the operation of moving the robot 1 forward with the weight tilted to the left and the operation of moving the robot 1 backward with the weight tilted to the right. A turn (spot turn) can be performed on the shaft.

  The main casing 401, the first motor, the second motor, the third motor, and the display unit described above constitute the drive unit 302 in FIG.

  The control circuit controls various operations of the robot 1. The robot 1 forms a sphere as a whole. However, the present invention is not limited to this, and it is sufficient that the robot 1 has at least a moving mechanism.

  As described above, in the present embodiment, by gradually increasing the degree of involvement of the robot 1 with the infant, the infant's interest in the robot 1 is gradually increased, and the infant's interest is sufficiently increased. Tasks can be imposed on infants in a standing state. Therefore, in the present disclosure, the robot 1 can be involved for a long time with an infant having a psychological characteristic that makes it difficult to maintain concentration. As a result, the child can be tasked for a long time, and the child's ability can be effectively enhanced.

  The present disclosure can employ the following modifications.

  (1) In the above-described embodiment, an example in which the device according to the present disclosure is applied to the robot 1 has been shown. May be.

  (2) In the above embodiment, “leave” and “standby” are included as the participation stages, but these participation stages may be omitted. In this case, the robot 1 takes an action corresponding to one of the three participating stages of “task execution”, “question”, and “imitation”.

  (3) In the flowchart of FIG. 6, there are provided a process for determining the presence or absence of an interrupt disabled state (S302) and a process for determining the presence or absence of a hit in the rejection term dictionary (S304). Yes, either or both of the processes may be omitted.

  (4) Although the flowchart of FIG. 5 is started when the power is turned on, this is an example, and it may be started when a task to be imposed on the infant has occurred.

  According to the present disclosure, it is useful as an educational robot because the robot's involvement state can be maintained for a target person who is easily bored and difficult to get involved.

DESCRIPTION OF SYMBOLS 1 Robot 100 Sensor part 101 Microphone 102 Camera 200 Action execution part 201 Voice recognition part 202 Image recognition part 203 Initial participation stage determination part 204 Involvement execution part 205 Transition determination part 207 Command generation part 208 Memory 210 Processor 300 Output part 301 Speaker 302 Drive Part T1 Initial participation stage table T2 Transition table T3 Interrupt disable condition table T4 Dictionary table

Claims (17)

A device that communicates with a target person by executing a predetermined action,
A camera for acquiring video around the device;
A microphone for acquiring sound around the device;
A processing unit;
Speakers,
A drive unit for moving the device,
The processor is
According to the acquired video and the acquired sound, the apparatus is caused to execute any one of a first action, a second action, and a third action as an initial action for communicating with the target person. , The second action is one action higher than the third action, the first action is one action higher than the second action,
If there is a sound acquired by the microphone after the current action including the initial action is executed, the apparatus executes the action one level higher than the current action,
If there is no sound acquired by the microphone since the current action was executed, determine whether the elapsed time since the current action was executed is less than a threshold,
If it is determined that the elapsed time is less than the threshold, the device continues the current action,
If it is determined that the elapsed time is greater than or equal to the threshold value, the device performs an action one level lower than the current action,
Let the device execute a predetermined task as the first action,
Causing the speaker to output a voice to be spoken to the target person as the second action, and controlling the drive unit to cause the device to perform a movement in synchronization with the movement of the target person as the third action;
apparatus. The first action lower than the third action is the fourth action,
The first action lower than the fourth action is the fifth action,
The processor is
If there is a sound acquired by the microphone since the current action was executed, and the voice of the target person included in the acquired sound was included in the dictionary provided in the device If the phrase is included, the device performs the fifth action,
Controlling the drive unit to cause the device to perform a predetermined movement at the current position of the device as the fourth action,
Causing the device to stop communication with the target person as the fifth action,
The apparatus of claim 1. When the processing unit recognizes the target person from the acquired video and recognizes the voice of the target person from the acquired sound, the processing unit performs the first action as the initial action with respect to the device. To execute,
The apparatus of claim 1. When the processing unit does not recognize the target person from the acquired video and recognizes the voice of the target person from the acquired sound, the processing unit performs the second action as the initial action with respect to the device. Take action,
The apparatus of claim 1. When the processing unit recognizes the target person from the acquired video and does not recognize the voice of the target person from the acquired sound, the processing unit performs the third action as the initial action for the device. To execute,
The apparatus of claim 1. The processing unit, as the first action, causes the speaker to output a sound proposing to start communication with the target person,
The apparatus according to claim 3. The processor is
When the inclination of the head of the target person is recognized from the acquired video,
Controlling the drive unit to tilt the upper part of the device in the same direction and angle as the tilt of the head, as the third action,
The apparatus of claim 5. The processor is
When recognizing an operation in accordance with a predetermined rhythm of the target person from the acquired video,
By controlling the drive unit, as the third action, the device can be moved according to the rhythm,
The apparatus of claim 5. The processor is
As the second action, a sound including a name corresponding to the target person is output to the speaker.
The apparatus of claim 4. The processor is
As the fourth action, the device is swung left and right,
The apparatus of claim 2. The processor is
As the fourth action, the device is turned about the direction of gravity,
The apparatus of claim 2. The processor is
As the fifth action, the device is moved away from the target person.
The apparatus of claim 2. The processor is
As the fifth action, the device is turned 180 degrees around the direction of gravity.
The apparatus of claim 2. The processor is
When there is no sound acquired by the microphone since the current action is executed, and when a predetermined interrupt prohibition condition is set in the apparatus, the apparatus is caused to execute the fifth action,
The predetermined interrupt prohibition condition is
Including a condition for a predetermined time zone and a condition for the location of the target person,
The apparatus of claim 2. A method in an apparatus for communicating with a target person by executing a predetermined action,
The video around the device is acquired by the camera,
The sound around the device is acquired by a microphone,
According to the acquired video and the acquired sound, the apparatus is caused to execute any one of a first action, a second action, and a third action as an initial action for communicating with the target person. , The second action is one action higher than the third action, the first action is one action higher than the second action,
If there is a sound acquired by the microphone after the current action including the initial action is executed, the apparatus executes the action one level higher than the current action,
If there is no sound acquired by the microphone since the current action was executed, determine whether the elapsed time since the current action was executed is less than a threshold,
If it is determined that the elapsed time is less than the threshold, the device continues the current action,
If it is determined that the elapsed time is equal to or greater than the threshold, the device is caused to execute an action one level lower than the current action,
Let the device execute a predetermined task as the first action,
As a second action, the speaker outputs a voice to be spoken to the target person, and controls the drive unit to cause the device to make a movement synchronized with the movement of the target person as the third action.
Method.   A program for causing a computer to execute the method according to claim 15. A robot that communicates with a target person by executing a predetermined action,
A camera that acquires video around the robot;
A microphone for obtaining sounds around the robot;
A processing unit;
Speakers,
A drive unit for moving the robot,
The processor is
According to the acquired video and the acquired sound, the robot is caused to execute, as an initial action, any one of a first action, a second action, and a third action for communicating with the target person. , The second action is one action higher than the third action, the first action is one action higher than the second action,
If there is a sound acquired by the microphone since the current action including the initial action is executed, the robot is caused to execute an action one level higher than the current action,
If there is no sound acquired by the microphone since the current action was executed, determine whether the elapsed time since the current action was executed is less than a threshold,
If it is determined that the elapsed time is less than the threshold, the robot continues the current action,
When it is determined that the elapsed time is equal to or greater than the threshold, the robot is caused to execute an action that is one subordinate to the current action,
Let the robot execute a predetermined task as the first action,
Causing the speaker to output a voice to be spoken to the target person as the second action, and controlling the drive unit to cause the robot to move in synchronization with the movement of the target person as the third action;
robot.

Images