JP2019184813A - Robot and robot control program - Google Patents

Abstract

To provide a robot which enables conversation adjusted to a care receiver.SOLUTION: A robot 1 comprises: a speaker specification section 15 for specifying a care receiver; a parameter storage section 16 for storing a parameter at every care receiver; a parameter acquisition section 17 for acquiring the parameter of the specified care receiver; a conversation selection section 20 for selecting conversation to the care receiver; a voice synthesis section 21 for outputting the conversation by voice through a speaker S; and an operation command section 23 for giving a command of presentation operation which is previously set to a driving mechanism D.SELECTED DRAWING: Figure 2

Description

  The present invention relates to a robot and a robot control program that have a conversation with a person such as an elderly person or a care recipient.

  In the care field, robots are being introduced to reduce the burden on caregivers. Conventionally, a robot that automatically and actively provides personal assistance based on detection data related to a person has been proposed (see Patent Document 1). The robot described in Patent Document 1 performs personal assistance related to human health, exercise, or diet activity.

JP 2014-176963 A

  However, the robot described in Patent Document 1 cannot perform conversations according to persons such as elderly people and care recipients. As a result, the conversation with the robot is not always easy for the person to hear, and the conversation may be interrupted, and personal assistance may not be sufficiently provided.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a robot and a robot control program that enable conversation according to a person.

  In view of the above-described problems, a robot according to the present invention is a robot including a microphone to which a person's utterance is input, a camera that captures the person, and a speaker that outputs sound to the person. A question selection unit that selects a question for the voice, a voice output unit that outputs the question selected by the question selection unit by voice via the speaker, and a voice of the person's response to the question that is input from the microphone The person is identified based on the voice recognition unit that recognizes, the face image recognition unit that recognizes the face image of the person photographed by the camera, and the voice recognition result of the response or the image recognition result of the face image A person specifying unit that performs high frequency emphasis information that represents a high frequency side frequency component of the voice to be emphasized, a question replacement rule that replaces the content of the question, and a story A parameter storage unit for storing parameters, and a parameter acquisition unit for acquiring parameters from the parameter storage unit for the person specified by the person specifying unit, wherein the voice output unit uses the parameters acquired by the parameter acquisition unit as parameters Accordingly, the content of the question is replaced, and the question is output at a frequency and a speech speed corresponding to the parameter.

  In view of the above-described problems, the robot control program according to the present invention is configured to cause a computer to function as the robot according to the present invention.

  According to the robot and the robot control program according to the present invention, words and sentences that are difficult for a person such as an elderly person or a cared person to replace are replaced with each other by using a frequency and a speech speed adapted to the person. Can have a conversation.

1 is an external view of a robot according to an embodiment of the present invention. It is a block diagram which shows the structure of the robot of FIG. In embodiment, (a)-(f) is explanatory drawing explaining the definition of the movable range of an upper arm. It is a flowchart which shows operation | movement of the robot of FIG. It is a flowchart which shows the speaker recognition process of FIG. It is a flowchart which shows the operation | movement analysis process of FIG. It is a flowchart which shows the operation | movement analysis process of FIG. It is a flowchart which shows the singing process of FIG. In an embodiment, it is a flow chart which shows voice analysis processing.

(Embodiment)
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings as appropriate. In the embodiment, the same means is denoted by the same reference numeral, and description thereof is omitted.

[Robot outline]
The outline of the robot 1 according to the embodiment of the present invention will be described with reference to FIG.
As shown in FIG. 1, the robot 1 is a human-type nursing robot that is used at a nursing care site and is opposed to an elderly care recipient. Specifically, the robot 1 has a conversation with a cared person on behalf of the caregiver, and performs singing and gymnastics with the cared person.

  Here, an elderly care receiver may have difficulty in hearing a specific frequency sound or a quick-speaking sound. In nursing facilities and rehabilitation hospitals, singing and gymnastics such as nursery rhymes are performed as part of rehabilitation, but the cared person often has difficulty in singing and gymnastics tailored to the carer. In such a case, the caregiver performs singing and gymnastics according to the care recipient. Therefore, the robot 1 performs conversation, singing, and gymnastics according to the cared person, like a human carer.

The robot 1 includes a camera C that captures the cared person, a microphone M that acquires the cared person's voice, a speaker S that outputs voice and a song to the cared person, and a control unit 10 that performs various controls of the robot 1. (FIG. 2). Two cameras C are attached to the front of the head of the robot 1. Two microphones M are attached before and after the top of the robot 1. Two speakers S are attached to the left and right of the head of the robot 1. The robot 1 has movable parts such as a neck, a shoulder, an elbow, a hip joint, a knee, and an ankle, and drives the movable part by a drive mechanism D such as a servo motor.
In FIG. 1, only one microphone M and one speaker S are shown.

[Robot configuration]
The configuration of the robot 1 will be described with reference to FIG.
The camera C is a general camera that captures a face image of a cared person and a captured image of the whole body. In the present embodiment, the camera C captures a face image that includes the face area of the care recipient and outputs the captured face image to the face image recognition unit 11. In addition, the camera C captures a captured image that includes the whole body of the care recipient, and outputs the captured image to the motion analysis unit 12.
The microphone M is a general microphone that acquires the care receiver's voice. The microphone M outputs the acquired voice to the utterance recognition unit 13 and the voice analysis unit 14.
The speaker S is a general speaker that outputs synthesized speech from the speech synthesizer 21 to a care recipient.
The drive mechanism D drives each movable part of the robot 1 in accordance with a command from the operation command unit 23. For example, the drive mechanism D can be a general servo motor.

  The control unit 10 includes a face image recognition unit 11, a motion analysis unit 12, an utterance recognition unit (voice recognition unit) 13, a voice analysis unit 14, a speaker identification unit (person identification unit) 15, and a parameter storage unit. 16, a parameter acquisition unit 17, a conversation analysis unit 18, a conversation storage unit 19, a conversation selection unit (question selection unit) 20, a speech synthesis unit (speech output unit) 21, a presentation motion storage unit 22, An operation command unit 23.

  The face image recognition unit 11 recognizes a face image input from the camera C. In the present embodiment, the face image recognition unit 11 extracts feature points of the face image by a known method, and outputs the extracted feature points to the speaker specifying unit 15 as an image recognition result.

Here, as a method for recognizing a face image, the face authentication technique described in Reference 1 can be used. This face authentication technique is roughly divided into two processes, a face detection process and a face matching process. In the face detection process, a face region is determined from the face image, and then the face feature points are detected to obtain the positions of the face feature points such as eyes, nose, and mouth edge. Further, after the position and size of the face area are normalized using the position of the face feature point, face matching processing is performed.
Reference 1: "Face authentication mechanism", [online], [March 30, 2018 search], Internet <URL: https://jpn.nec.com/biometrics/face/technology/structure.html>

The motion analysis unit 12 analyzes the follow-up motion of the care recipient using the captured image input from the camera C. For example, the motion analysis unit 12 analyzes a care receiver's motion part, motion amount, motion start time, motion duration, and the like by a known method such as a motion vector. Then, the motion analysis unit 12 outputs the motion analysis result to the speaker specifying unit 15, the conversation selection unit 20, and the motion command unit 23.
The following operation is an operation performed by the care recipient following the operation of the robot 1.

  The utterance recognition unit 13 recognizes voice of a cared person (for example, a cared person's response to a question) input from the microphone M. In the present embodiment, the utterance recognition unit 13 recognizes a word included in the sound from the microphone M by a known method, and outputs the word recognition result to the speaker identification unit 15 and the conversation analysis unit 18.

  The voice analysis unit 14 analyzes the care receiver's voice and singing input from the microphone M. In the present embodiment, the voice analysis unit 14 analyzes acoustic characteristics such as phoneme, phoneme, speech rate, etc., for the voice from the microphone M by a known method (Reference Document 2). Then, the speech analysis unit 14 outputs the acoustic characteristic analysis result to the speaker identification unit 15, the parameter acquisition unit 17, and the speech synthesis unit 21.

  Reference 2: Hiroya Fujisaki, “Prosody Analysis, Formulation and Modeling”, [online], [March 30, 2018 search], Internet <URL: http: //www.gavo.tu-tokyo .ac.jp / tokutei_pub / houkoku / model / model.pdf # search =% 27% E9% 9F% B3% E9% 9F% BB% E3% 80% 81% E9% 9F% B3% E7% B4% A0% E3% 80% 81% E7% 99% BA% E8% A9% B1% E9% 80% 9F% E5% BA% A6 +% E8% A7% A3% E6% 9E% 90% 27>

  The speaker specifying unit 15 specifies a care receiver based on the analysis results of the face image recognition unit 11, the motion analysis unit 12, the speech recognition unit 13, and the voice analysis unit 14. Here, the speaker specifying unit 15 can adopt an appropriate speaker specifying method in consideration of the ability of the arithmetic unit of the robot 1 and the number of care recipients. For example, examples of the speaker specifying method include deep learning, pattern matching, and a positional relationship between feature points. Then, the speaker specifying unit 15 outputs a speaker specifying result representing the specified care receiver to the parameter acquiring unit 17.

Moreover, the speaker specific | specification part 15 calculates | requires an estimated age and estimated sex of a cared person based on the image recognition result and acoustic characteristic analysis result of a face image, when a cared person cannot be specified. In this embodiment, the speaker specific | specification part 15 can estimate the age and sex of a care receiver by a known method.
For example, as a method for estimating age and sex, there is a method using an IMDB-WIKI data set (Reference Document 3). This IMDB-WIKI data set can be used for the task of estimating age and gender from face images.
Moreover, as a technique for estimating gender, there is a method for recognizing formant distribution (Reference Document 4). In this method, since the formant distribution of speech differs between men and women, the sexes are determined by recognizing the formant distribution. For example, in the case of a male voice, the pitch frequency is 100 Hz to 150 Hz, and in the case of a female voice, the pitch frequency is 250 Hz to 300 Hz.
Further, as a method for estimating the age, there are methods using segmental features (acoustic characteristics of the vocal tract) and prosodic features (acoustic characteristics of the sound source). The former segmental feature utilizes formant shift due to the increase in vocal tract length accompanying aging and gain reduction in the spectral high band accompanying aging. The latter prosodic feature measures the decrease in average fundamental frequency with aging and the disturbance (simmer and jitter) of the sound source waveform (periodic waveform for voiced sound) with aging (reference) Reference 5).

Reference 3: “IMDB-WIKI”, [online], [searched on February 14, 2018], Internet <URL: https://data.vision.ee.ethz.ch/cvl/rrothe/imdb-wiki />
Reference 4: “Formant distribution”, [online], [Search February 14, 2018], Internet <URL: http://media.sys.wakayama-u.ac.jp/kawahara-lab/LOCAL/ diss / diss7 / S3_6.htm>
Reference 5: Nobuaki Hamamatsu, “Estimating Perceptual Age Using Acoustic Features of Speech and Improving Its Accuracy (Watching People)”

In addition, the speaker specifying unit 15 may obtain the estimated age and estimated sex of the care recipient from the motion analysis result of the captured image. For example, the speaker specifying unit 15 can estimate the age and sex of the care recipient by gait authentication. This gait authentication estimates the age and gender by analyzing the individuality of the way of walking from the stride and how to swing the arm, and comparing the analysis results with changes in the general way of walking with aging. is there.
Thereafter, the speaker identification unit 15 outputs an age-gender estimation result representing the estimated age and sex to the parameter acquisition unit 17.

The parameter storage unit 16 is a storage unit such as a memory that stores various parameters necessary for a conversation with the care recipient. For example, an administrator or a caregiver of the robot 1 manually sets various parameters in the parameter storage unit 16 for each care recipient. This parameter includes a voice parameter related to the voice to be output to the care recipient, a presentation operation parameter related to the presentation operation presented to the care receiver, and a singing parameter related to singing with the care receiver.
Further, the parameter storage unit 16 stores parameters corresponding to the estimated age and the estimated gender in case the care receiver cannot be identified.

<Audio parameters>
Hereinafter, various parameters stored in the parameter storage unit 16 will be described.
Here, it is said that humans gradually lose their hearing when they are over 20 years old. In the inner ear, cells (hair cells) with tens of thousands of hairs that play a role in transmitting sound are arranged. These tens of thousands of hairs change in response to the sound transmitted from the ear hole to the eardrum, thereby converting the sound into an electrical signal. It is thought that this hair cell hair decreases with aging is the cause of age-related hearing loss.

In addition, it is said that humans cannot hear high frequencies as they age, and they are muffled throughout and sound indistinct. Therefore, the speech synthesizer 21 may increase the reduced frequency component in order to output a sound that can be easily heard by the care recipient.
In addition, it is said that elderly people can hardly hear small sounds but suddenly increase at a certain volume. Simply speaking, you don't have to speak loudly in the ears of the elderly. In other words, the speech synthesizer 21 is preferably configured to suppress loud sounds and suppress loud sounds.
In addition, it is said that elderly people have less hair cells and lack information that can be transmitted from the inner ear to the brain, so that time resolution is reduced and it takes time to understand the content of words. Therefore, the speech synthesizer 21 may output the speech slowly and clearly.
In addition, it is said that elderly people can not understand the subtle frequency difference included in the sound and the listening ability is reduced. For example, elderly people are difficult to hear the sounds of pa, ta, ka and sa. Also, for example, elderly people are difficult to hear high frequency component sounds such as “shu”, “tsu”, “te”, “su”, “ka”, “hi”, “sa”, and “shi”.

  Therefore, paying attention to the following (1) to (3), the parameter storage unit 16 sets high frequency enhancement information and speech speed appropriate for each cared person as voice parameters. The high-frequency enhancement information is high-frequency emphasis information that represents the high-frequency component of the voice to be emphasized among the frequency components of the voice that the robot 1 outputs.

(1) Slowly and clearly speaking with a slightly louder voice rather than loud (2) Speak clearly clearly in the pa, ta, ka, and sa lines (3) The vowel part is overly loud Be careful not to become confused and speak long for the beginning of words.

  Further, the parameter storage unit 16 sets a speech replacement rule (question replacement rule) appropriate for each care recipient as a voice parameter. Examples of the conversation replacement rule include the following (A) to (C).

(A) Word replacement Replace the word “7 (shichi)” in the sentence “Let's meet at 7:00 tomorrow night” with “Nana”.
Replace the word “I miss” with “I miss”.

(B) Replacing the sentence Replace the sentence so that it is understood in advance that you are going to speak. For example, the document “I am planning to meet at the station square at 7 o'clock tomorrow” is replaced with “Meeting is scheduled for 7 o'clock tomorrow and the place will be at the station square”.

(C) Giving a space When a word is long, a space is given to take a break. For example, the word “karakushi vinegar miso” is replaced with “karashi □□□ vinegar miso”. Note that “□” represents a blank given to a word, and when converted to synthesized speech, it becomes a silent section.

<Presentation operation parameters>
It is said that humans have a narrow joint movable range as they age, and it takes a long time to start a motion, and a time during which a certain motion can be continued is shortened. In view of this, in the parameter storage unit 16, a motion part, a motion amount, a motion start time, and a motion continuation time appropriate for each care recipient are set as presentation motion parameters.
The presentation operation is an operation that the robot 1 presents to the care recipient.

  An example of the definition of the motion part and the motion amount will be described with reference to FIG. For example, when the motion part is the upper arm, the motion amount of the motion part can be represented by an angle for each motion direction. At this time, when the movement direction is flexion or extension, the movement portion and the movement amount have a line connecting the peaks of both shoulders as a basic axis and a line connecting the crown and shoulder peaks as a movement axis. When the direction of movement is fist up or down, the movement part and the movement amount are the same on the basic axis, but the line connecting the acromion and the upper sternum is the movement axis.

In addition, since the definition of an operation | movement part and operation amount is described in the reference document 6, for example, the description beyond this is abbreviate | omitted.
Reference 6: “Definition of range of motion of upper arm”, [online], [searched on February 14, 2018], Internet <URL: http://www.study-channel.com/2015/06/ROM -upper-limbs.html />

<Singing parameters>
It is said that humans cannot hear a high sound range with aging, and the singing speed decreases (Reference Documents 7 and 8). Therefore, in the parameter storage unit 16, a sound range and singing speed appropriate for each care recipient are set as singing parameters.
Reference 7: “Aging of voice” [online], [Search on March 30, 2018], Internet <URL: http://hozawa.jp/news/2011/11/post-48.html/>
Reference 8: Masateru Nishio, Seiji Niimi, “Change in spoken voice position with aging”, Spoken Language Medicine 46: 136-144,2005

Returning to FIG. 2, the description of the configuration of the robot 1 will be continued.
The parameter acquisition unit 17 acquires parameters from the parameter storage unit 16 based on the speaker identification result from the speaker identification unit 15 when the speaker identification unit 15 can identify the care recipient.
Further, the parameter acquisition unit 17 acquires parameters from the parameter storage unit 16 based on the age-sex estimation result from the speaker identification unit 15 when the speaker identification unit 15 cannot identify the care recipient.
Thereafter, the parameter acquisition unit 17 outputs the presentation operation parameter among the acquired parameters to the operation command unit 23, and outputs the speech parameter and the singing parameter to the speech synthesis unit 21.

The conversation analysis unit 18 analyzes the care receiver's conversation based on the word recognition result from the utterance recognition unit 13. In the present embodiment, the conversation analysis unit 18 analyzes specific conversation contents by a known method and outputs a conversation analysis result to the conversation selection unit 20.
Moreover, the conversation analysis part 18 commands a singing to the conversation selection part 20 (singing instruction | indication), when the conversation analysis result is singing hope with which it was preset. This singing request is a voice indicating that the care recipient desires to sing, such as “Let's sing xxx”. “XXX” represents the name of the song.
The conversation analysis unit 18 instructs the operation command unit 23 to perform a presentation operation (presentation operation command) when the conversation analysis result is a desired presentation operation set in advance. This presentation motion request is, for example, a voice indicating that the care recipient desires the presentation motion, such as “Let ’s try”. Note that “ΔΔΔ” represents the type of presentation operation such as gymnastics or Tai Chi.

  The conversation storage unit 19 is a storage unit such as a memory for storing a conversation (conversation program) with the care recipient. This conversation includes a question for the care recipient. The conversation storage unit 19 also stores lyrics / musical scores (songs) necessary for singing with the care recipient.

The conversation selection unit 20 selects a conversation stored in the conversation storage unit 19 based on the conversation analysis result from the conversation analysis unit 18. In the present embodiment, the conversation selection unit 20 selects a conversation having an appropriate content with respect to the care receiver's utterance by a known method.
In addition, when a singing instruction is input from the conversation analyzing unit 18, the conversation selecting unit 20 selects lyrics / musical scores corresponding to the instruction from the conversation storing unit 19.
After that, the conversation selection unit 20 outputs the selected conversation or the lyrics / score to the speech synthesis unit 21.

  The voice synthesizer 21 outputs the conversation from the conversation selector 20 via the speaker S by voice. In the present embodiment, the speech synthesizer 21 replaces the conversation based on the speech parameter (conversation replacement rule) from the parameter storage unit 16. Then, the speech synthesizer 21 generates synthesized speech representing the conversation based on the speech parameters (high frequency enhancement information, speech speed), and outputs the generated synthesized speech to the speaker S. Thus, since the robot 1 outputs the synthesized speech at a frequency and speech speed that are easy for the care recipient to hear, it becomes easy for the care recipient to continue the conversation.

  The voice synthesizer 21 outputs the lyrics / score from the conversation selector 20 via the speaker S. In the present embodiment, the speech synthesizer 21 generates lyrics synthesized speech from the conversation selector 20. The voice synthesizer 21 performs singing with the generated synthesized voice based on the singing parameters (sound range, singing speed) from the parameter storage unit 16 together with the accompaniment by the score. Thus, since the robot 1 performs singing and accompaniment at a singing speed and a singing speed that are easy for the cared person to sing, the cared person can easily continue singing.

  The presentation operation storage unit 22 is a storage unit such as a memory that stores presentation operations such as gymnastics and tai chi. This presenting operation is an operation that the robot 1 presents to the care recipient, such as gymnastics or Tai Chi.

  The motion command unit 23 acquires a presentation motion from the presentation motion storage unit 22 and commands the acquired presentation motion to the drive mechanism D. In the present embodiment, when a presentation motion command is input from the conversation analysis unit 18, the motion command unit 23 acquires a presentation motion corresponding to the command from the presentation motion storage unit 22. Then, the motion command unit 23 adjusts the presentation motion based on the presentation motion parameter from the parameter storage unit 16 and the motion analysis result from the motion analysis unit 12. Thus, since the robot 1 adjusts the presentation operation in accordance with the movement of the cared person, the cared person can easily continue the exercise.

[Robot motion]
The operation of the robot 1 will be described with reference to FIG.
As shown in FIG. 4, the robot 1 determines whether or not to perform a presentation operation. For example, the robot 1 determines whether or not to perform the presentation operation by the conversation analysis unit 18 depending on whether or not the conversation analysis result is a presentation operation request (step S1).
When the presentation operation is not performed (No in step S1), the robot 1 determines whether or not to sing. For example, in the robot 1, the conversation analysis unit 18 determines whether or not the conversation analysis result is a singing request (step S2).
When singing is not performed (No in step S2), the robot 1 performs the speaker recognition process of FIG. 5 and acquires parameters (step S3).

<Speaker recognition processing>
The speaker recognition process will be described with reference to FIG.
As shown in FIG. 5, the camera C captures a face image of the care recipient (step S100).
The face image recognition unit 11 recognizes the face image captured in step S100.
The speaker specifying unit 15 specifies the care recipient based on the image recognition result of the face image recognition unit 11 (step S101).

  When the care receiver can be identified (Yes in step S101), the parameter acquisition unit 17 acquires parameters from the parameter storage unit 16 for the care recipient identified in step S101, and ends the speaker recognition process (step S101). S102).

When the cared person cannot be identified (No in step S101), the speaker identifying unit 15 estimates the age and sex of the cared person based on the image recognition result of the face image (step S103).
The parameter acquisition unit 17 acquires the parameters corresponding to the age and sex estimated in step S103 from the parameter storage unit 16, and ends the speaker recognition process (step S104).

Returning to FIG. 4, the description of the operation of the robot 1 will be continued.
The conversation selection unit 20 selects a question for the care recipient because the care recipient does not wish to present or sing. For example, the conversation selection unit 20 selects a question such as “How are you feeling?” That prompts the cared person to speak.
The speech synthesizer 21 replaces the selected question based on the speech parameter (conversation replacement rule) acquired in step S3. Then, the speech synthesizer 21 generates synthesized speech of the selected question based on the speech parameters (high frequency enhancement information, speech speed), and outputs the generated synthesized speech to the speaker S (step S4).

  The utterance recognition unit 13 recognizes a word included in the care receiver's voice input from the microphone M, and determines whether or not the care receiver has answered the question in step S4 (step S5).

When the cared person responds (Yes in step S5), the conversation analysis unit 18 analyzes the cared person's response based on the word recognition result in step S5.
The conversation selection unit 20 selects a conversation from the conversation storage unit 19 based on the response of the care recipient analyzed by the conversation analysis unit 18.
The speech synthesizer 21 replaces the selected conversation based on the speech parameter (conversation replacement rule) acquired in step S3. Then, the speech synthesizer 21 generates synthesized speech of the selected conversation based on the speech parameters (high frequency enhancement information, speech speed), and outputs the generated synthesized speech to the speaker S (step S6).

Thereafter, the robot 1 continues the process of step S6 until the cared person finishes the conversation. When the cared person finishes the conversation, the robot 1 returns to the process of step S1.
In some cases, the cared person cannot be identified by the speaker recognition process in FIG. 5 even though the cared person responds. In this case, the robot 1 performs a voice analysis process (FIG. 9) to be described later, and identifies the care recipient.

  When the cared person does not respond (No in step S5), the speaker specifying unit 15 estimates the age and sex of the cared person based on the image recognition result of the face image (step S7).

The conversation selection unit 20 selects a question for the care recipient.
The speech synthesizer 21 replaces the selected question based on the acquired speech parameter (conversation replacement rule). Then, the speech synthesizer 21 further generates a synthesized speech in which the speech speed is further reduced and the high frequency component is emphasized based on the age and sex estimated in step S7. Here, since the speech synthesizer 21 is easy to hear when the speech speed is slowed by 20% (Reference Document 9), the speech speed is reduced by 20% (Step S8).
Reference 9: "People-friendly speech speed conversion", [online], [Search on March 30, 2018], Internet <URL: https://www.nhk.or.jp/strl/onepoint/data/ wasoku.pdf # search =% 27% E9% AB% 98% E9% BD% A2% E8% 80% 85 +% E8% A9% B1% E9% 80% 9F% 27>

The voice synthesizer 21 outputs the synthesized voice of the question generated in step S8 to the speaker S (step S9).
The utterance recognition unit 13 recognizes a word included in the care receiver's voice input from the microphone M, and determines whether or not the care receiver has answered the question in step S9 (step S10).
When the cared person responds (Yes in step S10), the robot 1 performs the speaker recognition process of FIG. 5 and acquires parameters (step S11).

Similar to step S6, the robot 1 has a conversation using the parameters acquired in step S11 (step S12).
Thereafter, the robot 1 continues the process of step S12 until the cared person finishes the conversation. When the cared person finishes the conversation, the robot 1 proceeds to the process of step S13.

  The parameter acquisition unit 17 writes the care receiver identification information (for example, the feature amount of the face image) and the parameter acquired in step S11 in association with each other in the parameter storage unit 16, and returns to the process of step S1 (step S13). ).

If the care receiver does not respond (No in step S10), the speaker specifying unit 15 adds “1” to the number of questions. The number of questions represents the number of times the robot 1 has asked the care recipient, and the initial value is “0” (step S14).
The speaker specifying unit 15 determines whether or not the number of questions added in step S14 is less than a preset designated number. This designated number of times represents the number of times that the robot 1 makes a question to the care recipient (step S15).
If the number of questions is less than the specified number (Yes in step S15), the robot 1 returns to the process in step S8.

When the number of questions is equal to or greater than the specified number (No in step S15), the conversation selection unit 20 selects the conversation end from the conversation storage unit 19. The end of the conversation is, for example, the content of ending the conversation with the cared person, such as “Sorry, I cannot speak easily”.
The voice synthesizer 21 generates a synthesized voice at the end of the conversation, outputs the generated synthesized voice to the speaker S (step S16), and returns to the process of step S1.

When performing the presentation operation (Yes in step S1), the robot 1 performs the operation analysis process of FIG. 6 and returns to the process of step S1 (step S17).
When singing (Yes in step S2), the robot 1 performs the singing process of FIG. 8 and returns to the process of step S1 (step S18).

<Operation analysis processing>
The operation analysis process will be described with reference to FIGS.
As shown in FIG. 6, the robot 1 performs the speaker recognition process of FIG. 5 and acquires parameters (step S200).
The operation command unit 23 acquires the presentation operation desired by the care recipient from the presentation operation storage unit 22, and instructs the drive mechanism D with the presentation operation parameter in step S200 (step S201).
The camera C takes a photographed image of the care recipient (step S202).

  The motion analysis unit 12 analyzes the follow-up motion of the care recipient using the captured image captured in step S202. For example, the motion analysis unit 12 obtains a motion analysis result such as the motion part, motion amount, motion start time, motion duration, and the like of the care recipient from the captured image (step S203).

The motion analysis unit 12 compares the presentation motion performed by the robot 1 with the follow-up motion performed by the care recipient, and determines whether or not the motion portions match. For example, the motion analysis unit 12 determines whether or not the movable unit driven by the robot 1 and the joint being moved by the care recipient match (step S204).
If the motion parts do not match (No in step S204), the motion analysis unit 12 determines whether or not the cared person is in the sitting position (step S205).

When the cared person is in the sitting position (Yes in step S205), the motion analysis unit 12 commands the sitting mode to the conversation selection unit 20 or the motion command unit 23. This sitting mode is a mode in which the robot 1 moves in small steps or guides by voice a part of the actions that can be performed while the cared person is in the sitting position.
The motion command unit 23 commands the presentation mechanism to the drive mechanism D in the sitting position mode (step S206).

When the cared person is not in the sitting position (No in step S205), the motion analysis unit 12 commands the standing mode to the conversation selection unit 20 or the motion command unit 23. This standing mode is a mode in which the robot 1 moves in small steps or guides by voice a portion of the motions that can be performed while the care recipient is standing.
The operation command unit 23 instructs the drive mechanism D to perform a presentation operation in the standing mode (step S207).

For example, in rehabilitation, it is basic that the cared person performs a follow-up operation in accordance with the presentation operation of the robot 1, while the robot 1 also needs to operate in accordance with the cared person.
Therefore, the motion analysis unit 12 compares the motion amount between the presentation motion performed by the robot 1 and the follow-up motion performed by the care recipient. For example, the motion analysis unit 12 presets a motion amount threshold value for each presentation motion, and compares this motion amount threshold value with the motion amount of the follow-up motion (step S208).

  When the motion amount of the follow-up motion is smaller than the motion amount threshold value (small in step S208), the motion analysis unit 12 commands the conversation selection unit 20 or the motion command unit 23 to increase the motion amount of the follow-up motion. In this case, the robot 1 moves a portion where the movement amount of the follow-up operation is small or guides it by voice (step S209).

  When the motion amount of the follow-up motion is larger than the motion amount threshold (large in step S208), the motion analysis unit 12 commands the conversation selection unit 20 or the motion command unit 23 to decrease the follow-up motion amount. In this case, the robot 1 moves a portion where the amount of the follow-up motion is large or guides it by voice (step S210).

  The motion analysis unit 12 compares the motion speed of the presentation motion performed by the robot 1 and the follow-up motion performed by the care recipient. For example, the motion analysis unit 12 presets a motion speed threshold value for each presentation motion, and compares the motion speed threshold value with the motion speed of the follow-up motion (step S211).

  When the operation speed of the follow-up operation is slower than the operation speed threshold (delayed in step S211), the operation analysis unit 12 instructs the conversation selection unit 20 and the operation command unit 23 to increase the speed of the follow-up operation. In this case, the robot 1 moves a portion where the operation speed of the follow-up operation is slow or guides it by voice (step S212).

  When the operation speed of the follow-up operation is faster than the operation speed threshold (fast in step S211), the operation analysis unit 12 instructs the conversation selection unit 20 and the operation command unit 23 to decrease the speed of the follow-up operation. In this case, the robot 1 moves a portion where the operation speed of the follow-up operation is fast or guides it by voice (step S213).

The motion analysis unit 12 determines whether or not the operation start time is adjusted for the first time (step S214).
When the adjustment of the motion start time is the first time (Yes in step S214), the motion analysis unit 12 compares the motion start time between the presentation motion performed by the robot 1 and the follow-up motion performed by the care recipient.
The motion analysis unit 12 adjusts the motion start time of the follow-up motion. That is, the motion analysis unit 12 delays the operation start time of the tracking operation when the operation start time of the tracking operation is earlier than the presentation operation, and increases the start time of the tracking operation when the operation start time of the tracking operation is later than the presentation operation. The operation command unit 23 is instructed to do so. In this case, the robot 1 moves a portion requiring adjustment of the operation start time of the follow-up operation in small steps or guides it by voice (step S215).
Note that the motion analysis unit 12 does not perform adjustment when the motion start times of the follow-up motion and the presentation motion match.

  When the adjustment of the operation start time is not the first time (No in step S214), it is considered that the care recipient cannot start the operation earlier than this. Therefore, the motion analysis unit 12 commands the motion command unit 23 to delay the motion start time of the presentation motion (step S216).

The motion analysis unit 12 determines whether or not the operation duration time is adjusted for the first time (step S217).
When the adjustment of the operation duration is the first time (Yes in step S217), the operation analysis unit 12 compares the operation duration of the presentation operation performed by the robot 1 and the follow-up operation performed by the care recipient.
The motion analysis unit 12 adjusts the motion continuation time of the follow-up motion. That is, the motion analysis unit 12 increases the operation duration time of the tracking operation when the motion duration time of the tracking operation is shorter than the presentation operation, and increases the operation duration time of the tracking operation when the motion duration time of the tracking operation is longer than the presentation operation. Is commanded to the operation command unit 23 so as to shorten In this case, the robot 1 moves a portion that needs to be adjusted for the duration of the follow-up operation in small steps or guides it by voice (step S218).
Note that the motion analysis unit 12 does not perform adjustment when the motion durations of the follow-up motion and the presentation motion match.

  When the adjustment of the operation duration time is not the first time (No in step S217), it is considered that the care recipient cannot operate any faster. Therefore, the motion analysis unit 12 commands the motion command unit 23 to increase the motion continuation time of the follow-up motion (step S219).

  The motion analysis unit 12 determines whether the follow-up motion of the care receiver is stable. For example, the motion analysis unit 12 determines that the operation is stable when the displacement of the motion amount or the motion start time is within the motion stable range. This operation stable range is set in advance according to the age and sex of the cared person, the type and difficulty of the presentation operation (step S220).

  When the tracking operation is stable (Yes in step S220), the parameter acquisition unit 17 writes the care receiver identification information and the adjusted presentation operation parameter in association with each other in the parameter storage unit 16 (step S221).

The operation command unit 23 determines whether or not the presentation operation has ended (step S222).
If the presentation operation has not ended (No in step S222), the operation command unit 23 returns to the process of step S202.
When the presentation operation is completed (Yes in step S222), the operation command unit 23 ends the operation analysis process.

<Singing process>
The singing process will be described with reference to FIG.
As shown in FIG. 8, the robot 1 performs the speaker recognition process of FIG. 5 and acquires parameters (step S300).

The conversation selection unit 20 selects lyrics / scores from the conversation storage unit 19 for the song desired by the care recipient.
The speech synthesizer 21 performs singing and accompaniment with the singing parameters (sound range, singing speed) acquired in step S300 (step S301).

The voice analysis unit 14 analyzes the acoustic characteristics of the cared person's song input from the microphone M, and extracts the cared person's range and singing speed (step S302).
The voice synthesizer 21 changes the sound range and singing speed in accordance with the acoustic characteristics of the cared person extracted in step S302 (step S303).

  The voice analysis unit 14 determines whether or not the cared person's singing is stable. For example, the voice analysis unit 14 determines that the singing is stable when the care receiver's range or singing speed is within the singing stable range. This singing stable range is set in advance according to the age and sex of the cared person, the kind of song, and the difficulty level (step S304).

  When the singing is stable (Yes in step S304), the parameter acquisition unit 17 writes the identification information of the care recipient and the adjusted singing parameter in the parameter storage unit 16 in association with each other (step S305).

The conversation selection unit 20 determines whether or not the singing is finished (step S306).
When the singing has not ended (No in step S306), the conversation selection unit 20 returns to the process of step S302.
When the singing is finished (Yes in step S306), the conversation selecting unit 20 finishes the singing process.

<Audio analysis processing>
The voice analysis process will be described with reference to FIG.
As shown in FIG. 9, the microphone M acquires the care receiver's voice (step S400).
The voice analysis unit 14 analyzes the acoustic characteristics of the cared person acquired in step S400 (step S401).
The speaker specifying unit 15 specifies the care receiver (speaker) based on the acoustic characteristic analysis result of step S401 (step S402).

  When the care recipient can be identified (Yes in step S402), the parameter acquisition unit 17 acquires parameters from the parameter storage unit 16 for the care recipient identified in step S402, and ends the voice analysis process (step S403). ).

  When the care receiver cannot be specified (No in step S402), the parameter acquisition unit 17 acquires a new care receiver parameter. For example, the parameter acquisition unit 17 may use a preset parameter initial value as a new care receiver parameter. In addition, the parameter acquisition part 17 may use the parameter when a song was stabilized by step S304, when a new cared person sings. Then, the parameter acquisition unit 17 writes the new care receiver parameter in the parameter storage unit 16 in association with the care receiver identification information (for example, the feature amount of the face image), and ends the voice analysis process ( Step S404).

[Action / Effect]
As described above, the robot 1 according to the present embodiment replaces words and sentences that are difficult for the cared person to hear, and also performs conversation according to the cared person, so that the cared person continues without interrupting the conversation. It becomes easy.
Furthermore, since the robot 1 operates according to the cared person, the cared person can easily continue the exercise without interruption.
Furthermore, since the robot 1 performs singing and accompaniment in accordance with the cared person, the cared person can easily continue without interrupting the song.
Furthermore, since the robot 1 learns parameters suitable for the cared person, the cared person can more easily continue the conversation, operation, and singing.
As described above, the robot 1 can deal with the care recipient in place of the caregiver.

(Modification)
As mentioned above, although embodiment of this invention was explained in full detail, this invention is not limited to above-described embodiment, The design change etc. of the range which does not deviate from the summary of this invention are included.
In the above-described embodiment, it has been described that the person who is a conversation partner with the robot is a cared person, but is not limited to the cared person.
In the above-described embodiment, the robot is described as being a stand-alone, but a faster computer can also be used with the cloud.

In the above-described embodiment, it has been described that there is one cared person, but it can also be applied to a group of a plurality of cared persons. In this case, the robot may perform utterances, presentation operations, and singing suitable for more than half of the care recipients.
In the above-described embodiment, it has been described that the robot performs singing and accompaniment, but either one of singing or accompaniment may be performed. For example, the robot may include a display, perform accompaniment, and display lyrics on the display in accordance with the accompaniment.
In the above-described embodiment, the robot has been described as performing only one of conversation, presentation operation, and singing, but may be performed in combination. For example, the robot can perform a combination of conversation and presentation operation, and a combination of presentation operation and singing.

  In the above-described embodiment, the robot has been described as independent hardware, but the present invention is not limited to this. For example, the present invention can also be realized by a program that causes hardware resources such as a CPU, a memory, and a hard disk included in a computer to operate cooperatively as the control unit of the robot. These programs may be distributed via a communication line, or may be distributed by writing in a recording medium such as a CD-ROM or a flash memory.

DESCRIPTION OF SYMBOLS 1 Robot 10 Control part 11 Face image recognition part 12 Motion analysis part 13 Speech recognition part (voice recognition part)
14 Voice analysis part 15 Speaker specific part (person specific part)
16 parameter storage unit 17 parameter acquisition unit 18 conversation analysis unit 19 conversation storage unit 20 question selection unit (conversation selection unit)
21 Speech synthesis unit (speech output unit)
22 presentation motion storage unit 23 motion command unit C camera M microphone S speaker D drive mechanism

Claims (6)

A robot comprising a microphone for inputting a person's speech, a camera for photographing the person, and a speaker for outputting sound to the person,
A question selection unit for selecting a question for the person;
Via the speaker, a voice output unit that outputs the question selected by the question selection unit by voice; and
A voice recognition unit that recognizes the person's response to the question input from the microphone;
A face image recognition unit for recognizing a face image of a person photographed by the camera;
A person identifying unit that identifies the person based on the voice recognition result of the reply or the image recognition result of the face image;
As the parameter for each person, high frequency emphasis information representing the high frequency side frequency component of the voice to be emphasized, a question replacement rule for replacing the content of the question, and a parameter storage unit for storing speech speed;
A parameter acquisition unit that acquires a parameter from the parameter storage unit for the person specified by the person specifying unit;
The voice output unit replaces the content of the question according to the parameter acquired by the parameter acquisition unit, and outputs the question at a frequency and a speech speed according to the parameter. The person specifying unit, when the person cannot be specified, obtains the estimated age and estimated sex of the person from the image recognition result of the face image,
The parameter storage unit further stores parameters corresponding to the estimated age and the estimated sex,
The robot according to claim 1, wherein the parameter acquisition unit acquires parameters corresponding to the estimated age and the estimated sex when the person specifying unit cannot specify the person.   The parameter acquisition unit writes the identification information of the person and the parameter at the time of outputting the question in the parameter storage unit when a response to the question output with the parameters of the estimated age and the estimated gender is input. The robot according to claim 2. A drive mechanism for driving the movable part of the robot;
An operation command unit that commands the drive mechanism to perform a preset presentation operation;
The parameter storage unit further stores an operation part, an operation amount, an operation start time, and an operation duration as the parameters,
The said operation command part commands the said presentation operation | movement with the operation | movement part according to the parameter which the said parameter acquisition part acquired, the operation amount, the operation start time, and the operation continuation time. The robot according to any one of the above. The parameter storage unit further stores a range and singing speed as the parameter,
The robot according to any one of claims 1 to 4, wherein the voice output unit sings at a sound range and a singing speed corresponding to the parameter acquired by the parameter acquisition unit.   The robot control program for functioning a computer as a robot as described in any one of Claims 1-5.

Images