JP4529091B2 - Learning apparatus, learning method, and robot apparatus - Google Patents

Description

  The present invention relates to a learning apparatus, a learning method, and a robot apparatus, and is suitable for application to, for example, an entertainment robot.

  Conventionally, when a recognizer such as a fingerprint recognizer or voiceprint recognizer in a security system, for example, tries to learn a new category, there is a “learning mode” in which learning is clearly indicated to the user. To do.

  In such a “learning mode”, the purpose of sensing information being used for security is clear when the user remembers fingerprints or voiceprints for security as described above. It is preferable to be explicit to inform the user that it is registered.

  However, in an entertainment robot that uses sensing information to recognize who the user is during the interaction, it is important to be able to recognize the user in a natural interaction.

  For this reason, in such an entertainment robot, when learning the user's face, for example, by telling the robot that the robot is learning the face by saying "Please remember to remember the face" In other words, there is a problem that obstructs natural interaction with the user, which is the original purpose.

  On the other hand, in an entertainment robot that learns the user's name, in order to perform natural interaction with the user, the characteristics (sensing information associated with the name) of the user's name taught by the user should be as much as possible. It is necessary to devise so that it can be remembered once.

  However, with the entertainment robots that have been proposed in the past, the success or failure of learning is fixed such that learning fails when sufficient data is not obtained within a certain period of time. There was a problem that troubles occurred frequently and the user was troublesome interaction that the robot asked the name many times without connecting the sensing information and the name.

  Furthermore, in the conventionally proposed entertainment robot, when learning must be terminated without sufficiently acquiring learning data, the learning at that time has failed, and all the data obtained by the learning has been discarded. For this reason, there is a problem that it is difficult to perform efficient learning without utilizing the learning results up to that point.

  Therefore, if the entertainment robot can recognize the user in a natural interaction, and if the number of learning failures can be reduced as much as possible, the entertainment property can be further improved. .

  The present invention has been made in consideration of the above points, and an object of the present invention is to propose a learning apparatus, a learning method, and a robot apparatus that can improve entertainment.

In order to solve such a problem, in the present invention, in the learning device, a storage unit that associates and stores a user name and a feature, a user detection unit that detects a user based on an image obtained from a camera, and a user detection When the user is detected by the unit, before asking the user for a name, the user's characteristics are detected using at least one of an image obtained from the camera and a sound obtained from the microphone. The user is asked for the name through an interaction between the recognition unit for recognizing the name of the user and the user detected by the user detection unit by comparing the feature of the user and the feature already stored in the storage unit Thus, the dialogue unit for obtaining the name of the user, the user name obtained by the dialogue unit, and the characteristics of the user detected by the recognition unit are described. When it is determined that the user is a new user because it is not stored in the part, the feature of the user is learned by the recognition part using at least one of the image and the sound, and the obtained as a result of the learning A control unit that associates a user feature with the name of the user acquired by the dialogue unit and stores it in the storage unit, and the number of images used by the recognition unit to detect the feature obtained when the feature is detected And whether the feature learning achievement is determined based on at least one of the time length of the voice and the speech, and the dialogue unit is not sufficiently learning the feature by the recognition unit using the learning achievement degree determined by the recognition unit determine, sometimes learning is determined to be insufficient, and to execute a process for extending the interaction with the user. As a result, this learning device can learn the user's name through dialogue with a normal user without being recognized by the user.

In the present invention, in the learning method, when a user is detected based on an image obtained from a camera, before asking the user for the name, at least an image obtained from the camera and an audio obtained from a microphone are used. The user's feature is detected using one of them, and the detected feature of the user is compared with the feature already stored in the storage unit that stores the user's name and the feature in association with each other. A first step of recognizing the name of the user, a second step of acquiring the name of the user by asking the name of the user through interaction with the detected user, and a user acquired in the second step When it is determined that the user is a new user because the name of the user and the characteristics of the user detected in the first step are not stored in the storage unit The feature of the user is learned using at least one of the image and the sound, and the feature of the user obtained as a result of the learning and the name of the user acquired in the second step are associated and stored in the storage unit A degree of learning achievement of the feature based on at least one of the number of images and the time length of the sound used for feature detection obtained at the time of feature detection. determined, it is determined whether insufficiently learning feature using the determined learning achievement, sometimes learning is judged to be insufficient, to execute a process for extending the dialogue with the user I made it. As a result, according to this learning method, the user's name can be learned through dialogue with a normal user without being recognized by the user.

Furthermore, in the present invention, in the robot apparatus, the storage unit that stores the user name and the feature in association with each other, the user detection unit that detects the user based on the image obtained from the camera, and the user detection unit that detects the user Then, before asking the user for the name, the user's feature is detected using at least one of the image obtained from the camera and the sound obtained from the microphone, and the detected feature of the user is stored. By comparing the feature already stored in the unit, the user recognizes the name of the user through dialogue with the recognition unit that recognizes the name of the user and the user detected by the user detection unit. The dialogue unit for acquiring the name of the user, the user name obtained by the dialogue unit, and the characteristics of the user detected by the recognition unit are stored in the storage unit. When it is determined that the user is a new user, the recognition unit learns the user's characteristics using at least one of the image and the sound, and the characteristics of the user obtained as a result of the learning, A control unit that associates the name of the user acquired by the dialogue unit with the storage unit and stores it in the storage unit, and the recognition unit obtains the number of images and the time length of the sound obtained when detecting the feature. The learning achievement level of the feature is determined based on at least one of the above, and the dialogue unit determines whether the learning of the feature by the recognition unit is insufficient or not by using the learning achievement level determined by the recognition unit. When it is determined that the above is insufficient, a process for extending the dialogue with the user is executed. As a result, this robot apparatus can learn the user's name through dialogue with a normal user without being recognized by the user.

According to the present invention, in the learning device, the storage unit that stores the user name and the feature in association with each other, the user detection unit that detects the user based on the image obtained from the camera, and the user detection unit that detects the user Then, before asking the user for the name, the user's feature is detected using at least one of the image obtained from the camera and the sound obtained from the microphone, and the detected feature of the user is stored. By comparing the feature already stored in the unit, the user recognizes the name of the user through dialogue with the recognition unit that recognizes the name of the user and the user detected by the user detection unit. The storage unit does not store the dialog unit for acquiring the name of the user, the user name acquired by the dialog unit, and the characteristics of the user detected by the recognition unit When it is determined that the user is a new user, the recognition unit learns the user's characteristics using at least one of image and sound, and the user's characteristics obtained as a result of the learning and the dialogue unit A control unit that associates and stores the name of the user acquired by the storage unit in the storage unit, and the recognition unit uses the number of images and the time length of sound used for feature detection obtained when the feature is detected. The learning achievement level of the feature is determined based on at least one of the above, and the dialogue unit determines whether or not the learning of the feature by the recognition unit is insufficient using the learning achievement level determined by the recognition unit. When it is judged that it is sufficient, a process for extending the dialogue with the user is executed. As a result, this learning device can learn the user's name through dialogue with a normal user without being recognized by the user, thus realizing a learning device that can improve entertainment.

According to the present invention, in the learning method, when a user is detected based on an image obtained from the camera, before asking the user for the name, the image obtained from the camera and the sound obtained from the microphone The feature of the user is detected using at least one, and the feature of the detected user is compared with the feature already stored in the storage unit that stores the user name and the feature in association with each other. Obtained in the first step of recognizing the name of the user, the second step of obtaining the name of the user by asking the name of the user through the interaction with the detected user, and the second step When it is determined that the user is a new user because the user's name and the characteristics of the user detected in the first step are not stored in the storage unit, The feature of the user is learned using at least one of the image and the voice, and the feature of the user obtained as a result of the learning and the name of the user acquired in the second step are associated and stored in the storage unit A degree of learning achievement of the feature based on at least one of the number of images and the time length of the sound used for feature detection obtained at the time of feature detection. determined, it is determined whether insufficiently learning feature using the determined learning achievement, sometimes learning is judged to be insufficient, to execute a process for extending the dialogue with the user I made it. As a result, according to this learning method, the user's name can be learned through dialogue with a normal user without being recognized by the user, and thus a learning method that can improve entertainment properties can be realized.

Further, according to the present invention, in the robot apparatus, the storage unit that stores the user name and the feature in association with each other, the user detection unit that detects the user based on the image obtained from the camera, and the user detection unit When detected, before asking the user for the name, the user's feature is detected using at least one of an image obtained from the camera and a voice obtained from the microphone, and the detected feature of the user, By comparing the feature already stored in the storage unit with the recognition unit for recognizing the name of the user and the dialogue with the user detected by the user detection unit, the user is asked for the name, The storage unit stores the dialogue unit for obtaining the user name, the user name obtained by the dialogue unit, and the characteristics of the user detected by the recognition unit. When it is determined that the user is a new user, the recognition unit learns the user's characteristics using at least one of an image and audio, and the user's characteristics obtained as a result of the learning; A control unit that associates the name of the user acquired by the dialogue unit with the storage unit and stores it in the storage unit, and the recognition unit obtains the number of images and the time length of the sound obtained when detecting the feature. The learning achievement level of the feature is determined based on at least one of the above, and the dialogue unit determines whether the learning of the feature by the recognition unit is insufficient or not by using the learning achievement level determined by the recognition unit. When it is determined that the above is insufficient, a process for extending the dialogue with the user is executed. As a result, this robot apparatus can learn the user's name through dialogue with a normal user without being recognized by the user, and thus can realize a robot apparatus that can improve entertainment properties.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

(1) Configuration of Robot According to this Embodiment In FIGS. 1 and 2, 1 indicates a bipedal walking robot according to this embodiment as a whole, and a head unit 3 is disposed above the body unit 2. In addition, arm units 4A and 4B having the same configuration are respectively disposed on the upper left and right of the body unit 2, and leg units 5A and 5B having the same structure are disposed on the lower left and right of the body unit 2, respectively. It is configured by being attached at a predetermined position.

In the torso unit 2, a frame 10 that forms the upper part of the trunk and a waist base 11 that forms the lower part of the trunk are connected via a hip joint mechanism 12. By driving the actuators A ₁ and A ₂ of the fixed hip joint mechanism 12, the upper part of the trunk can be independently rotated around the orthogonal roll axis 13 and the pitch axis 14 shown in FIG. Has been made.

The head unit 3 is attached to the center of the upper surface of the shoulder base 15 fixed to the upper end of the frame 10 via a neck joint mechanism 16, and the actuators A ₃ and A ₄ of the neck joint mechanism 16 are respectively connected to the head unit 3. By being driven, it can be rotated independently around the orthogonal pitch axis 17 and yaw axis 18 shown in FIG.

Furthermore arm units 4A, 4B are attached to the left and right shoulder base 15 respectively via a shoulder joint mechanism 19, FIG by driving corresponding to each actuator A _5, A ₆ of the shoulder joint mechanism 19 respectively 3 can be rotated independently around the orthogonal pitch axis 20 and roll axis 21 shown in FIG.

In this case, each of the arm units 4A, 4B, the actuator A ₈ to form a forearm via an elbow joint mechanism 22 is connected to each output shaft of the actuator A ₇ forming the upper arm, the tip end of the forearm It is configured by attaching the hand portion 23.

The arm units 4A, in 4B, the forearm is rotated around the yaw axis 24 shown in FIG. 3 by driving the actuator A _7, pitch indicating the forearm in Fig. 3 by driving the actuator A ₈ Each of them can be rotated around an axis 25.

On the other hand, each leg unit 5A, 5B is attached to the waist base 11 at the lower part of the trunk via the hip joint mechanism 26, and the actuators of the corresponding hip joint mechanism 26 are respectively A _{9 to} A _11. By driving each of them, the yaw axis 27, the roll axis 28 and the pitch axis 29 which are orthogonal to each other shown in FIG.

  In this case, each leg unit 5A, 5B is connected to a lower end of a frame 30 that forms a thigh, a frame 32 that forms a lower leg through a knee joint mechanism 31, and to the lower end of the frame 32. The foot portion 34 is connected via an ankle joint mechanism 33.

Thus leg units 5A, in 5B, by driving the actuator A ₁₂ which forms a knee joint mechanism 31, it is possible to rotate the lower leg around the pitch axis 35 shown in FIG. 3, also ankles By driving the actuators A ₁₃ and A ₁₄ of the joint mechanism 33, the foot 34 can be rotated independently around the orthogonal pitch axis 36 and roll axis 37 shown in FIG.

  On the other hand, on the back side of the waist base 11 that forms the lower part of the trunk of the trunk unit 2, as shown in FIG. 4, a main control unit 40 that controls the operation of the entire robot 1, a power supply circuit, a communication circuit, and the like. A control unit 42 in which the peripheral circuit 41 and the battery 45 (FIG. 5) are housed in a box is provided.

  The control unit 42 includes sub-control units 43A to 43A disposed in the constituent units (the body unit 2, the head unit 3, the arm units 4A and 4B, and the leg units 5A and 5B). It is connected to 43D and can supply a necessary power supply voltage to these sub-control units 43A to 43D and can communicate with these sub-control units 43A to 43D.

The sub-control units 43A to 43D are connected to the actuators A _{1 to} A _{14 in} the corresponding constituent units, respectively, and the actuators A _{1 to} A ₁₄ in the constituent units are given from the main control unit 40. It can be driven to a designated state based on various control commands.

  Further, as shown in FIG. 5, the head unit 3 includes various external sensors such as a CCD (Charge Coupled Device) camera 50 that functions as an “eye” of the robot 1 and a microphone 51 that functions as an “ear”. A speaker 52 functioning as a “mouth” and the like are disposed at predetermined positions, respectively, and a touch sensor 53 as an external sensor is disposed on the bottom portion of the hand portion 23 and the foot portion 34. Further, an internal sensor including a battery sensor 54 and an acceleration sensor 55 is disposed in the control unit 42.

The CCD camera 50 images the surrounding situation and sends the obtained image signal S1A to the main control unit, while the microphone 51 collects various external sounds and controls the audio signal S1B thus obtained as the main control. The data is sent to the unit 40.
Further, the touch sensor 53 detects a physical action from the user or a physical contact with the outside, and sends the detection result to the main control unit 40 as a pressure detection signal S1C.

  The battery sensor 54 detects the remaining amount of energy of the battery 45 at a predetermined cycle, and sends the detection result to the main control unit 40 as a remaining battery amount detection signal S2A. On the other hand, the acceleration sensor 56 has three axis directions (x-axis, (y-axis and z-axis) are detected at a predetermined cycle, and the detection result is sent to the main control unit 40 as an acceleration detection signal S2B.

  The main control unit 40 includes an image signal S1A, an audio signal S1B, a pressure detection signal S1C, and the like as external sensor outputs supplied from a CCD camera 50, a microphone 51, a touch sensor 53, and the like, a battery sensor 54, an acceleration sensor 55, and the like. Based on the remaining battery level detection signal S2A, the acceleration detection signal S2B, and the like as internal sensor outputs respectively supplied from the above, the situation around and inside the robot 1, contact with an external object, and the like are determined.

The main control unit 40 determines the action to be continued based on the determination result, the control program stored in the internal memory 40A in advance, and various control parameters stored in the external memory 56 loaded at that time. Then, the control command based on the determination result is sent to the corresponding sub-control units 43A to 43D. As a result, based on this control command, the corresponding actuators A _{1 to} A ₁₄ are driven under the control of the sub-control units 43A to 43D, thus swinging the head unit 3 up and down, left and right, Actions such as raising the unit units 4A and 4B or walking are expressed by the robot 1.

  Further, the main control unit 40 recognizes the content of the user's utterance by the voice recognition process based on the voice signal S1B, and gives the voice signal S3 corresponding to the recognition to the speaker 52, so that the synthesized voice for interacting with the user is obtained. Output to the outside.

  In this way, the robot 1 can act autonomously based on the surrounding and internal conditions and can also interact with the user.

(2) Processing of main control unit 40 related to name learning function (2-1) Configuration of main control unit 40 related to name learning function Next, the name learning function installed in the robot 1 will be described.

  The robot 1 acquires the user's name through dialogue with the user, and the acquired name is detected based on the output of the microphone 51 or the CCD camera 50 at that time, and the acoustic features and face of the user's voice. The morphological features are stored in association with each other, and the appearance of a new user is recognized based on the stored data, and the new user's name, voice acoustic features, and facial morphological features are recognized. A name learning function for acquiring and storing the user's name in the same manner as described above is provided. In the following, a user who has stored the user's name, voice acoustic features, and facial morphological features in association with each other will be referred to as a “known user”, and a user who has not been stored will be referred to as a “new user”. It shall be called.

  This name learning function is realized by various processes in the main control unit 40.

  Here, when the processing contents of the main control unit 40 relating to the name learning function are functionally classified, as shown in FIG. 6, a speech recognition unit 60 that recognizes words spoken by the user, and an acoustic feature of the user's voice And a speaker recognition unit 61 for identifying and recognizing the user based on the detected acoustic feature, and detecting the morphological feature of the user's face and the user based on the detected morphological feature. A face recognition unit 62 that recognizes and recognizes, a dialogue control unit that performs various controls for learning a user's name including dialogue control with the user, a known user's name, voice acoustic features, and facial The associative storage unit that manages the association of the morphological features is divided into the speech synthesis unit 64 that generates voice signals S3 for various dialogues and sends them to the speaker 54 (FIG. 5) under the control of the dialogue control unit 63. Can Kill.

  In this case, the voice recognition unit 60 has a function of recognizing words included in the voice signal S1B in units of words by executing a predetermined voice recognition process based on the voice signal S1B from the microphone 51 (FIG. 5). These recognized words are sent to the dialogue control unit 63 as character string data D1.

  The speaker recognition unit 61 is provided with a function of storing and holding voice data obtained based on the voice signal S1B from the microphone 51 in, for example, the internal memory 40A (FIG. 5) and the stored and held voice data or the microphone 51 in real time. Using the audio data obtained based on the audio signal S1B, the acoustic characteristics of the user's voice are described in, for example, “Segregation of Speakers for Recognition and Speaker Identification (CH2977-7 / 91 / 0000-0873 S1.00 1991 IEEE)” It has a function of detecting by predetermined signal processing using the method or the like.

  The speaker recognizing unit 61 sequentially compares the detected acoustic feature data with all the known user acoustic feature data stored at that time, and any of the detected acoustic features is known. If it matches the acoustic feature of the user, the dialogue controller 63 is notified of an identifier (hereinafter referred to as SID) unique to the acoustic feature associated with the known user acoustic feature. On the other hand, when the detected acoustic feature does not match any known acoustic feature of the user, the dialogue control unit 63 is notified of SID (= −1) indicating that recognition is impossible.

  In addition, the speaker recognition unit 61 stores or holds in real time according to a new learning start command and a learning end command given from the dialog control unit 63 when the dialog control unit 63 determines that the user is a new user. In the meantime, the acoustic feature of the user's voice is detected using the obtained voice data, and the detected acoustic feature data is stored in association with a new unique SID, and this SID is stored in the dialogue control unit 63. Notify

  Further, the speaker recognizing unit 61 then uses the voice data stored and stored in real time when the start and end commands for the additional learning for the user are given from the dialogue control unit 63, Perform additional learning to additionally collect acoustic feature data.

  Further, the speaker recognizing unit 61 has a reply function for answering an inquiry about the degree of achievement of learning by designating a user from the dialogue control unit 63. Here, the learning achievement level means the degree of collection of data (in this case, acoustic feature data) used for recognizing the user, and the learning achievement level in the speaker recognition unit 61 is the voice level of the user. It is determined on the basis of a value given by a function having the time length of speech used for collecting acoustic feature data as a parameter.

  In this embodiment, as the degree of learning achievement, the “A (sufficiently learned)” level, which is a level that can be used practically for recognition, and additional learning that can be used for recognition. The “B (a little anxiety)” level, which is a better level, and the “C” level that should not be used for recognition because it is insufficient for recognition, and “C” should be used for the next opportunity. (Insufficient) ”level is set as a numerical value.

  Thus, in the speaker recognition unit 61, when the dialogue control unit 63 designates a user and inquires about the learning achievement level of the user, the voice used to collect the acoustic feature data of the user's voice Whether the user learning achievement level corresponds to “A” to “C” is determined from the value given by the function using the time length of the parameter as a parameter, and the determination result is notified to the dialog control unit 63.

  In the face recognition unit 62, for example, learning image data obtained based on the image signal from the CCD camera 50 (FIG. 5) is stored and held in, for example, the internal memory 40A (FIG. 5), and the stored image data or A function of detecting morphological features of the user's face included in an image based on the image data by predetermined signal processing using image data obtained based on the image signal S1A given in real time from the CCD camera 50; Have.

  Then, the face recognition unit 62 sequentially compares the data of the detected morphological features with the data of the morphological features of the faces of all known users stored at that time. When the morphological feature of the known user's face matches, an identifier (hereinafter referred to as FID) unique to the morphological feature associated with the known user's morphological feature is used as the dialogue control unit 63. On the other hand, if the detected morphological feature does not match any known morphological feature of the user's face, the dialogue control unit 63 is notified of FID (= -1) indicating that recognition is impossible. .

  The face recognizing unit 62 is stored and held or obtained in real time based on a new learning start command and an end command given from the dialog control unit 63 when the dialog control unit 63 determines that the user is a new user. In the meantime, the morphological features of the user's face are detected using the image data, the detected morphological feature data is stored in association with a new unique FID, and the dialogue control unit 63 is notified of this FID. .

  Further, the face recognizing unit 62 then uses the image data stored and stored in real time when an additional learning start command and end command for the user are given from the dialogue control unit 63, and the form of the user's face Perform additional learning to collect additional feature data.

  Further, like the speaker recognizing unit 61, the face recognizing unit 62 has a reply function that answers a user when a user is designated from the dialogue control unit 63 and an inquiry is made about the learning achievement level of the user. In the case of this embodiment, the learning achievement level in the face recognition unit 62 is given by a function using the number of face images based on the image signal S1A used for collecting data of morphological features of the user's face as a parameter. It is determined based on the value obtained.

  Thus, when the face recognition unit 62 designates a user from the dialogue control unit 63 and inquires about the learning achievement level of the user, the face recognition degree 62 corresponds to any of “A” to “C” based on the value. And the dialogue result is notified to the dialogue control unit 63 as a learning achievement level.

  The voice synthesizing unit 64 has a function of converting the character string data D2 given from the dialogue control unit 63 into a voice signal S3. By sending the voice signal S3 thus obtained to the speaker 54 (FIG. 5), A sound based on the sound signal S3 is output from the speaker 54.

  The associative memory unit 65 is an object composed of, for example, the internal memory 40A (FIG. 5) and software. Under the control of the dialogue control unit 63, as shown in FIG. The SID associated with the acoustic feature data of the user's voice stored in the person recognition unit 61 and the morphological feature data of the user's face stored in the face recognition unit 62 The FID is stored.

  At this time, the associative storage unit 65 is configured to store the name, SID, and FID corresponding to the same user in association with each other, thereby searching for other information from one information (name, SID, FID) for the known user. It is made to be able to do.

  The associative storage unit 65 stores the learning achievement level of the speaker recognition unit 61 for the user in association with the SID of each known user under the control of the dialogue control unit 63. Similarly, the learning achievement level of the face recognition unit 62 for each user is stored in association with the FID of each known user.

  In the case of this embodiment, the associative storage unit 65 sets the learning achievement level for each user of the speaker recognition unit 61 and the face recognition unit 62 stored for a certain time (for example, several days) after the registration or the last update. ) (E.g., “A” is lowered to “B”, and “B” is lowered to “C”) (hereinafter, learning achievement is time decayed) To be updated).

  This is because the user's face and voice change over time, so that the speaker recognition unit 61 and the face recognition unit 62 use the acoustic feature data of the user's voice and the morphological form of the face used to recognize the user. This is because it is desirable to update the feature data at regular intervals.

  The dialogue control unit 63 sequentially gives necessary character string data D2 to the voice synthesis unit 64 based on the character string data D1 from the voice recognition unit 60, a program, and the like, thereby performing an appropriate response or question to the user. Has a control function.

  Then, the dialogue control unit, based on the name of the user acquired through the response or question, and the recognition results of the speaker recognition unit 61 and the face recognition unit for the user at this time, each stored in the associative storage unit It is determined whether or not the user is a new user while referring to information such as the name, SID and FID of the known user.

  When the dialog control unit 63 determines that the user is a new user, the dialogue control unit 63 gives a new learning start command and an end command to the speaker recognition unit 61 and the face recognition unit 62, thereby recognizing these speakers. The unit 61 and the face recognition unit 62 collect and store (learn) data of the acoustic characteristics of the voice of the new user and the morphological characteristics of the face.

  Further, the dialogue control unit 63 inquires of the speaker recognition unit 61 and the face recognition unit 62 about the achievement level of learning for the user at a predetermined timing, and either one of the speaker recognition unit 61 and the face recognition unit 62 When the answer “C” is obtained, the speaker recognition unit 61 and / or the face recognition unit 61 and / or the face recognition unit 62 are given a start instruction and an end command for additional learning. While causing the unit 62 to perform additional learning, control is performed to prolong the dialog with the user during that time.

  In contrast, when the dialog control unit 63 determines that the user is a known user, the dialogue control unit 63 determines the learning achievement level of the speaker recognition unit 61 and the face recognition unit 62 for the user stored in the associative storage unit 65. And confirming the learning achievement level to the speaker recognition unit 61 and / or the face recognition unit 62 whose learning achievement level is “B” or “C”, and to the speaker recognition unit 61 and / or the face recognition unit 62 Then, additional learning is performed by giving a start instruction and an end instruction for additional learning.

  Then, after completing the additional learning of the speaker recognition unit 61 and / or the face recognition unit 62, the dialogue control unit 63 inquires the speaker recognition unit 61 and / or the face recognition unit 62 about the learning achievement level for the user at that time. Then, the corresponding learning achievement degree stored in the associative memory unit 65 is updated based on the answer from the speaker recognition unit 61 and / or the face recognition unit 62.

(2-2) Specific Processing of Dialog Control Unit 63 Regarding Name Learning Function Next, specific processing contents of the dialogue control unit 63 regarding the name learning function will be described.

  The dialogue control unit 63 executes various processes for sequentially learning new person names in accordance with the name learning process procedure RT1 shown in FIG. 8 based on the control program stored in the internal memory 40A.

  That is, when the face recognition unit 62 recognizes the user's face based on the image signal S1A from the CCD camera 50 and the FID is given from the face recognition unit 62, the dialogue control unit 63 performs this name learning processing procedure RT1. Starting in SP0, in the subsequent step SP1, the speaker recognition unit 61 and the face recognition unit 62 are controlled, whereby the speaker recognition unit 61 stores voice data based on the voice signal S1B from the microphone 51 (FIG. 5). At the same time, the face recognition unit 62 starts storing image data based on the image signal S1A from the CCD camera 50.

  Subsequently, the dialogue control unit 63 proceeds to step SP2 and determines the name of the user. Specifically, based on the FID of the user obtained in advance, whether or not the name of the user can be searched from the name, SID, and FID of each known user stored in association with the associative storage unit 65. If it is determined and searched, the character string data D2 corresponding to the name is sent to the speech synthesizer 64, so that the name of the user such as “You are Mr. XX” is searched from the FID. A voice for confirming whether or not it matches with (name corresponding to the above-mentioned XX) is output.

  When the dialogue control unit 63 recognizes the voice recognition result of the positive response such as “Yes, yes” of the user to the question based on the character string data D1 from the voice recognition unit 60, The user's name is confirmed as “Mr. XX”.

  On the other hand, when the dialogue control unit 63 can recognize the voice recognition result of a negative response such as “No, it is different” based on the character string data D1 from the voice recognition unit 60, it responds accordingly. By sending the character string data D2 to the voice synthesizer 64, for example, as shown in FIG. 9, a voice for hearing a name such as “Tell me your name” is output.

  Then, the dialogue control unit 63 obtains a voice recognition result (that is, a name) of a response such as “It is ○○” of the user with respect to such a question, and after that, the user confirms the confirmation such as “What is Mr. ○○?” When recognizing that there is a positive response based on the character string data D1 from the voice recognition unit 60, the user's name is determined to be “Mr. XX”.

  When the dialog control unit 63 determines the user's name in this way, the process proceeds to step SP3 and controls the speaker recognition unit 61 and the face recognition unit 62, thereby allowing the speaker recognition unit 61 to have a microphone. The audio data storage process based on the audio signal S1A from 51 is terminated, and the face recognition unit 62 ends the image data storage based on the image signal S1A from the CCD camera 50.

  Subsequently, the dialogue control unit 63 proceeds to step SP4, and based on each known user name stored in the associative storage unit 65 and the SID and FID corresponding to the user, the dialogue control unit 63 determines that It is determined whether there is an SID and FID associated with the user's name.

  Here, obtaining a negative result in step SP4 means that the user has not collected any acoustic feature data of the user's voice by the speaker recognition unit 61 and the face recognition unit 62 has This means that the user is a new user who has not collected any data on the morphological features of his face.

  Thus, at this time, the dialog control unit 63 proceeds to step SP8, and starts new learning using the speech data or image data stored between step SP1 to step SP3 for the speaker recognition unit 61 and the face recognition unit 62. Notify instructions. As a result, the speaker recognition unit 61 and the face recognition unit 62 newly use the voice data or image data to newly generate acoustic feature data of the user's voice or morphological feature data of the user's face. A new learning to collect and store is started.

  On the other hand, obtaining a positive result in step SP4 means that the user has already collected the acoustic feature data of the user's voice or the facial morphological feature data, respectively. Means you are a known user.

  Thus, at this time, the dialogue control unit 63 proceeds to step SP5, and the learning achievement level of the speaker recognition unit 61 for the user stored in the association storage unit 65 in association with the SID of the user, and the association storage unit 65. Confirms the learning achievement level of the face recognition unit 62 for the user stored in association with the FID of the user.

  Here, when it is confirmed that the learning achievement levels of the speaker recognition unit 61 and the face recognition unit 62 for the user are both “A”, the speaker recognition unit 61 and the face recognition are confirmed. It can be determined that the unit 62 has already collected the acoustic feature data of the user's voice or the facial morphological feature data sufficient for the user's recognition.

  Therefore, in this case, the dialog control unit 63 proceeds to step SP6, and issues a command to discard the voice data or image data stored between step SP1 to step SP3 in the speaker recognition unit 61 and the face recognition unit 63. Notice.

  The dialog control unit 63 then proceeds to step SP14 and updates the learning achievement level of the speaker recognition unit 61 and the face recognition unit 62 for the user stored in the associative storage unit 65 to “A” again. Then, the process proceeds to step SP15, and the name learning processing procedure RT1 is terminated. After that, the robot 1 performs various interactions such as dialogue and dance with the user regardless of learning for the user.

  On the other hand, as a confirmation result in step SP5, it was confirmed that the learning achievement level of one or both of the speaker recognition unit 61 and the face recognition unit 62 for the user is “B” or “C”. In this case, the speaker recognition unit 61 and / or the face recognition unit 62 has not yet collected the acoustic feature data of the user's voice or the facial morphological feature data sufficient for the user's recognition. I can judge.

  Therefore, in this case, the dialog control unit 63 proceeds to step SP7 to notify the speaker achievement unit 61 and / or the face recognition unit 62 whose learning achievement level is “B” or “C” to the learning achievement level. Then, the process proceeds to step SP8 to notify the speaker recognition unit 61 and / or the face recognition unit 62 that notified the learning achievement level (that is, the learning achievement level for the user is still “B” or “C”). An instruction to start additional learning using audio data or image data stored between steps SP1 to SP3 is notified.

  As a result, in the speaker recognition unit 61 and / or the face recognition unit 62, the acoustic feature data of the user's voice or the morphological feature data of the user's voice is used as a starting point with the learning achievement level notified in step SP7. Incremental learning using such audio data or image data is started so as to collect further from the current collection state where collection is not yet sufficient.

  Further, when the dialog control unit 63 thereafter receives a notification from the speaker recognition unit 61 and / or the face recognition unit 62 that learning using the voice data or image data stored between steps SP1 to SP3 has been completed. Then, proceeding to step SP9, the speaker recognition unit 61 and / or the face recognition unit 62 that performed the learning are inquired about the degree of learning achievement for the user, and the answer of the speaker recognition unit 61 and / or the face recognition unit 62 for the inquiry is received. It is determined whether or not both are “A” or “B”.

  Here, obtaining a positive result in step SP9 means that the acoustic feature data of the user's voice is such that both the speaker recognition unit 61 and the face recognition unit 62 that have learned can recognize the user. Or it means that the data of the morphological features of the face have been collected and stored (fully learned).

  Thus, at this time, the dialog control unit 63 proceeds to step SP14, and the learning achievement levels of the speaker recognition unit 61 and the face recognition unit 62 for the user stored in the associative storage unit 65 are not learned. Is updated to the learning achievement level confirmed in step SP5, and the learning achievement level is updated to the learning achievement level obtained in step SP9. Thereafter, the process proceeds to step SP15 and the name learning processing procedure RT1 is terminated. After that, the robot 1 performs various interactions such as dialogue and dance with the user regardless of learning for the user.

  On the other hand, obtaining a negative result in step SP9 means that at least one of the learned speaker recognition unit 61 and / or face recognition unit 62 can still recognize the user so that the acoustic feature of the user's voice can be recognized. Means that data or facial morphological feature data is not collected and stored (not fully learned).

  Thus, at this time, the dialogue control unit 63 proceeds to step SP10, and the voice data or CCD camera obtained based on the voice signal S1B given in real time from the microphone 51 to the speaker recognition unit 61 or the face recognition unit 62. 50, an instruction to start additional learning using image data obtained based on the image signal S1A given in real time is sent.

  As a result, in the speaker recognition unit 61 and / or the face recognition unit 62, the voice of the user is obtained using the voice data or the image data using the learning achievement degree notified to the dialogue control unit 63 in step SP9 as a starting point. Additional learning is started to additionally collect and store the acoustic feature data or the morphological feature data of the user's face.

  Further, the dialogue control unit 63 thereafter proceeds to step SP11 and executes a process for extending the dialogue with the user. Specifically, in order to prevent the robot 1 from recognizing that the robot 1 is learning about the user, for example, as shown in FIG. 10, “Will you be my friend?”, “Thank you! Then, talk to the user from the robot 1 such as “What do you want to hear about Mr. OO?” And “What is Mr. OO's favorite food?” The character string data D2 to be transmitted is sent to the voice synthesis unit 64 while being selected according to the voice recognition result of the user's utterance content by the voice recognition unit 60.

  Further, the dialogue control unit 63 thereafter proceeds to step SP12 at a predetermined timing, inquires the speaker recognition unit 61 and / or the face recognition unit 62 that notified the start instruction of the additional learning about the learning achievement level for the user. It is determined whether the answer of the speaker recognition unit 61 and / or the face recognition unit 62 is either “A” or “B”.

  When the dialog control unit obtains a negative result in step SP12, it returns to step SP11, and thereafter repeats the loop of step SP11-SP12-SP11 until a positive result is obtained in step SP12.

  Then, the dialogue control unit 63 collects the acoustic feature data of the user's voice or the morphological feature data of the face so that both the speaker recognition unit 61 and the face recognition unit 62 can recognize the user thereafter. If a positive result is obtained in step SP12 by completing the storage, the process proceeds to step SP13 to notify the speaker recognition unit 61 and / or the face recognition unit 62 performing additional learning of an additional learning end command.

  Further, the dialogue control unit 63 thereafter proceeds to step SP14 to determine the learning achievement level of the speaker recognition unit 61 and the face recognition unit 62 for the user stored in the associative storage unit 65 between step SP10 and step SP13. For those not performing additional learning, the learning achievement level confirmed in step SP5 or step 9 is updated. For those for which additional learning between steps SP10 to SP13 is performed, the learning achievement level obtained in step SP12 is updated. Then, the process proceeds to step SP15, and the name learning processing procedure RT1 is terminated. After that, the robot 1 performs various interactions such as dialogue and dance with the user regardless of learning for the user.

  In this way, the robot 1 can perform new learning for a new user or additional learning for a known user under the control of the dialogue control unit 63, and thereby the name of the new user can be changed. Learning can be sequentially performed in association with acoustic feature data and facial morphological feature data.

(2-3) Error processing during name learning processing Next, during the name learning processing in accordance with the name learning processing procedure ST1, learning for the user is being performed for some reason, such as the user who is learning has left. A process in the case where it must be terminated will be described.

If you have to finish learning for the user,
(1) In step SP2 of the name learning processing procedure ST1, learning must be terminated before the user's name is determined
(2) Between the step SP1 to step SP3 of the name learning processing procedure ST1, the speaker recognition unit or the face recognition unit has to end the learning before starting the storage and holding of the voice data or the image data after the start. Case
(3) Between step SP4 and step SP7 of the name learning processing procedure ST1, the speaker recognition unit 61 and the face recognition unit 62 must finish learning before starting learning using the stored voice data and image data. If you don't get
(4) Between step SP8 and step SP9 of the name learning processing procedure RT1, the speaker recognition unit 61 and the face recognition unit 62 end the learning during new learning or additional learning using stored voice data or image data. If you must
(5) Between step SP10 to step SP13 of the name learning processing procedure RT1, the speaker recognition unit 61 and the face recognition unit 62 finish learning during additional learning using the audio signal S1B or the image signal S1A obtained in real time. There are five possible patterns that must be considered. In the following, these patterns are referred to as unlearnable patterns (1) to (5), respectively.

  In these cases, it is determined that learning for the user is unsuccessful, and the speech data or image data for learning stored by the speaker recognition unit 61 or the face recognition unit 62 so far is discarded, or the speaker is in the meantime. It may be possible to discard the acoustic feature data of the user's voice and the facial morphological feature data collected by the recognition unit 61 and the face recognition unit 62. However, in this case, the learning voice data collected so far Alternatively, image data, acoustic feature data of the user's voice, or facial morphological feature data is wasted.

  Therefore, in the case of the above-described unlearnable pattern (1), the robot 1 discards the learning voice data and image data stored so far, while the unlearnable pattern (2) In the case of the impossible pattern (3), the necessary speaker recognition unit 61 and / or face recognition unit 62 are made to learn using the learning voice data and image data stored up to that time, and the learning is performed. When the learning achievement level is “A” or “B”, such learning is treated as effective, and when the learning achievement level is still “C”, such learning is treated as invalid.

  Further, in the robot 1, in the case of the above-described unlearnable pattern (4) or unlearnable pattern (5), the speaker recognition unit 61 and the face recognition unit 62 at the time when learning must be ended. Depending on the degree of learning achievement, when the degree of learning achievement is “A” or “B”, such learning is treated as effective, and when the degree of learning achievement is “C”, such learning is invalid. It is made to handle as.

  Here, such error processing is performed under the control of the dialogue control unit 63 according to the error processing procedure RT2 shown in FIG.

  In practice, the dialogue control unit 63 continues learning for the user when the name learning processing procedure RT1 described above is executed, such as an error notification indicating that the user to be recognized cannot be detected from the face recognition unit 61, for example. When a predetermined condition that cannot be performed occurs, the name learning processing procedure RT1 is terminated and the error processing procedure RT2 is started in step SP20. In the subsequent step SP21, the name of the user who has been targeted so far is determined. Judge whether or not.

  Obtaining a negative result in step SP21 means that a situation has arisen in which learning has to be terminated before such an error determines the user's name in step SP2 of the name learning processing procedure RT1 (unlearnable pattern). In the case of (1)), the dialogue control unit 63 proceeds to step SP22 at this time, and the learning speech stored up to that point after the start of the name learning processing procedure RT1 in the speaker recognition unit 61 and the face recognition unit 62 A command to discard the data or image data is notified, and then the process proceeds to step SP34 to end the error processing procedure RT2.

  On the other hand, when the dialogue control unit 63 obtains a positive result in step SP21, the dialogue control unit 63 proceeds to step SP23 and gives an instruction to notify the speaker recognition unit 61 and the face recognition unit 62 of its own state, Based on the responses of the speaker recognizing unit 61 and the face recognizing unit 62 to this, the current states of the speaker recognizing unit 61 and the face recognizing unit 62 are confirmed.

  Then, in step SP23, the dialog control unit 63 confirms that the speaker recognition unit 61 and the face recognition unit 62 have started storing and storing voice data or image data, but before the end (learning impossible pattern (2 )), The process proceeds to step SP24 to notify the speaker recognition unit 61 and the face recognition unit 62 of an instruction to end the storage of the speech data or image data for learning, and then step SP25 to step SP29. Are processed in the same manner as Step SP4 to Step SP8 of the name learning processing procedure RT1 described above.

  When the dialogue control unit 63 finishes processing step SP25 to step SP29, the dialogue control unit 63 proceeds to step SP30 and inquires the speaker recognition unit 61 and the face recognition unit 62 about the achievement level of learning for the user. The learning achievement levels of the speaker recognition unit 61 and the face recognition unit 62 are acquired, and then the process proceeds to step SP31, where both the acquired learning achievement levels of the speaker recognition unit 61 and the face recognition unit 62 for the user are “C”. Is determined.

  Here, obtaining a negative result in step SP31 means that both the speaker recognition unit 61 and the face recognition unit 62 can recognize the user's voice acoustic feature data or the face to such an extent that the user can be recognized. It means that the data of the morphological features of can be collected.

  Thus, at this time, the dialog control unit 63 proceeds to step SP33, and when the user is a new user, the SID and FID newly issued from the speaker recognition unit 61 and the face recognition unit 62, respectively, The name of the user determined in step SP2 of the name learning processing procedure RT1 and the learning achievement levels of the speaker recognition unit 61 and the face recognition unit 62 for the user acquired in step SP30 of the error processing procedure RT2 are as described above. And stored in the associative storage unit 65.

  If the user is a known user, the dialogue control unit 63 determines the learning achievement level of the speaker recognition unit 61 and the face recognition unit 62 for the user stored in the associative storage unit 65 by using step SP30. Is updated to the current learning achievement level of the speaker recognition unit 61 and the face recognition unit 62 for the user acquired in step S2. Then, the dialogue control unit 63 proceeds to step SP34 and ends this error processing procedure RT2.

  On the other hand, obtaining a negative result in step SP31 means that either one or both of the speaker recognition unit 61 and the face recognition unit 62 can recognize the user's voice to such an extent that the user can be recognized. This means that feature data or facial morphological feature data has not been collected.

  Thus, at this time, the dialogue control unit 63 proceeds to step SP32 and asks the speaker recognition unit 61 and the face recognition unit 62 for the voice associated with the SID or FID in the subsequent speaker recognition processing and recognition processing. An instruction is given not to use acoustic feature data or facial morphological feature data. As a result, in the speaker recognizing unit 61 and the face recognizing unit 62, the acoustic achievement data of the voice and the morphological characteristic data of the face have a learning achievement level “B” or “ It will not be used for speaker recognition processing or face recognition processing until “A” is reached.

  Then, the dialogue control unit 63 proceeds to step SP33, and when the user is a new user, the SID and FID newly issued from the speaker recognition unit 61 and the face recognition unit 62, respectively, The name of the user determined in step SP2 of the name learning processing procedure RT1 and the learning achievement levels of the speaker recognition unit 61 and the face recognition unit 62 for the user acquired in step SP30 of the error processing procedure RT2 are as described above. And stored in the associative storage unit 65.

  If the user is a known user, the dialogue control unit 63 determines the learning achievement level of the speaker recognition unit 61 and the face recognition unit 62 for the user stored in the associative storage unit 65 by using step SP30. Is updated to the current learning achievement level of the speaker recognition unit 61 and the face recognition unit 62 for the user acquired in step S2. Then, the dialogue control unit 63 proceeds to step SP34 and ends this error processing procedure RT2.

  Therefore, in the case of this robot 1, even if the learning achievement level of the speaker recognition unit 61 or the face recognition unit 62 for the user is “C”, the name of the user obtained at that time, the corresponding SID, Since the FID and the learning achievement level of the speaker recognition unit 61 and / or the face recognition unit 62 for the user are associated and stored in the associative storage unit 65 (step SP31 to step SP33), for example, the name is given next. When the user is recognized, for example, as shown in FIG. 12, an utterance that the user has been met before can be made.

  In step SP32, the acoustic feature data of the user's voice and the facial morphological feature data collected at that time from the dialogue control unit 63 to the speaker recognition unit 61 and / or the face recognition unit 62 are not used. Even when an instruction is given, as described above for step SP7 and step SP8 of the name learning processing procedure RT1, the next additional learning of the speaker recognition unit 61 and / or face recognition unit 62 for the user is collected. Therefore, the speaker recognition unit 61 and the face recognition unit 62 can efficiently perform learning because it starts on the premise of the existence of the acoustic feature data of the user's voice and the morphological feature data of the face. it can.

  On the other hand, in step SP23, the dialog control unit 63 finishes storing the voice data or the image data as the current state of the speaker recognition unit 61 and the face recognition unit 62, but before starting the learning using this. If it is confirmed (in the case of unlearnable pattern (3)), the process proceeds to step SP25, and the SID and FID associated with the name based on the name of the user determined in step SP2 of the name learning processing procedure RT1. Is stored in the associative storage unit 65, and thereafter, step SP26 to step SP34 are processed in the same manner as described above.

  In step SP23, when the dialog control unit 63 confirms that the current state of the speaker recognition unit 61 and the face recognition unit 62 is learning using the stored voice data or image data, the (unlearnable pattern) In the case of (4)), the process proceeds to step SP30, and the learning of the speaker recognition unit 61 and the face recognition unit 62 for the user is made by inquiring the speaker recognition unit 61 and the face recognition unit 62 about the degree of learning for the user. The achievement level is acquired, and thereafter, step SP31 to step SP34 are processed in the same manner as described above.

  Further, in step SP23, the dialog control unit 63 is provided with audio data based on the audio signal S1B provided in real time from the microphone 51 or in real time from the CCD camera 50 as the current state of the speaker recognition unit 61 and the face recognition unit 62. If it is confirmed that the additional learning using the image data based on the image signal S1A is being performed (in the case of the learning impossible pattern (5)), the process proceeds to step SP35 to be added to the speaker recognition unit 61 and / or the face recognition unit 62. Notify learning end command.

  The dialogue control unit 63 then proceeds to step SP30 and inquires of the speaker recognition unit 61 and the face recognition unit 62 about the degree of learning achievement for the user, so that the speaker recognition unit 61 and the face recognition unit 62 for the user. The learning achievement level is acquired, and thereafter, step SP31 to step SP34 are processed in the same manner as described above.

  In this way, in the robot 1, even when the user is forced to finish learning while learning, the robot 1 collects the learning data collected up to that time under the control of the dialogue control unit 63. The user can be performed using the voice data or image data of the user, the acoustic feature data of the user's voice, or the morphological feature data of the face.

(3) Operation and effect of the present embodiment In the above configuration, the robot 1 acquires the name of the user through dialogue with a new user, and the name is used for the microphone 51 (FIG. 5) and the CCD camera 50. (FIG. 5) is stored in association with the acoustic characteristics of the user's voice and the morphological characteristics of the face detected based on the output of (FIG. 5), and the name is not acquired based on the stored various data Further, the user's name is learned by recognizing the appearance of the new user and acquiring and storing the new user's name, the acoustic characteristics of the voice and the morphological characteristics of the face in the same manner as described above.

  Therefore, the robot 1 does not allow the user to recognize what he / she is learning, and through the dialogue with the user, the name of the new user, the acoustic features of the user's voice, and the facial Morphological features can be learned naturally.

  Further, in the robot 1, when performing such learning, the speaker recognition unit 61 and the face recognition unit 62 learn voice data for learning the target user's voice acoustic characteristics or the form of the user's face. Learning image data for learning a characteristic feature is stored in advance before the name of the user is determined, and learning is performed using the voice data and image data. If there is a situation in which it is unavoidable to end the process, there is a possibility that the user can be learned, and the user can be more efficiently learned.

  Further, the robot 1 holds the acoustic feature data of the user's voice and the morphological feature data of the face, which are the learning results so far, even when the user's learning must be terminated in the middle of the learning. Since the next learning for the user is started in the middle of the learning, the learning can be performed efficiently.

  Furthermore, the robot 1 expands the dialogue with the user when the learning is not performed by the learning using the voice data of the target user's voice and the face image data stored in advance. Since it is supposed to continue, there are many opportunities to complete learning for the user by one dialogue, and accordingly, it is possible to effectively prevent the occurrence of annoying interaction for the user who asks the same user for the name many times.

  According to the above configuration, the user's name is acquired through dialogue with a new user, and the name is detected based on the output of the microphone 51 or the CCD camera 50. It is stored in association with each data of morphological characteristics, and based on these stored various data, the appearance of a new user who has not obtained a name is recognized, and the acoustic characteristics of the name and voice of the new user are recognized. In addition, the user's name is learned by acquiring and storing the morphological features of the face in the same manner as described above, so that a human can do it normally without recognizing that the user is learning. In addition, it is possible to naturally learn the name of a new user, the acoustic characteristics of the user's voice, and the morphological characteristics of the face through user interaction, thus providing entertainment. A robot that can improve the cement resistance to remarkably can be realized.

(4) Other Embodiments In the above-described embodiment, the case where the present invention is applied to the biped robot 1 configured as shown in FIG. 1 has been described. The present invention is not limited to this, and the present invention can be widely applied to various other types of robot apparatuses and various other apparatuses other than the robot apparatus.

  In the above-described embodiment, the case where the learning target is a human (user) has been described. However, the present invention is not limited to this, and the present invention can also be applied to a case where an object other than a human is targeted for name learning. it can.

  In this case, in the above-described embodiment, the person is recognized from the acoustic characteristics of the target person's voice and the morphological characteristics of the face, and the person is a new person based on the recognition results. Although the case where it is determined whether or not there is described, the present invention is not limited to this, but instead of this, or in addition to this, an individual such as a body type or smell is biologically identified. The person may be recognized from various types of possible features, and it may be determined whether the person is a new person based on the recognition results. If the name learning target is an object other than a person, the object is recognized from a plurality of types of features that can identify the object such as color, shape, pattern, size, etc. It may be determined whether or not the object is a new object. In these cases, a plurality of different predetermined features of the object are detected, and the target object is recognized based on the detection result and the data of the corresponding features of the known object stored in advance. The recognition means may be provided.

  Further, in the above-described embodiment, the case where the learning achievement level is set to the three levels “A” to “C” has been described, but the present invention is not limited to this, and is set to two levels or four levels or more. You may do it.

  Furthermore, in the above-described embodiment, the case where the dialog control unit 61 simply stretches the dialog during the additional learning described above for step SP10 to step SP13 of the name learning processing procedure RT1, for example, has been described. In addition to the above, when extending the dialogue with the user, the dialogue control unit 63 makes the dialogue that the recognition means (speaker recognition unit 61 and face recognition unit 62) insufficiently learned for the user can easily learn. May be processed so as to be generated between them, and additional learning can be performed more efficiently.

  In practice, for example, in the case of the embodiment, if the speaker recognition unit 61 performs additional learning, the dialog is expanded while generating a dialog that makes the user speak as much as possible, and additional learning is performed. When the face recognition unit 62 performs, a dialogue that moves the user's face, such as “Would you turn to the right?” Is generated so that face images from various directions of the user can be acquired as much as possible. However, the dialogue should be expanded.

  Furthermore, in the above-described embodiment, the case where the associative memory unit 65 attenuates each stored learning achievement degree for example every several days has been described. However, the present invention is not limited to this, and time attenuation is performed. The time interval may be other than a few days, and the dialog control unit 63 may manage and cause such time decay.

  Furthermore, in the above-described embodiment, a dialog unit that acquires the name of the target object from the user through a dialog with the user, the name of the target object acquired by the dialog unit, and each recognition for the target object When it is determined that the target object is new based on the recognition result of the means and the association information stored in the storage means, the necessary recognition means learns the corresponding characteristics of the target object, and Although the case where the control unit that newly stores the association information about the target object in the storage unit is configured by the dialogue control unit 63 that is the same functional module has been described, the present invention is not limited thereto. These may be configured as separate modules.

  Further, in the above-described embodiment, the association information that associates the names of known users and the recognition results of the respective recognition units (speech recognition unit 60, speaker recognition unit 61, and face recognition unit 62) for the known users is stored. The case where the storage means is configured from the internal memory 40A and software has been described. However, the present invention is not limited to this, and for example, a functional part for storing association information in the storage means other than the internal memory 40A. For example, it may be replaced with a means having other storage functions such as a compact disk which can be freely read and written.

  The present invention can be applied to various devices having a learning function such as an entertainment robot.

It is a perspective view with which it uses for description of the external appearance structure of the robot by this Embodiment. It is a perspective view with which it uses for description of the external appearance structure of the robot by this Embodiment. It is a conceptual diagram with which it uses for description of the external appearance structure of the robot by this Embodiment. It is a block diagram with which it uses for description of the internal structure of the robot by this Embodiment. It is a block diagram with which it uses for description of the internal structure of the robot by this Embodiment. It is a block diagram with which it uses for description of the function of the main control part regarding a name learning function. It is a conceptual diagram with which it uses for description of the correlation of the various information in an associative memory part. It is a flowchart which shows a name learning process procedure. It is a basic diagram which shows the example of a dialogue between a robot and a user. It is a basic diagram which shows the example of a dialogue between a robot and a user. It is a flowchart which shows an error processing procedure. It is a basic diagram which shows the example of a dialogue between a robot and a user.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Robot, 40 ... Main control part, 50 ... CCD camera, 51 ... Microphone, 52 ... Speaker, 60 ... Voice recognition part, 61 ... Speaker recognition part, 62 ... Face recognition part, 63: Dialogue control unit, 64: Speech synthesis unit, 65: Associative memory unit, S1A: Image signal, S1B, S3 ... Audio signal, D1, D2: Character string data, RT1: Name learning process Procedure, RT2 ... Error handling procedure.

Claims (4)

A storage unit for storing the user's name and characteristics in association with each other;
A user detection unit for detecting a user based on an image obtained from a camera;
When the user is detected by the user detection unit, before asking the user for a name, the user's characteristics are detected using at least one of the image obtained from the camera and the sound obtained from the microphone, A recognition unit that recognizes the name of the user by comparing the detected feature of the user with the feature already stored in the storage unit;
Through dialogue with the user detected by the user detection unit, by asking the user for the name, the dialogue unit for obtaining the name of the user;
When it is determined that the user is a new user because the name of the user acquired by the dialogue unit and the characteristics of the user detected by the recognition unit are not stored in the storage unit, the image And the recognition unit learns the user's characteristics using at least one of the voices and associates the user's characteristics obtained as a result of the learning with the user's name acquired by the dialogue unit. A control unit to be stored in the storage unit;
With
The recognition unit
The learning achievement level of the feature is determined based on at least one of the number of images and the time length of the sound obtained at the time of detecting the feature and used for the detection of the feature,
The dialogue section above
Using learning achievement degree the recognition section determines determines whether insufficient learning of the feature by the recognition unit, sometimes learning is judged to be insufficient, for extending the dialogue with the user A learning device that executes processing. The recognition unit
Together to learn the user's morphological characteristics by using the image picture, it is adapted to learn the acoustic features of a user with the above sound voices,
The dialogue section above
When the dialogue with the user is extended, if one of the morphological feature and the acoustic feature is not sufficiently learned by the recognizing unit, the dialogue that facilitates the learning of the one with insufficient learning The learning apparatus according to claim 1, wherein a process for generating is executed. When a user is detected based on an image obtained from the camera, before asking the user for the name, the user's characteristics are obtained using at least one of the image obtained from the camera and the sound obtained from the microphone. A first step of recognizing the user's name by comparing the detected feature of the user with the feature already stored in the storage unit storing the user's name and the feature in association with each other When,
A second step of obtaining the name of the user by asking the name of the user through interaction with the detected user;
When it is determined that the user is a new user because the name of the user acquired in the second step and the characteristics of the user detected in the first step are not stored in the storage unit The feature of the user is learned using at least one of the image and the voice, and the feature of the user obtained as a result of the learning is associated with the name of the user acquired in the second step. A third step for storing in the storage unit;
With
In the third step,
The learning achievement level of the feature is determined based on at least one of the number of images and the time length of the sound obtained when the feature is detected and used for the detection of the feature, and the determined learning achievement level is used. A learning method that determines whether learning of the feature is insufficient, and executes processing for extending a dialog with the user when it is determined that learning is insufficient. A storage unit for storing the user's name and characteristics in association with each other;
A user detection unit for detecting a user based on an image obtained from a camera;
When the user is detected by the user detection unit, before asking the user for a name, the user's characteristics are detected using at least one of the image obtained from the camera and the sound obtained from the microphone, A recognition unit that recognizes the name of the user by comparing the detected feature of the user with the feature already stored in the storage unit;
Through dialogue with the user detected by the user detection unit, by asking the user for the name, the dialogue unit for obtaining the name of the user;
When it is determined that the user is a new user because the name of the user acquired by the dialogue unit and the characteristics of the user detected by the recognition unit are not stored in the storage unit, the image And the recognition unit learns the user's characteristics using at least one of the voices and associates the user's characteristics obtained as a result of the learning with the user's name acquired by the dialogue unit. A control unit to be stored in the storage unit;
With
The recognition unit
The learning achievement level of the feature is determined based on at least one of the number of images and the time length of the sound obtained at the time of detecting the feature and used for the detection of the feature,
The dialogue section above
Using learning achievement degree the recognition section determines determines whether insufficient learning of the feature by the recognition unit, sometimes learning is judged to be insufficient, for extending the dialogue with the user A robotic device that performs processing.

Images