JP3919726B2 - Learning apparatus and method - Google Patents

Description

  The present invention relates to a learning device that can be used in a robot or a doll toy.

  Conventional toy robots are mainly for realizing mechanical movement. Therefore, it is only used as a toy for small children that repeats a simple movement pattern.

  From now on, since the population of adults and elderly people will be larger than children, robots that can be enjoyed by adults and elderly people as toys are also needed. However, in order to artificially perform intelligent movements and realize complex actions, we must still wait for future technological development.

Also, robots that identify the user's face have been proposed (see, for example, Patent Document 1).
JP 2002-157596 A

  However, even if the face identification device is built in the robot as described above, it is not disclosed how to effectively use the identified information.

  Therefore, in the present invention, in learning devices built in robots, doll toys, and the like, by using face identification information to realize low-level intelligent movements with the motif of heroes of babies and anime, adults and elderly people But the aim is to provide something that can be enjoyed.

The present invention relates to a learning device built in a robot, a doll, or a toy, and an object recognition unit for recognizing an object from an image input from an image input unit, and a voice input device in a registration mode for registering the object Voice recognition means for recognizing a noun from the voice input from the voice, voice data including a symbol string relating to the noun recognized by the voice recognition means in the registration mode, and image data relating to the object recognized by the object recognition means. In the recognition mode for recognizing whether or not the registered object is a registered object and the registered storage means for storing in association with each other, when the object recognized by the object recognizing means matches the object stored in the registered storage means , Based on the speech data stored corresponding to the stored object and the speech segment of the segment dictionary stored in advance. Has a speech synthesis means for synthesizing speech, the Te, the object recognition unit recognizes the human face, the voice recognition unit recognizes demonstrative pronoun, particle, at least keywords belong to one of the auxiliary verb, Further, a character string excluding the recognized keyword in the input speech is recognized as a noun, and the recognized speech data includes pitch information indicating a tone of inflection, and the speech synthesis means Synthesizes speech from the character string of the noun, the pitch information, and the segment of the segment dictionary, and the segment dictionary has an appearance of a robot, a doll, or a toy that incorporates the learning device It is comprised from the segment matched to this, It is the learning apparatus characterized by the above-mentioned.

  The invention according to claim 2 is the learning apparatus according to claim 1, wherein the object recognition means is face recognition means for recognizing a human face.

The learning device of the present invention will be described.

  In the registration mode, a predetermined object such as a face is recognized from the image input from the image input means, and a noun is recognized from the voice input from the voice input device. Then, the voice data including the symbol string related to the recognized noun and the image data related to the recognized object are stored in association with each other.

  In the recognition mode, when the recognized object matches the stored object, the speech data stored in correspondence with the stored object and the speech unit of the segment dictionary stored in advance Synthesize speech based on

In addition, when a keyword that is a word belonging to a pronoun, a particle, an auxiliary verb, or the like is recognized, only the part that seems to be a noun can be extracted from the registered symbol string by removing the keyword from the registered symbol string. .

In addition, by reproducing the inflection from the pitch information , it can be heard that a human is really speaking.

You can also hear the robot, doll, or toy really speak.

According to the present invention, it is possible to provide a learning device that can be enjoyed by adults and elderly people by realizing low-level intelligent movements, and those incorporating the present invention are within the scope of future toy robots. For example, it will imitate how a baby remembers people and objects, and the child grows up, becomes independent, has no children, or feels lonely It can be expected to serve as a supporter for the elderly who are doing so.

  Alternatively, it can be expected to be attractive to children who are conventional users by imitating the manner in which the main character of the animation remembers the name of the owner who is the owner.

  An embodiment of the present invention will be described below with reference to FIGS.

(1) Configuration of Baby Robot 100 This embodiment is a typical embodiment of the present invention that is implemented using a movement of a baby learning a mother's face as a motif.

  FIG. 4 is an external view of the baby robot 100 incorporating the learning device 10 of the present embodiment. This baby robot includes a microphone 102 and a camera 104 such as a video camera or a digital camera. However, since the camera 104 and the microphone 102 do not need to be in the position of eyes or ears as in the case of a real baby, they may be in a hair that is not noticeable or a part of clothing. Preferably, it may be made to resemble clothing buttons.

(2) Configuration of Learning Device 10 FIG. 1 is a block diagram showing the learning device 10 of the present embodiment.

  The learning device 10 includes a voice recognition unit 12, an object recognition unit 14, a registration storage unit 16, a voice synthesis unit 18, and a mode change switch 20.

  The learning device 10 simulates a simple learning process by combining voice recognition and object recognition, and registering the face the baby robot 100 is viewing and the voice being heard in association with each other.

  First, as shown in the flowchart of FIG. 2, in the “registration mode”, the user asks the baby robot 100 his / her name while showing his / her face to the baby robot 100. As a most typical example, a woman shows her face to the baby robot 100 and asks her name “Mama”. As a result, the baby robot 100 recognizes the voice, converts it into a symbol string such as a character string, stores it, and registers a face dictionary in association with the symbol string by learning object recognition.

  Next, as shown in the flowchart of FIG. 3, in the “recognition mode”, when the baby robot 100 sees the face shown before again, the baby robot 100 utters the voice symbol heard simultaneously with the face with its own voice. To do. As the most typical example, say “Mama” with a baby-like voice.

  In this way, the baby robot 100 can learn the human face and name and imitate the action of speaking the learned name when viewing the same face.

  Hereinafter, the structure of each part 12-16 is demonstrated, respectively. In addition, each function of these structures 12-16 is implement | achieved by the program memorize | stored in the computer.

(2-1) Mode switch 20
The mode change switch 20 switches the function of the learning device 10 between the registration mode and the recognition mode, and is provided on the back of the baby robot 100 or the like.

(2-2) Voice recognition unit 12
(2-2-1) The voice recognition unit 12 of the first embodiment
The voice recognition unit 12 functions in the registration mode. The microphone 102, an amplifier that amplifies the signal from the microphone 102 to an appropriate level, a filter that cuts unnecessary bands, and an amplified analog signal are digitized. An A / D converter that recognizes the digitized signal using a known algorithm such as an HMM.

  In the registration mode, the name uttered by the user is stored in the speech dictionary in the registration storage unit 16.

(2-2-2) Voice recognition unit 12 of the second embodiment
If the user uses according to the convention that he speaks only the name of the object, the speech recognition unit 12 of the first embodiment described above can be realized.

  However, in general, unnecessary words such as “is” or “this is” may be entered in addition to the name of the object, such as “I am a mama” or “this is a dog”. In order to cope with this, as shown in FIG. 5, the speech recognition unit 12 is further provided with a keyword recognition unit 13.

  When a keyword that is a word belonging to a pronoun, a particle, an auxiliary verb, or the like is recognized, the keyword recognizing unit 13 removes the keyword from the registered symbol string and removes only the part that seems to be a noun depending on the positional relationship with the keyword. Take out. Then, the voice data of the symbol string of the name from which the keyword is removed is stored in the registration dictionary 16 in the voice dictionary.

  FIG. 6 is a flowchart showing an operation when it is assumed that the last keyword is “is”.

  It should be noted that even when the keyword “This is” or “I am” is first assumed, it is possible to cope with the same operation. In this case, since the position of the keyword is the first, it is necessary to set a part “front” in the flowchart to “back”.

(2-3) Configuration of Object Recognition Unit 14 and Registration Storage Unit 16 Next, the configuration of the object recognition unit 14 and the registration storage unit 16 will be described.

  The object recognition unit 14 and the registration storage unit 16 will be described separately for a face recognition registration mode and a recognition mode.

(2-3-1) Registration Mode A flowchart of the registration mode for face recognition is shown in FIG.

  In the registration mode, M ′ face images showing a person's face are captured, and M principal components smaller than M ′ are extracted and stored in the image dictionary of the registration storage unit 16 as image data. For this, a well-known principal component analysis method can be applied.

  Then, as shown in FIG. 11, the registration storage unit 16 stores the image dictionary and the voice dictionary in association with each other. In this case, the voice dictionary spoken at the same time and the recognized face image dictionary are stored in correspondence with the recognized time.

  As for the operation when the same name as the previously registered name is registered, two names and image data may be registered completely independently, or the previous one may be replaced.

  Further, previously registered image data may be updated with new data. At that time, not only the image data but also the correlation matrix used for the principal component analysis may be recorded simultaneously. A flowchart of this process is shown in FIG.

  In FIG. 10, f is a vector in which the density values of the meshes are arranged when the input image is represented by meshes. The symbols <·, ·> are called Schatten products, and are operations that create a matrix by multiplying a vector and its transpose. K and K ′ are a kind of correlation matrix obtained by adding the calculation results with weights and called a characteristic kernel. Here, only one K is written, but generally there are the same number of registered categories for each of a plurality of registered categories. K ′ is for a newly registered category and is treated the same as K after registration.

(2-3-2) Recognition Mode In the recognition mode, N ′ face images are captured, N principal components less than N ′ are extracted, an N-dimensional subspace is formed, and this is stored in the registration storage unit 16. An angle between the subspace and the M-dimensional subspace of the image data is calculated.

  To calibrate the subspace, the same principal component analysis as in the registration mode can be used. Moreover, in order to measure the angle between subspaces, a method called a mutual subspace method is used.

または

なる行列Ｘの最大固有値として、最も小さい角度を計算することができることが知られている。角度をθ_１ とすると、最大固有値λ_１ との関係は、λ_１ ＝ｃｏｓ^２ θ_１ である。この技術に関しては、特許文献２（特開平１１−２６５４５２号公報参照、前田 賢一，山口 修，福井 和広：“物体認識装置および物体認識方法”）。 Specifically, when M principal components are {φ _m } and N principal components are {ψ _n }, X = (x _ij )
However,

Or

It is known that the smallest angle can be calculated as the maximum eigenvalue of the matrix X. When the angle is θ ₁ , the relationship with the maximum eigenvalue λ ₁ is λ ₁ = cos ² θ ₁ . Regarding this technique, Patent Document 2 (see Japanese Patent Application Laid-Open No. 11-265452, Kenichi Maeda, Osamu Yamaguchi, Kazuhiro Fukui: “Object Recognition Device and Object Recognition Method”).

(2-4) Configuration of the voice synthesizing unit 18 In the recognition mode, when the result of the image recognition is stored in the registration storage unit 16, the voice data is called from the voice dictionary stored in association therewith, and the voice Performs speech synthesis using a symbol string of data.

  In speech synthesis, a process of synthesizing speech from a symbol string is performed. To this end, in addition to the symbol string, speech segments (sound material that synthesizes individual phonemes) and inflection information are required. .

  As phoneme segments, those registered in advance are used. For example, using pieces collected from a baby's voice makes it possible to synthesize with the baby's voice. As described above, when imitating a baby as described above, the one collected in advance is used, but in the case of the main character of the anime, the voice actor uttered it. Collect and use fragments from speech.

  However, if the registered symbol string is subjected to speech synthesis, speech without robotic intonation will be generated. In order to solve this problem, it is effective that the speech recognition unit 12 learns intonation. This can be done by extracting the pitch of the voice and storing it as pitch information while associating it with the symbol. Then, by reproducing the inflection from the pitch information, the speech synthesizer 18 can hear the person really speaking. A flowchart of the registration state is shown in FIG.

(Example of change)
The present invention is not limited to the above-described embodiment, and various modifications can be considered without changing the gist thereof.

  For example, a cartoon hero's robot may remember the owner's name.

  In the above embodiment, the user's face is recognized, but an object such as a ball may be recognized instead. For example, a dog-shaped robot can be made to remember a ball or bone.

  The present invention is used by being incorporated in a robot, a human or baby doll, an animal doll, an anime or cartoon character doll, a robot, or other toys.

It is a block diagram of the learning apparatus of one Embodiment of this invention. It is a flowchart of the registration mode of the learning apparatus of this embodiment. It is a flowchart of the recognition mode of the learning apparatus of this embodiment. It is an external view of the baby robot of this embodiment. It is a block diagram of the learning apparatus which has the speech recognition part of 2nd Embodiment. 5 is a flowchart of speech recognition having a keyword recognition unit 14. It is a flowchart of learning including intonation. It is a flowchart of face recognition in registration mode. It is a flowchart of face recognition in recognition mode. It is a flowchart of update registration. 3 is a configuration diagram illustrating a storage state of a registration storage unit 16. FIG.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Learning apparatus 12 Voice recognition part 14 Object recognition part 16 Registration memory | storage part 18 Speech synthesis part 100 Baby robot 102 Microphone 104 Camera

Claims (3)

A learning device built into a robot, doll, or toy,
Object recognition means for recognizing an object from an image input from the image input means;
In a registration mode for registering the object, speech recognition means for recognizing a noun from speech input from a speech input device;
In the registration mode, a registration storage unit that stores voice data including a symbol string related to a noun recognized by the voice recognition unit and image data related to the object recognized by the object recognition unit in association with each other;
In the recognition mode for recognizing whether or not the object is a registered object, when the object recognized by the object recognizing unit matches the object stored in the registered storage unit, corresponding to the stored object Speech synthesis means for synthesizing speech based on speech data stored and speech segments of a segment dictionary stored in advance;
Have,
The object recognition means recognizes a human face,
The speech recognition means recognizes a keyword belonging to at least one of a pronoun, a particle, or an auxiliary verb in the input speech, and a character string excluding the recognized keyword in the input speech Is recognized as a noun, and the recognized voice data further includes pitch information representing the tone of inflection,
The speech synthesis means synthesizes speech from the character string of the noun, the pitch information, and the segment of the segment dictionary, and the segment dictionary includes a robot, a doll, or It consists of pieces that match the appearance of the toy.
A learning apparatus characterized by that. A learning method in a learning device built in a robot, a doll, or a toy,
An object recognition step for recognizing an object from an image input from an image input means;
In a registration mode for registering the object, a speech recognition step for recognizing a noun from speech input from a speech input device;
In the registration mode, a registration storage step of storing voice data including a symbol string related to the noun recognized in the voice recognition step and image data relating to the object recognized in the object recognition step in association with each other;
In the recognition mode for recognizing whether or not the object is a registered object, when the object recognized in the object recognition step matches the object stored in the registration storage step, corresponding to the stored object A speech synthesis step of synthesizing speech based on speech data stored in advance and speech segments of a segment dictionary stored in advance;
Have,
Recognizing a human face in the object recognition step;
In the speech recognition step, a character string that recognizes a keyword belonging to at least one of a demonstrative pronoun, a particle, and an auxiliary verb in the input speech and that excludes the recognized keyword in the input speech Is recognized as a noun, and the recognized voice data further includes pitch information representing the tone of inflection,
In the speech synthesis step, a speech is synthesized from the character string of the noun, the pitch information, and the segment of the segment dictionary, and the segment dictionary is a robot, doll, or It consists of pieces that match the appearance of the toy.
A learning method characterized by that. A program that causes a computer built in a robot, doll, or toy to function as a learning device,
An object recognition function for recognizing an object from an image input from an image input means;
In a registration mode for registering the object, a speech recognition function for recognizing a noun from speech input from a speech input device;
In the registration mode, a registration storage function for storing voice data including a symbol string relating to a noun recognized by the voice recognition function and image data relating to an object recognized by the object recognition function in association with each other;
In the recognition mode for recognizing whether or not the object is a registered object, when the object recognized by the object recognition function matches the object stored by the registration storage function, corresponding to the stored object A speech synthesis function for synthesizing speech based on speech data stored in advance and speech segments in a segment dictionary stored in advance;
Realized,
In the object recognition function, a human face is recognized,
In the speech recognition function, a character string that recognizes a keyword belonging to at least one of a demonstrative pronoun, a particle, and an auxiliary verb in the input speech and that excludes the recognized keyword in the input speech Is recognized as a noun, and the recognized voice data further includes pitch information representing the tone of inflection,
In the speech synthesis function, the speech is synthesized from the character string of the noun, the pitch information, and the segment of the segment dictionary, and the segment dictionary is a robot incorporating the computer, a doll, or Consists of pieces that match the appearance of the toy,
A program characterized by that.

Images