JP2002116794A - Robot controller and method for robot control and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To prevent voice from being misrecognized owing to noise that a robot generates. SOLUTION: A voice signal inputted to a microphone is converted by an AD conversion part 41 into a digital signal, which is outputted to a voice section detection part 47 and a feature extraction part 42. The voice section detection part 47 decides that the signal is not a voice section while an actuator nearby the microphone is in operation according to state recognition information and attitude transition information and then cancels voice recognition processing and varies the value of a threshold for determining the start of a voice section according to the contents of the operation and state. When the robot is walking, the feature extraction part 42 extracts features by using voice data from which pulsative grounding noise generated during the walk is removed through frame removable, filtering, etc.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a robot control device, a robot control method, and a recording medium, and more particularly to a robot control device and a robot control suitable for a robot acting on the basis of a result of voice recognition by a voice recognition device. The present invention relates to a method and a recording medium.

[0002]

2. Description of the Related Art In recent years, for example, a toy or the like performs voice recognition of a voice uttered by a user, and performs a certain gesture or outputs a synthetic sound based on the voice recognition result. (In this specification, including a stuffed one) has been commercialized.

[0003]

Such a robot is always adapted to receive a voice input.
However, the noise of the actuator generated during the operation of the robot, the grounding pulse sound generated when the robot is walking, or the touch noise generated when the user touches the robot is erroneously detected as the voice spoken by the user. There was a case.

[0004] The present invention has been made in view of such a situation, and performs voice recognition based on the state and behavior of a robot to reduce the noise uttered by the user and the noise generated by the operation of the robot. In order to prevent erroneous recognition.

[0005]

According to the present invention, there is provided a robot control apparatus comprising: a voice input means for receiving voice data; a first generating means for generating first information indicating a state of the robot; Based on the first information generated by the first generating means or the second information generated by the second generating means. Recognizing means for recognizing voice data input by the means.

The second information can include information indicating which one of the plurality of driving units of the robot performs a driving operation, and the position of the driving unit to be driven is determined by voice input. If it is close to the means, the recognition means can be prevented from recognizing the voice data.

[0007] The second information can include information indicating whether or not the robot is performing a walking operation. When the robot is performing a walking operation, the recognizing means includes: It is possible to recognize the voice data excluding the frame including the generated noise component.

[0008] A storage means for storing data corresponding to a noise component generated when the robot is performing a walking operation may be further provided. The second information indicates whether the robot is performing a walking operation. Can be included, and when the robot is performing a walking motion, the recognition unit uses the data corresponding to the noise component stored by the storage unit to filter the audio data after filtering. Voice data can be recognized.

The second information can include information indicating which one of the plurality of driving units of the robot performs a driving operation, and the recognizing means includes the second information. Based on this, speech recognition can be performed in consideration of noise generated by driving of the driving unit.

[0010] The first information may include information indicating whether or not the robot is being touched by the user, and the recognizing means uses the first information to indicate that the user is touching the robot. Therefore, voice recognition can be performed in consideration of the noise generated.

Storage means for storing a predetermined threshold value corresponding to noise generated by the state or action of the robot;
The recognition means may further include an estimation means for estimating an environmental sound when the speech recognition is not performed. The recognition means may include the first information generated by the first generation means or the second information. Based on the second information generated by the generating means, the start of the section in which the voice recognition is performed can be determined using the threshold value stored in the storing means and the environmental sound estimated by the estimating means.

[0012] The apparatus may further include storage means for storing a predetermined threshold value corresponding to noise generated by the state or action of the robot, wherein the recognition means includes first information generated by the first generation means, Alternatively, based on the second information generated by the second generation unit, the start of the section in which voice recognition is performed can be determined using the threshold value stored in the storage unit.

[0013] A setting means for setting a threshold value corresponding to a noise generated by the state or action of the robot based on the voice data input by the voice input means may be further provided. Using the set threshold value, it is possible to determine the start of a section in which speech recognition is performed.

According to the robot control method of the present invention, a voice input step for receiving voice data, a first generation step for generating first information indicating a state of the robot, and a second information indicating behavior of the robot are provided. Based on the first information generated by the processing of the first generation step or the second information generated by the processing of the second generation step. And a recognition step of recognizing the voice data input by the processing.

The program recorded on the recording medium according to the present invention includes a voice input step for receiving voice data input, a first generation step for generating first information indicating a state of the robot, and an action of the robot. A second generation step of generating the second information shown, and the first information generated by the processing of the first generation step or the second information generated by the processing of the second generation step. And a recognition step of recognizing the voice data input by the processing of the input step.

In the robot control apparatus, the robot control method, and the program recorded on the recording medium of the present invention, voice data is input, first information indicating the state of the robot is generated, and the behavior of the robot is indicated. Second information is generated, and the input voice data is recognized based on the generated first information or the generated second information.

[0017]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 shows an example of an external configuration of an embodiment of a robot to which the present invention is applied, and FIG. 2 shows an example of an electrical configuration thereof.

In the present embodiment, for example, the robot
It has the shape of a four-legged animal such as a dog, and has leg units 3A,
3B, 3C, and 3D are connected, and a head unit 4 and a tail unit 5 are connected to a front end and a rear end of the body unit 2, respectively.

The tail unit 5 is drawn out from a base unit 5B provided on the upper surface of the body unit 2 so as to bend or swing with two degrees of freedom.

The body unit 2 contains a controller 10 for controlling the entire robot, a battery 11 as a power source of the robot, and an internal sensor unit 14 including a battery sensor 12 and a heat sensor 13.

The head unit 4 includes a microphone (microphone) 15 corresponding to “ears” and a CCD corresponding to “eyes”.
(Charge Coupled Device) A camera 16, a touch sensor 17 corresponding to tactile sensation, a speaker 18 corresponding to a "mouth", and the like are arranged at predetermined positions. A lower jaw 4A corresponding to the lower jaw of the mouth is movably attached to the head unit 4 with one degree of freedom.
When A moves, the opening and closing operation of the mouth of the robot is realized.

The joints of the leg units 3A to 3D, the joints of the leg units 3A to 3D and the body unit 2, the joints of the head unit 4 and the body unit 2, the head unit 4 and the lower jaw linking moiety parts 4A, and the like in the connecting portion of the tail unit 5 and the body unit 2, as shown in FIG. 2, each actuator 3AA ₁ to 3AA _K, 3BA ₁ to 3BA _K, 3CA
₁ to 3CA _K, 3DA ₁ to 3DA _K, 4A ₁ to 4A _L,
5A ₁ and 5A ₂ are disposed.

Further, a sensor 3A ₁ is provided on the ground portion (the portion corresponding to the sole of the foot) of each of the leg units 3A to 3D.
To the sensor 3D ₁ is provided, the leg units 3A to 3
Each of D detects whether it is grounded (for example, whether it is touching the floor or the like) and outputs it to the controller 10.

The microphone 15 in the head unit 4 collects surrounding sounds (sounds) including utterances from the user, and sends out the obtained sound signals to the controller 10. CCD
The camera 16 captures an image of the surroundings, and sends the obtained image signal to the controller 10.

The touch sensor 17 is provided, for example, above the head unit 4 and “strokes” from the user.
It detects the pressure received by a physical action such as tapping or tapping, and sends the detection result to the controller 10 as a pressure detection signal.

The battery sensor 12 in the body unit 2 detects the remaining amount of the battery 11 and sends the detection result to the controller 10 as a battery remaining amount detection signal. The heat sensor 13 detects heat inside the robot,
The detection result is sent to the controller 10 as a heat detection signal.

The controller 10 has a CPU (Central Pr
10A, a memory 10B, and the like. The CPU 10A performs various processes by executing a control program stored in the memory 10B.

That is, the controller 10 controls the microphone 1
5, CCD camera 16, touch sensor 17, sensor 3A
_Based on the audio signal, image signal, pressure detection signal, battery remaining amount detection signal, and heat detection signal given from _{1 to} sensor 3D ₁ , battery sensor 12, and heat sensor 13, the surrounding conditions and user commands are given. Then, it is determined whether or not there is a request from the user.

Furthermore, the controller 10, based on the determination results and the like, followed by action determines, based on the determination result, the actuators 3AA ₁ to 3AA _K, 3BA ₁ to 3BA _K, 3CA ₁ to 3CA _K, 3DA _{1 to} 3D
Drive the required one of A _K , 4A _{1 to} 4A _L , 5A ₁ , and 5A ₂ . Thereby, the head unit 4 is swung up, down, left and right, and the lower jaw 4A is opened and closed. Further, the tail unit 5 can be moved, and the leg units 3A to 3D are driven to perform actions such as walking the robot.

Further, the controller 10 generates a synthesized sound as necessary, and supplies the synthesized sound to the speaker 18 for outputting the synthesized sound, or an LED (Light Emitting Diode) (not shown) provided at the position of the “eye” of the robot. Turn on, turn off or blink.

As described above, the robot autonomously acts based on the surrounding situation and the like.

Next, FIG. 3 shows an example of a functional configuration of the controller 10 of FIG. Note that the functional configuration illustrated in FIG. 3 is realized by the CPU 10A executing a control program stored in the memory 10B.

The controller 10 includes a sensor input processing unit 31 for recognizing a specific external state, a model storage unit 32 for accumulating the recognition results of the sensor input processing unit 31, and expressing a feeling, an instinct, and a growth state. Action determining mechanism 3 that determines a subsequent action based on the recognition result of input processing unit 31 and the like.
3, based on the determination result of the action decision mechanism section 33, the posture transition mechanism unit 34 for actually take action on the robot, the actuators 3AA ₁ to control mechanism 35 for driving and controlling the actuator 5A _2, sound to generate a synthesized speech Synthesizing unit 3
6 and an output control unit 37 for controlling the output of the synthesized sound synthesized by the voice synthesis unit 36.

The sensor input processing unit 31 includes the microphone 15, C
CD camera 16, touch sensor 17 or the sensor 3A ₁ to audio signals supplied from the sensor 3D ₁ etc., the image signal, based on the pressure detection signal and the like, and specific external conditions,
It recognizes a specific action from the user, an instruction from the user, and the like, and notifies the model storage unit 32 and the action determination mechanism unit 33 of state recognition information indicating the recognition result.

That is, the sensor input processing section 31 has a voice recognition section 31A, and the voice recognition section 31A performs voice recognition on a voice signal given from the microphone 15. Then, the voice recognition unit 31A uses the model storage unit 32 and the action determination mechanism unit as state recognition information, for example, commands such as “walk”, “down”, “chase the ball” and the like as the voice recognition result. Notify 33.

The speech recognition unit 31A is provided with a pressure processing unit 3
Based on the state recognition information input from 1C and the posture transition information of the robot input from the posture transition mechanism unit 34,
The voice recognition processing is executed while monitoring the state of the robot.

The sensor input processing section 31 has an image recognition section 31B, and the image recognition section 31B performs an image recognition process using an image signal given from the CCD camera 16. When the image recognition unit 31B detects, for example, a “red round object” or a “plane that is perpendicular to the ground and equal to or more than a predetermined height” as a result of the processing,
Image recognition results such as “there is a ball” and “there is a wall” are notified to the model storage unit 32 and the action determination mechanism unit 33 as state recognition information.

[0039] Further, the sensor input processing unit 31 has a pressure processing unit 31C, the pressure processing unit 31C includes a touch sensor 17 and processes the pressure detection signal supplied from the sensor 3A ₁ to sensor 3D ₁ .

As a result of the processing, when the pressure sensor 31C detects a pressure equal to or higher than a predetermined threshold and for a short time from the touch sensor 17, the pressure processing unit 31C recognizes "hit". When a pressure that is less than the predetermined threshold value and is detected for a long time is detected as “stroke (praised)”, the recognition result is used as state recognition information as the model storage unit 32 and the action determination mechanism unit. 33 and voice recognition unit 3
Notify 1A. Further, the pressure processing unit 31C, based on the signal inputted from the sensor 3A ₁ to sensor 3D _1, leg units 3A to 3D are, when any is detected that is not grounded on the floor, the lift up by the user Then, the recognition result is notified to the model storage unit 32, the action determination mechanism unit 33, and the speech recognition unit 31A as state recognition information.

The model storage unit 32 stores and manages an emotion model, an instinct model, and a growth model expressing the emotion, instinct, and growth state of the robot.

Here, the emotion model expresses emotion states (degrees) such as "joy,""sadness,""anger," and "fun," by values in a predetermined range, respectively. The value is changed based on the state recognition information from the input processing unit 31, the elapsed time, and the like. The instinct model is
For example, the state (degree) of the desire by instinct such as “appetite”, “sleep desire”, and “exercise desire” is represented by a value in a predetermined range, and the state recognition information from the sensor input processing unit 31, the elapsed time, and the like. , The value is changed. The growth model represents, for example, the growth state (degree) such as “childhood”, “adolescence”, “mature”, “elderly” by a value in a predetermined range, and the sensor input processing unit 31.
The value is changed on the basis of the state recognition information or the passage of time.

The model storage unit 32 sends the emotion, instinct, and growth state represented by the values of the emotion model, instinct model, and growth model as described above to the action determination mechanism unit 33 as state information.

The model storage unit 32 is supplied with the state recognition information from the sensor input processing unit 31, and the current or past behavior of the robot, specifically, for example, “ Behavior information indicating the content of the action such as "walking for a long time" is supplied, and even if the same state recognition information is given, different state information is given according to the action of the robot indicated by the action information. Is to be generated.

That is, for example, when the robot greets the user and strokes his head, the behavior information indicating that the user greets the user and the state recognition information indicating that the head has been stroked are represented by the model. It is provided to the storage unit 32, and in this case, the value of the emotion model representing “joy” is increased in the model storage unit 32.

On the other hand, when the robot is stroked on the head while performing any work, the action information indicating that the robot is performing the work and the state recognition information indicating that the robot has been stroked on the head include:
The value is given to the model storage unit 32. In this case, the value of the emotion model representing “joy” is not changed in the model storage unit 32.

As described above, the model storage unit 32 sets the value of the emotion model with reference to not only the state recognition information but also the behavior information indicating the current or past behavior of the robot. Thus, for example, while performing some task,
When the user strokes his head with the intention of mischief,
It is possible to avoid an unnatural change in emotion, such as increasing the value of the emotion model representing “joy”.

The model storage unit 32 increases and decreases the values of the instinct model and the growth model based on both the state recognition information and the action information as in the case of the emotion model. The model storage unit 32 increases or decreases the values of the emotion model, the instinct model, and the growth model based on the values of other models.

The action determining mechanism unit 33 determines the next action based on the state recognition information from the sensor input processing unit 31, the state information from the model storage unit 32, the passage of time, and the like. The content is sent to the posture transition mechanism unit 34 as action command information.

That is, the action determining mechanism 33 manages a finite state automaton in which actions that can be taken by the robot correspond to states, as an action model that defines the actions of the robot. The state in the finite state automaton is changed based on the state recognition information from the sensor input processing unit 31, the value of the emotion model, the instinct model, or the growth model in the model storage unit 32, the lapse of time, and the like. Is determined as the next action to be taken.

Here, upon detecting that a predetermined trigger has occurred, the action determining mechanism 33 changes the state. That is, for example, when the time during which the action corresponding to the current state is being executed reaches a predetermined time, or when specific state recognition information is received, the action determining mechanism unit 33 is supplied from the model storage unit 32. The state is changed when the value of the emotion, instinct, or growth state indicated by the state information is equal to or less than a predetermined threshold.

As described above, the action determining mechanism unit 33 stores not only the state recognition information from the sensor input processing unit 31 but also the values of the emotion model, the instinct model, the growth model, and the like in the model storage unit 32. Based on the state transition based on the behavior model, the destination of the state transition differs depending on the emotion model, the instinct model, and the value of the growth model (state information) even if the same state recognition information is input. Become.

As a result, for example, when the state information indicates “not angry” and “not hungry”, the action determination mechanism 33 When the palm has been presented before, the action command information for taking the action of "hand" is generated in accordance with the palm being presented in front of the eyes, It is sent to the posture transition mechanism 34.

Further, for example, when the state information indicates “not angry” and “stomach is hungry”, the action recognition mechanism unit 33 determines that the state recognition information is “in front of the eyes”. When the palm is displayed, the action command information for performing an action such as "palm licking the palm" is generated in response to the palm being displayed in front of the eyes. This is sent to the posture transition mechanism 34.

Further, for example, when the state information indicates “angry”, the action determining mechanism unit 33 determines that the state recognition information indicates that “the palm is put in front of the eyes”. When it indicates, even if the status information indicates that "stomach is hungry", or indicates that "stomach is not hungry", even if the state information indicates "being hungry", an action such as "turns to the side with a little bit" The action command information for performing the action is generated, and the action command information is transmitted to the posture transition mechanism unit 34.

The behavior determining mechanism 33 stores the emotions indicated by the state information supplied from the model storage 32, instinct,
Based on the state of growth, as a parameter of the action corresponding to the state of the transition destination, for example, the speed of walking, the magnitude and speed of the movement when moving the limbs can be determined, in this case, Is transmitted to the posture transition mechanism unit 34.

Further, as described above, the action determining mechanism 33 generates action command information for causing the robot to speak, in addition to action command information for operating the robot's head, limbs, and the like. The action command information that causes the robot to speak is
The voice command is supplied to the voice synthesizing unit 36, and the action command information supplied to the voice synthesizing unit 36 includes the voice synthesizing unit 3.
6 includes a text corresponding to the synthesized sound to be generated. Then, upon receiving the action command information from the action determination section 32, the voice synthesis section 36 generates a synthesized sound based on the text included in the action command information, and supplies the synthesized sound to the speaker 18 via the output control section 37. Output. Thereby, for example, the cry of the robot,
Further, various requests to the user such as “I am hungry”, a response to the user's call such as “What?”, And other voice output are performed.

The posture transition mechanism section 34 includes the action determination mechanism section 3
Based on the action command information supplied from 3, posture transition information for transitioning the posture of the robot from the current posture to the next posture is generated and transmitted to the control mechanism unit 35 and the voice recognition unit 31 </ b> A. .

Here, the posture that can be changed next from the current posture is, for example, the physical shape of the robot such as the shape and weight of the body, hands and feet, the connection state of each part, the direction in which the joint is bent, and the like. It is determined by the mechanism of the actuator 3AA ₁ to 5A ₁ and 5A _2, such as angle.

As the next posture, there are a posture that can directly transition from the current posture and a posture that cannot directly transition. For example, a four-legged robot can make a direct transition from lying down with its limbs throwing down to lying down, but not directly into a standing state. It is necessary to perform a two-stage operation of pulling down to a prone position and then standing up. There are also postures that cannot be safely executed. For example, a four-legged robot easily falls down when trying to banzai with both front legs raised from its standing posture.

For this reason, the posture transition mechanism unit 34 pre-registers the posture to which a direct transition can be made, and if the action command information supplied from the behavior determination mechanism unit 33 indicates the posture to which a direct transition can be made, The action command information is sent to the control mechanism section 35 as posture transition information as it is. on the other hand,
When the action command information indicates a posture that cannot directly transition, the posture transition mechanism unit 34 generates posture transition information that causes a transition to another possible posture and then transitions to a target posture. To the control mechanism unit 35. As a result, it is possible to avoid a situation in which the robot forcibly executes an untransitionable posture or a situation in which the robot falls.

The posture transition information is also output to the voice recognition unit 31A. Robot, if shifting the posture thereof, of the actuators 3AA ₁ through actuator 5A _2, either actuator operates. Therefore, the posture transition mechanism unit 34 outputs the posture transition information to the voice recognition unit 31A so that the voice recognition unit 31A does not recognize the operation sounds of these actuators as the user's voice.

The control mechanism 35 changes the position of the actuator 3AA in accordance with the posture transition information from the posture transition mechanism 34.
₁ generates a control signal for driving the actuator 5A _2, which is sent to the actuator 3AA ₁ to actuator 5A _2. Thus, the actuator 3AA ₁ to actuator 5A ₂ is driven in accordance with the control signals, the robot causes the autonomous motions.

The output control section 37 is supplied with the digital data of the synthesized sound from the voice synthesizing section 36. The output control section 37 converts the digital data into an analog voice signal by D / D conversion. A conversion, speaker 18
And output it.

Next, FIG. 4 shows an example of the configuration of the speech recognition section 31A of FIG.

The audio signal from the microphone 15 is AD (Anal
og Digital) converter 41. AD converter 41
In, an audio signal as an analog signal from the microphone 15 is sampled and quantized, and A / D converted into audio data as a digital signal. This audio data is supplied to the feature extraction unit 42 and the audio section detection unit 47.

The feature extracting unit 42 receives the input of the state recognition information and the posture transition information for each appropriate frame of the input voice data, and for example, according to the state of the robot, for example, a MFCC (Mel Frequency Cepstrum). Coe
fficient) analysis, and outputs the analysis result to the matching unit 43 as a feature parameter (feature vector). The feature extraction unit 42 can also extract, for example, a linear prediction coefficient, a cepstrum coefficient, a line spectrum pair, power (output of a filter bank) for each predetermined frequency band, and the like as a feature parameter.

The matching section 43 uses the feature parameters from the feature extraction section 42 to refer to the acoustic model storage section 44, the dictionary storage section 45, and the grammar storage section 46 as necessary, and to be input to the microphone 15. Voice (input voice)
For example, a continuous distribution HMM (Hidden Markov Model)
Speech recognition based on the law.

That is, the acoustic model storage unit 44 stores acoustic models representing acoustic features such as individual phonemes and syllables in the language of the speech to be recognized. Here, since speech recognition is performed based on the continuous distribution HMM method, an HMM (Hidden Markov Model) is used as an acoustic model. The dictionary storage unit 45 stores a word dictionary in which information (phonological information) regarding pronunciation of each word to be recognized is described. The grammar storage unit 46 stores grammar rules that describe how the words registered in the word dictionary of the dictionary storage unit 45 are linked (connected). Here, the grammar rules include, for example, context-free grammar (CFG) and statistical word chain probability (N-gra
m) etc. can be used.

The matching unit 43 refers to the word dictionary in the dictionary storage unit 45 and connects the acoustic models stored in the acoustic model storage unit 44 to form a word acoustic model (word model). . Further, the matching unit 43 connects some word models by referring to the grammar rules stored in the grammar storage unit 46, and uses the word models connected in this way, based on the feature parameters, The microphone 15 is obtained by the continuous distribution HMM method.
Recognize the voice input to. That is, the matching unit 43
Detects a sequence of a word model having the highest score (likelihood) at which a time-series feature parameter output by the feature extraction unit 42 is observed, and obtains phonemic information (reading) of a word string corresponding to the sequence of the word model. Is output as a speech recognition result.

More specifically, the matching unit 43 accumulates the appearance probabilities of the respective characteristic parameters for the word string corresponding to the connected word model, and uses the accumulated value as a score to determine the word having the highest score. The phoneme information of the column
Output as speech recognition result.

The microphone 15 output as described above
The recognition result of the voice input to the
It is output to the model storage unit 32 and the action determination mechanism unit 33.

It is to be noted that the voice section detection unit 47 determines that the feature extraction unit 42
The speech input level (power) is calculated for each frame similar to that in which the CC analysis is performed. Further, the voice section detection unit 47 receives the input of the state recognition information and the posture transition information, and detects the voice section in which the user's voice is input by comparing the voice input level of each frame with a predetermined threshold. I do. That is, the voice section is a section in which a predetermined operation serving as a noise source (for example, moving the head unit 4) is not performed, and includes a frame having a voice input level equal to or higher than a predetermined threshold. Indicates a section.
Then, the voice section detection unit 47 converts the detected voice section into
The information is supplied to the feature extraction unit 42 and the matching unit 43, and the feature extraction unit 42 and the matching unit 43 perform processing only on the voice section.

Next, the speech recognition processing will be described with reference to the flowcharts of FIGS.

In step S1, the voice section detection unit 4
7 estimates the environmental sound level based on the audio data input via the AD converter 41.

Although various noises are input to the microphone 15 even when the user is not speaking to the robot, recognizing the noises as the user's speech causes a malfunction. Therefore, it is necessary to estimate the environmental sound level in a state where the speech of the user is not speech-recognized (speech recognition OFF state).

As shown in FIG. 7, the voice input level of the voice data input via the microphone 15 and the AD converter 41 is not constant even in the voice recognition OFF state. Then, assuming that the environmental sound level is ENV and the current voice input level is P, by the following equations (1) and (2),
The environmental sound level is calculated for each predetermined short time. ENV = a × ENV + b × P (1) a + b = 1.0 (2)

Here, a is set to a number relatively close to 1 such as 0.9, and b is set to 0.1 or the like, so that noise having a large instantaneous power (for example, when the door is Noises, etc.) do not significantly affect the overall environmental sound.

The environmental sound level is estimated based on the predetermined threshold value T1 until the voice input level exceeds ENV + T1 (step S8 or step S1 to be described later).
(3) until the sound input level exceeds ENV + T1.

In step S2, the voice section detection unit 4
7 is the posture transition mechanism unit 34 or the pressure processing unit 31
From -C, it is determined whether or not the input of the posture transition information or the state recognition information has been received. If it is determined in step S2 that the input of the posture transition information or the state recognition information has not been received, the process proceeds to step S8.

If it is determined in step S2 that the input of the posture transition information or the state recognition information has been received, in step S3, the voice section detection unit 47 performs an operation based on the input information with a microphone. It is determined whether it is close to 15. If it is determined in step S3 that the actuator performing the operation is not close to the microphone 15, the process proceeds to step S6.

If it is determined in step S3 that the actuator performing the operation is close to the microphone 15, the voice section detection unit 47 cancels the voice recognition processing in step S4. That is, during the operation of the actuator close to the microphone 15, the sound section is not set.

In the robot described with reference to FIGS. 1 and 2, the microphone 15 is provided on the head unit 4. That is, the actuator 4A ₁ to 4A are provided in the head unit 4 _L (e.g., coupling portion of the head unit 4 and body unit 2 or the lower jaw 4A,) noise of the speech input power generated with the operation of the Is relatively large, the reliability of the speech recognition result during operation becomes extremely low. Therefore, during the operation of the actuator close to the microphone 15, it is possible to prevent erroneous operation by not setting it as a voice section.

In step S5, the voice section detection unit 4
7 determines whether or not the operation has been completed based on the input information. If it is determined in step S5 that the operation has not been completed, the process returns to step S4, and the subsequent processes are repeated. If it is determined in step S5 that the operation has been completed,
Returning to step S1, the subsequent processing is repeated.

In step S3, when it is determined that the actuator performing the operation is not close to the microphone 15,
In step S6, the voice segment detection unit 47 determines whether or not the posture transition information indicates the start of the walking motion based on the input information. If it is determined in step S6 that the start of the walking motion is indicated, the process proceeds to step S13.

At step S6, the posture transition information is
If it is determined that it does not indicate the start of the walking motion, the robot does not operate the actuator near the microphone 15;
And an operation other than the walking operation is performed, or
The user is being held up or stroked. In step S7, the voice section detection unit 4
Reference numeral 7 changes the threshold value T1 (FIG. 7) for determining that a user's utterance or the like, which is not environmental sound (noise), is input as a voice, to a value corresponding to posture transition information or state recognition information. I do.

For example, when the state recognition information input from the pressure processing unit 31C indicates that the robot is being held by the user, the voice data input from the microphone 15 includes the user's housing of the robot. Includes touch noise and rubbing noise caused by touching the surface. Also shows that the posture transition information input from the state transition mechanism section 34, an actuator provided in a position relatively far to the microphone 15 (e.g., an actuator 5A ₁ of the tail unit 5) is operating If
The sound data input from the microphone 15 includes the operation sounds of those actuators. The operation sound included in the audio data differs depending on the type of the actuator and the distance from the microphone 15.

For example, the voice section detection unit 47 includes a threshold T1a when the robot is held by the user and a threshold T1 when the leg unit 3A or 3B is operating.
1b, a threshold T1c when the leg unit 3C or 3D is operating, or a threshold T1d when the tail unit 5 is operating is prepared in advance, and the input posture transition information or state According to the recognition information, a setting is made so as to detect a voice section using an appropriate threshold.

If it is determined in step S2 that the input of the posture transition information or the state recognition information has not been received, or if the processing in step S7 has been completed, in step S8, the voice section detection unit 47 determines the voice input level. Is greater than or equal to a threshold value (T1 + ENV).
In step S8, the voice input level is set to the threshold (T1
+ ENV), if it is determined that it does not exceed
Returning to step S1, the subsequent processing is repeated.

If it is determined in step S8 that the voice input level has exceeded the threshold value (T1 + ENV), in step S9, the voice section detection unit 47 stops estimating the environmental sound level, and the A speech recognition start count is started using a counter (timer) not to be used.

In step S10, the voice section detection unit 47 determines whether or not the voice recognition start count has exceeded a predetermined value (for example, a value indicated by CNT_ON in FIG. 7). If it is determined in step S10 that the voice recognition start count has not exceeded the predetermined value, the process of step S10 is repeated until it is determined that the voice recognition start count has exceeded the predetermined value.

When it is determined in step S10 that the speech recognition start count has exceeded a predetermined value, in step S11, the speech section detection section 47 outputs the start of the speech section to the feature extraction section 42 and the matching section 43. . The feature extraction unit 42 and the matching unit 43 execute the speech recognition processing described with reference to FIG.

In step S12, the voice section detection section 47 determines whether or not the voice input level has become equal to or less than the threshold value (T2 + ENV).

Various noises are input to the microphone 15 even after the user has finished speaking to the robot. Recognizing the noises as the user's speech causes malfunctions. Therefore, when the voice input level falls below a certain value, it is necessary to prevent the voice recognition processing from being performed (turn off the voice recognition).

As shown in FIG. 8, when the audio input level of the audio data input via the microphone 15 and the AD converter 41 is a predetermined threshold value T2 and the environmental sound at the time when the voice recognition process is started. Sum with level ENV (T2 +
By determining whether the value falls below ENV), the voice section detection unit 47 can determine whether or not the utterance of the user has ended.

If it is determined in step S12 that the voice input level is not lower than the threshold value (T2 + ENV), the process returns to step S11, and the subsequent processes are repeated. If it is determined in step S12 that the audio input level has become equal to or lower than the threshold value (T2 + ENV), the process proceeds to step S18.

If it is determined in step S6 that the posture transition information indicates the start of the walking motion, in step S13, the voice section detection unit 47 determines whether the voice input level has exceeded a threshold value (T1 + ENV). Judge. If it is determined in step S13 that the audio input level has not exceeded the threshold value (T1 + ENV), the process returns to step S1, and the subsequent processes are repeated.

If it is determined in step S13 that the voice input level exceeds the threshold value (T1 + ENV), in step S14, the voice section detection unit 47
The estimation of the ambient sound level is stopped, and a speech recognition start count is started using a counter (timer) (not shown) provided therein.

In step S15, the voice section detection section 47 determines whether or not the voice recognition start count has exceeded a predetermined value (for example, a value indicated by CNT_ON in FIG. 7). If it is determined in step S15 that the voice recognition start count has not exceeded the predetermined value, the process of step S15 is repeated until it is determined that the voice recognition start count has exceeded the predetermined value.

If it is determined in step S15 that the speech recognition start count has exceeded a predetermined value, in step S16, the speech section detection section 47 outputs the start of the speech section to the feature extraction section 42 and the matching section 43. . The feature extracting unit 42 and the matching unit 43 execute a voice recognition process dedicated to the walking motion.

As shown in FIG. 9, during the walking operation of the robot, in addition to the utterance of the user, grounding noise when the leg units 3A to 3D are grounded on, for example, the floor, is input to the audio data. Will be included in The feature extraction unit 4 notified that the user is walking, as the posture transition information
2 monitors, for example, an increase or decrease (ΔP) in the audio input level to remove this ground noise from the audio data;
The pulse noise is detected based on the magnitude of ΔP (pulse sound data has a feature that its sound level rapidly rises and then sharply decreases after the peak as shown in FIG. 9. ). Then, the feature extracting unit 42 does not perform the feature extracting process on the frame.
Ground noise occurs in a very short time,
Even if the frame detected as the ground noise is excluded from the voice recognition processing, there is no problem in recognizing the utterance of the user.

Further, a standard voice component of the ground noise at the time of the walking operation is stored in advance in the feature extracting unit 42, and at the time of the walking operation, the input voice data is filtered using the voice component. Then, the feature extraction processing may be performed using the audio data after the ground noise has been removed.

In step S17, the voice section detection section 47 determines whether or not the voice input level has become equal to or less than the threshold value (T2 + ENV) by the same processing as in step S12. If it is determined in step S17 that the voice input level is not lower than the threshold value (T2 + ENV), the process returns to step S16, and the subsequent processes are repeated.

If it is determined in step S12 that the voice input level has fallen below the threshold value (T2 + ENV), or if it is determined in step S17 that the voice input level has fallen below the threshold value (T2 + ENV), In S18, the voice section detection unit 47
Starts a voice recognition end count using a counter (timer) (not shown) provided therein.

In step S19, the voice section detection section 47 determines whether or not the voice recognition end count has exceeded a predetermined value (for example, a value indicated by CNT_OFF in FIG. 8). If it is determined in step S19 that the voice recognition end count has not exceeded the predetermined value, the process proceeds until the voice recognition end count is determined to have exceeded the predetermined value.
Step S19 is repeated.

If it is determined in step S19 that the speech recognition end count has exceeded a predetermined value, in step S20 the speech section detection section 47 outputs a signal indicating that the speech section has ended to the feature extraction section 42 and the feature extraction section 42. Output to the matching unit 43. The feature extracting unit 42 and the matching unit 43 end the voice recognition process, the process returns to step S1, and the subsequent processes are repeated.

In step S7 of FIG. 5, the voice section detection unit 47 sets the threshold value T1 for determining that the user's utterance or the like, which is not the environmental sound (noise), is input as a voice, by the posture transition. Although the description has been made on the assumption that the value is changed to the value corresponding to the information or the state recognition information, the threshold T3 which is not influenced by the environmental sound level may be used as the threshold for detecting the voice segment. That is, in step S8, the voice input level is set to the threshold (T1 +
ENV) is not determined, but it is determined whether a threshold T3 irrespective of the environmental sound level ENV has been exceeded.

The threshold value T3 may be prepared in advance in accordance with the posture transition information or the state recognition information. It may be set each time by acquiring a typical noise component.

Although the present invention has been described for the case where the present invention is applied to an entertainment robot (robot as a pseudo pet), the present invention is not limited to this. For example, the present invention is applied to various robots such as industrial robots. It can be widely applied. In addition, the present invention is applicable not only to a robot in the real world but also to a virtual robot displayed on a display device such as a liquid crystal display.

Furthermore, in the present embodiment, the above-described series of processing is performed by causing the CPU 10A (FIG. 2) to execute a program, but the series of processing is performed by dedicated hardware. Is also possible.

The program is stored in the memory 1 in advance.
0B (FIG. 2), a floppy disk, a CD-ROM (Compact Disc Read Only Memor
y), MO (Magnetooptical) disk, DVD (Digital Ve)
It can be temporarily or permanently stored (recorded) on a removable recording medium such as a rsatile disc, a magnetic disk, or a semiconductor memory. Then, such a removable recording medium can be provided as so-called package software, and can be installed in the robot (memory 10B).

The program can be transmitted wirelessly from a download site via an artificial satellite for digital satellite broadcasting, or transmitted via a cable via a network such as a LAN (Local Area Network) or the Internet.
It can be installed in the memory 10B.

In this case, when the program is upgraded, the upgraded program can be easily installed in the memory 10B.

Here, in this specification, the CPU 10A
The processing steps for writing a program for causing the CPU to perform various types of processing do not necessarily need to be processed in chronological order in the order described in the flowchart, and may be performed in parallel or individually (for example, parallel processing). Or processing by an object).

The program may be processed by only one CPU or may be processed in a distributed manner by a plurality of CPUs.

[0116]

According to the robot control apparatus, the robot control method, and the program recorded on the recording medium of the present invention, the input of voice data, the first information indicating the state of the robot is generated, Is generated, and the input voice data is recognized based on the generated first information or the generated second information. By performing the voice recognition based on, the voice uttered by the user and the noise generated by the operation of the robot or the like can be distinguished from each other to prevent erroneous recognition.

[Brief description of the drawings]

FIG. 1 is a perspective view illustrating an external configuration example of a robot according to an embodiment of the present invention.

FIG. 2 is a block diagram illustrating an example of an internal configuration of a robot.

FIG. 3 is a block diagram illustrating a functional configuration example of a controller.

FIG. 4 is a block diagram illustrating a configuration example of a speech recognition unit.

FIG. 5 is a flowchart illustrating a speech recognition process.

FIG. 6 is a flowchart illustrating a speech recognition process.

FIG. 7 is a diagram for explaining the start of a speech recognition section.

FIG. 8 is a diagram for explaining the end of a speech recognition section.

FIG. 9 is a diagram for explaining ground noise of a leg unit during a walking operation.

[Explanation of symbols]

4 head unit, 4A lower jaw, 10 controller, 10A CPU, 10B memory, 15
Microphone, 16 CCD camera, 17 touch sensor, 18 speaker, 31 sensor input processing unit,
31A voice recognition unit, 31B image recognition unit, 31C
Pressure processing section, 32 model storage section, 33 action determination mechanism section, 34 attitude transition mechanism section, 35 control mechanism section, 36 speech synthesis section, 37 output control section, 41
A / D conversion unit, 42 feature extraction unit, 43 matching unit, 44 acoustic model storage unit, 45 dictionary storage unit,
46 grammar storage unit, 47 voice section detection unit

──────────────────────────────────────────────────続 き Continued on the front page (51) Int.Cl. ⁷ Identification symbol FI Theme coat ゛ (Reference) G10L 15/20 G10L 3/02 301D 21/02 (72) Inventor Wataru Onoki 6-chome Kita Shinagawa, Shinagawa-ku, Tokyo 7-35 Inside Sony Corporation (72) Inventor Takashi Toyoda 6-7-35 Kita-Shinagawa, Shinagawa-ku, Tokyo F-term inside Sony Corporation (reference) 2C150 CA02 DA05 DA24 DA26 DA27 DA28 DF03 DF04 ED42 ED52 EF07 EF16 EF23 EF29 3F059 AA00 BB06 DD08 DD18 FA03 FC15 3F060 AA00 CA14 5D015 DD03 EE05 KK01 LL10

Claims (11)

[Claims] 1. A robot control device for controlling a robot acting based on at least a voice recognition result, comprising: a voice input unit for receiving input of voice data; and a first unit for generating first information indicating a state of the robot. 1, generating means for generating second information indicating the behavior of the robot, and the first information generated by the first generating means;
Alternatively, a robot control device comprising: a recognition unit that recognizes voice data input by the input unit based on the second information generated by the second generation unit. 2. The second information includes information indicating which one of the plurality of driving units of the robot performs a driving operation, and the position of the driving unit to be driven is The robot control device according to claim 1, wherein the recognition unit does not recognize the voice data when the voice data is close to the voice input unit. 3. The method according to claim 2, wherein the second information includes information indicating whether the robot is performing a walking operation. If the robot is performing the walking operation, the recognition unit determines whether the robot performs the walking operation. The robot control device according to claim 1, wherein the voice data excluding a frame including a noise component generated for the purpose is recognized. 4. The apparatus according to claim 1, further comprising a storage unit configured to store data corresponding to a noise component generated when the robot performs the walking operation, wherein the second information indicates whether the robot performs the walking operation. If the robot performs the walking motion, the recognition unit filters the voice data using data corresponding to the noise component stored by the storage unit. The robot control device according to claim 1, wherein the voice data is recognized later. 5. The second information includes information indicating which of the plurality of driving units of the robot performs a driving operation, wherein the recognizing unit includes the second information. The robot control device according to claim 1, wherein the voice recognition is performed in consideration of noise generated by the driving unit based on the driving. 6. The first information includes information indicating whether or not the robot is being touched by a user, and the recognition unit is configured to allow the user to touch the robot based on the first information. The robot control device according to claim 1, wherein the voice recognition is performed in consideration of noise generated due to the recognition. 7. A storage unit for storing a predetermined threshold value corresponding to noise generated by a state or action of the robot, and an estimation unit for estimating an environmental sound when the recognition unit does not perform voice recognition. The recognition unit is stored in the storage unit based on the first information generated by the first generation unit or the second information generated by the second generation unit. The robot control device according to claim 1, wherein a start of a section in which voice recognition is performed is determined using the threshold value and the environmental sound estimated by the estimation unit. 8. A storage unit for storing a predetermined threshold value corresponding to noise generated by a state or action of the robot, wherein the recognizing unit generates the first information generated by the first generating unit. Alternatively, based on the second information generated by the second generation unit, using the threshold value stored in the storage unit, determine the start of a section for performing voice recognition. The robot control device according to claim 1, wherein: 9. The apparatus further comprising: setting means for setting a threshold value corresponding to noise generated by a state or action of the robot based on the voice data input by the voice input means, The robot control device according to claim 1, wherein a start of a section in which voice recognition is performed is determined using the threshold value set by the means. 10. A robot control method for a robot control device for controlling a robot acting based on at least a voice recognition result, wherein: a voice input step of receiving voice data input; and generating first information indicating a state of the robot. A first generation step of generating, a second generation step of generating second information indicating the behavior of the robot, and the first information generated by the processing of the first generation step, or the first A recognition step of recognizing voice data input by the processing of the input step based on the second information generated by the processing of the second generation step. 11. A program for a robot control device that controls a robot that acts based on at least a voice recognition result, comprising: a voice input step of receiving voice data input; and a first information indicating a state of the robot. A first generation step of generating; a second generation step of generating second information indicating the behavior of the robot; and the first information generated by the processing of the first generation step, or A computer-readable program comprising: a recognition step of recognizing voice data input by the processing of the input step based on the second information generated by the processing of the second generation step. Recording medium on which is recorded.