JP2004170756A - Unit and method for robot control, recording medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To store a word registered in normal interaction while making it correspond to an operation of a robot and to make an unknown word and an operation of the robot correspond to each other to use it for the operation of the robot. <P>SOLUTION: When a speech recognition result matches a template 1 where pause names are registered in a step S121, the cluster ID and category of the word are stored in a step S122 while they are made to correspond to each other, and the cluster ID and an actuator control angle are stored in a step S123 while they are made to correspond to each other. When the speech recognition result matches a template 2 to command pauses in a step S124, information on the actuator control angle is extracted based upon the cluster ID in a step S125 and an actuator is controlled in a step S126. When the speech recognition result matches a template 3 to command storage of a character name and a user name in a step S127, the cluster ID and category are stored in a step S128 while they are made to correspond to each other and if no match is found in the template 3, specified response processing is carried out in a step S129. This invention is applicable to a robot. <P>COPYRIGHT: (C)2004,JPO

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to a robot control device and method, a recording medium, and a program, and particularly to a robot control device, method, and recording medium suitable for controlling a robot capable of voice recognition, voice output, and driving. , And the program.
[0002]
[Prior art]
In a dialogue system, there are many cases where a name of some kind is registered by voice. For example, a user registers his / her name, names a dialogue system, or inputs a place name or a shop name.
[0003]
Conventionally, as a method for easily realizing such voice registration, there is a method in which the mode is shifted to the registration mode by some command, and the normal interactive mode is returned when the registration is completed. In this case, for example, the mode is shifted to the registration mode by a voice command of “user name registration”, and when the user utters the name thereafter, the name is registered, and thereafter, the process of returning to the normal mode is performed.
[0004]
For example, by causing a voice-recognizable robot to take an action indicating that a name is to be given, the user is notified that the mode is switched to the robot name registration mode, and the input is performed after the action of the robot is controlled. There is a technique for detecting an optimal phoneme sequence from a recorded voice and registering the optimal phoneme sequence as a name (for example, see Patent Document 1).
[0005]
[Patent Document 1]
JP-A-2002-120177
[0006]
[Problems to be solved by the invention]
However, in such a voice registration method, the mode must be switched by a command, which is unnatural as a dialogue and has a problem that the user is troublesome. Further, when there are a plurality of naming targets, the number of commands is increased, which is more troublesome.
[0007]
For example, as described above, when the user is notified of the transition to the registration mode by the action of the robot, the action of the robot is changed according to the naming target, and the user grasps the relationship between the action of the robot and the registered content. Voice registration becomes very troublesome, for example.
[0008]
In addition, during the registration mode, the user word other than the name (for example, "Hello") If you've talked to, even the words of except for the name will be registered as the name. In addition, for example, when the user speaks with a word other than the name such as “My name is Taro” instead of the name “Taro”, the whole (“My name is Taro. )) Is registered as a name.
[0009]
Further, the registered word is only used for voice output processing using the registered word and recognition processing when the same word is uttered by the user after registration.
[0010]
The present invention has been made in view of such a situation, and a word registered in a normal dialogue is stored in association with a motion of a robot so that the word can be used for the motion of the robot. Things.
[0011]
[Means for Solving the Problems]
A first robot control device according to the present invention includes: a recognition unit that recognizes a continuous input voice; and, when it is determined that an unknown word is included in the recognition result recognized by the recognition unit, the unknown word corresponds to the unknown word. An acquisition unit for acquiring a word and a registration unit for registering the word acquired by the acquisition unit in association with information for controlling the operation of the robot are provided.
[0012]
Pattern determination means for determining whether or not the recognition result recognized by the recognition means matches a specific pattern can be further provided. By the pattern determination means, the recognition result matches the specific pattern. If it is determined that the word is present, the registration means can register the word in association with information for controlling the operation of the robot.
[0013]
Detection means for detecting the state of the robot may be further provided, and the detection means may detect the state of the robot at the time when the recognition result is determined to match the specific pattern. The registration means can associate and register a word and information for controlling the operation of the robot so as to be in the state of the robot detected by the detection means.
[0014]
Control means for controlling the driving of the robot, and pattern determining means for determining whether or not the recognition result recognized by the recognition means matches a specific pattern, further comprising: When it is determined that the recognition result matches the specific pattern, the control unit controls the driving of the robot based on information for controlling the operation of the robot registered in association with the word by the registration unit. It can be controlled.
[0015]
It is possible to further include a storage unit that classifies the words acquired by the acquisition unit into a plurality of categories and stores the words, and the registration unit stores the words stored in the predetermined category in the storage unit. The information can be registered in association with information for controlling the operation of the robot.
[0016]
A first robot control method according to the present invention includes a recognition step of recognizing a continuous input voice; a determination step of determining whether or not an unknown word is included in a recognition result recognized by the processing of the recognition step; When it is determined that the unknown word is included in the recognition result by the processing of the determining step, the acquiring step of acquiring the word corresponding to the unknown word, and the word acquired by the processing of the acquiring step are performed by the operation of the robot. And registering the information in association with information for controlling
[0017]
The program recorded on the first recording medium of the present invention determines a recognition step of recognizing a continuous input voice, and determines whether or not an unknown word is included in a recognition result recognized by the processing of the recognition step. When the recognition result is determined to include an unknown word by the processing of the determining step, the obtaining step of obtaining a word corresponding to the unknown word, and the word obtained by the processing of the obtaining step And registering the information in association with information for controlling the operation of the robot.
[0018]
A first program according to the present invention includes: a recognition step of recognizing a continuous input voice; a determination step of determining whether an unknown word is included in a recognition result recognized by the processing of the recognition step; When it is determined that the unknown word is included in the recognition result by the processing of the above, the acquiring step of acquiring the word corresponding to the unknown word, and controlling the operation of the robot with the word acquired by the processing of the acquiring step And a registration step of registering the information in association with the information to be performed.
[0019]
In the first robot control apparatus and method and the program according to the present invention, continuous input speech is recognized, and whether or not an unknown word is included in the recognition result is determined. If determined, a word corresponding to the unknown word is acquired, and the acquired word is registered in association with information for controlling the operation of the robot.
[0020]
A second robot control device according to the present invention includes a control unit that controls driving of the robot, a registration unit that registers information indicating a state of the robot in association with a corresponding word, and a robot whose driving is controlled by the control unit. Detecting means for detecting the state of the robot, voice synthesizing means for synthesizing the voice, and output means for outputting the voice synthesized by the voice synthesizing means, and the state of the robot detected by the detecting means is registered by the registering means. The voice synthesizing unit synthesizes a voice including a word associated with the information indicating the state of the robot registered by the registration unit when the information matches the information indicating the state of the registered robot.
[0021]
Input means for receiving a command from the user may be further provided, and the detecting means may detect the state of the robot when the input means does not receive an operation input for a predetermined time. Can be.
[0022]
A recognition unit for recognizing a continuous input voice, and a pattern determination unit for determining whether a recognition result by the recognition unit matches a specific pattern may be further provided. If it is determined that the pattern matches a specific pattern, the detecting means can detect the state of the robot.
[0023]
When it is determined that the unknown word is included in the recognition result recognized by the recognition unit, an acquisition unit that acquires a word corresponding to the unknown word can be further provided. A word corresponding to the unknown word acquired by the acquiring means and information indicating the state of the robot can be stored in association with each other.
[0024]
According to a second robot control method of the present invention, there is provided a detecting step for detecting a state of the robot, and whether or not the state of the robot detected by the processing of the detecting step matches the registered information registered in the registered information. When it is determined that the state of the robot matches the information indicating the state of the robot registered in the registration information by the processing of the determining step of determining the state of the robot, the information is associated with the information indicating the state of the robot. And a speech synthesizing step of synthesizing speech including the selected word.
[0025]
The program recorded on the second recording medium of the present invention includes a detection step of detecting a state of the robot and a state of the robot detected by the processing of the detection step matching the registration information registered in the registration information. A determining step of determining whether or not the robot is in conformity with the information indicating the state of the robot registered in the registration information; And synthesizing a speech including a word associated with the information indicating
[0026]
A second program according to the present invention includes a detecting step of detecting a state of the robot, and determining whether or not the state of the robot detected by the processing of the detecting step matches the registered information registered in the registered information. When it is determined that the state of the robot matches the information indicating the state of the robot registered in the registration information by the processing of the determining step and the determining step, the robot is associated with the information indicating the state of the robot. And a speech synthesizing step of synthesizing speech including a word.
[0027]
In the second robot control device and method, and the program according to the present invention, the state of the robot is detected, and it is determined whether or not the detected state of the robot matches the registered information registered in the registered information. If it is determined that the state of the robot matches the information indicating the state of the robot registered in the registration information, a voice including a word associated with the information indicating the state of the robot is synthesized. Is output.
[0028]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0029]
FIG. 1 is a front perspective view of a bipedal walking robot 1 to which the present invention is applied, and FIG. 2 is a perspective view of the robot 1 as viewed from the rear. FIG. 3 is a diagram for explaining the axis configuration of the robot 1.
[0030]
In the robot 1, a head unit 12 is disposed above a body unit 11, and an arm unit 13 </ b> A and an arm unit 13 </ b> B having the same configuration are provided at predetermined positions on the upper right and left sides of the body unit 11. And a leg unit 14A and a leg unit 14B having the same configuration are attached to predetermined positions on the lower left and right sides of the body unit 11, respectively. The head unit 12 is provided with a touch sensor 51.
[0031]
In the torso unit 11, the frame 21 forming the upper trunk and the waist base 22 forming the lower trunk are connected to each other via a waist joint mechanism 23, and the lower base 22 of the lower trunk is formed. By driving the actuator A1 and the actuator A2 of the waist joint mechanism 23 fixed to each other, the upper trunk is independently rotated around the orthogonal roll axis 24 and pitch axis 25 shown in FIG. It has been made possible.
[0032]
The head unit 12 is attached to the center of the upper surface of a shoulder base 26 fixed to the upper end of the frame 21 via a neck joint mechanism 27, and drives the actuator A3 and the actuator A4 of the neck joint mechanism 27, respectively. By doing so, they can be independently rotated around the orthogonal pitch axis 28 and yaw axis 29 shown in FIG.
[0033]
Further, the arm unit 13A and the arm unit 13B are respectively attached to the left and right of the shoulder base 26 via the shoulder joint mechanism 30, and the corresponding actuators A5 and A6 of the shoulder joint mechanism 30 are respectively attached. By driving, each can be independently rotated around a pitch axis 31 and a roll axis 32 which are orthogonal to each other as shown in FIG.
[0034]
In this case, in the arm unit 13A and the arm unit 13B, the actuator A8 forming the forearm is connected to the output shaft of the actuator A7 forming the upper arm via the elbow joint mechanism 44. Is configured by attaching a hand portion 34 to the tip of the.
[0035]
In the arm unit 13A and the arm unit 13B, the forearm can be rotated with respect to the yaw axis 35 shown in FIG. 3 by driving the actuator A7, and the forearm can be rotated by driving the actuator A8. Can be rotated with respect to a pitch axis 36 shown in FIG.
[0036]
The leg unit 14A and the leg unit 14B are respectively attached to the waist base 22 below the trunk via the hip joint mechanism 37, and by driving the actuators A9 to A11 of the corresponding hip joint mechanism 37, respectively. , The yaw axis 38, the roll axis 39, and the pitch axis 40, which are orthogonal to each other, can be rotated independently of each other.
[0037]
In the leg unit 14A and the leg unit 14B, the lower end of the frame 41 forming the thigh is connected to the frame 43 forming the lower leg through the knee joint mechanism 42, and The lower end is connected to the foot 45 via the ankle joint mechanism 44.
[0038]
Thereby, in the leg unit 14A and the leg unit 14B, the lower leg can be rotated with respect to the pitch axis 46 shown in FIG. 3 by driving the actuator A12 forming the knee joint mechanism 42. By driving the actuator A13 and the actuator A14 of the ankle joint mechanism 44, the foot 45 can be independently rotated with respect to the orthogonal pitch axis 47 and roll axis 48 shown in FIG. It has been made possible.
[0039]
On the back side of the waist base 22 that forms the lower trunk of the body unit 11, a control unit 52, which is a box containing a main control unit 61 and a peripheral circuit 62 (both shown in FIG. 4) described below, is provided. It is arranged.
[0040]
FIG. 4 is a diagram illustrating an actuator of the robot 1 and a control system thereof.
[0041]
The control unit 52 contains a main control unit 61 for controlling the operation of the entire robot 1, peripheral circuits 62 such as a power supply circuit and a communication circuit, and a battery 74 (FIG. 5).
[0042]
The control unit 52 is disposed in each of the constituent units (the body unit 11, the head unit 12, the arm unit 13A and the arm unit 13B, and the leg unit 14A and the leg unit 14B). It is connected to the sub-control units 63A to 63D, and supplies necessary power supply voltages to the sub-control units 63A to 63D and performs communication with the sub-control units 63A to 63D.
[0043]
Further, the sub-control units 63A to 63D are respectively connected to the actuators A1 to A14 in the corresponding constituent units, and based on various control commands supplied from the main control unit 61, the sub-control units 63A to 63D Are controlled to drive the actuators A1 to A14 to the designated state.
[0044]
FIG. 5 is a block diagram showing the internal configuration of the robot 1.
[0045]
The head unit 12 includes a CCD (Charge Coupled Device) camera 81 functioning as an “eye” of the robot 1, a microphone 82 functioning as an “ear”, an external sensor unit 71 including a touch sensor 51, and a “mouth”. And a speaker 72 functioning as a "" are provided at predetermined positions, and an internal sensor unit 73 including a battery sensor 91 and an acceleration sensor 92 is provided in the control unit 52.
[0046]
Then, the CCD camera 81 of the external sensor unit 71 captures an image of the surroundings, and sends out the obtained image signal S1A to the main control unit 61. The microphone 82 collects various command sounds such as “walk”, “stop” or “raise your right hand” given as a voice input from the user, and sends out the obtained voice signal S1B to the main control unit 61.
[0047]
The touch sensor 51 is provided, for example, on the upper part of the head unit 12 as shown in FIGS. 1 and 2, and receives a pressure applied by a physical action such as “stroke” or “hit” from the user. And sends the detection result to the main control unit 61 as a pressure detection signal S1C.
[0048]
The battery sensor 91 of the internal sensor unit 73 detects the remaining energy of the battery 74 at a predetermined cycle, and sends the detection result to the main control unit 61 as a remaining battery detection signal S2A. The acceleration sensor 92 detects the acceleration of the movement of the robot 1 in three axial directions (x-axis, y-axis, and z-axis) at a predetermined cycle, and outputs the detection result to the main control unit 61 as an acceleration detection signal S2B. Send out.
[0049]
The main control unit 61 receives an image signal S1A, an audio signal S1B, and a pressure detection signal S1C (hereinafter collectively referred to as an external sensor signal S1) supplied from the CCD camera 81, the microphone 82, and the touch sensor 51 of the external sensor unit 71, respectively. ), Based on the remaining battery level detection signal S2A and the acceleration detection signal S2B (hereinafter collectively referred to as an internal sensor signal S2) supplied from the battery sensor 91 and the acceleration sensor of the internal sensor unit 73, respectively. Then, the situation around and inside the robot 1, the command from the user, the presence or absence of the user's action, and the like are determined.
[0050]
Then, the main control unit 61 determines a situation around and inside the robot 1, a command from the user, or a result of the determination as to whether or not there is an action from the user, a control program stored in the internal memory 61 </ b> A in advance, or The action of the robot 1 is determined based on various control parameters stored in the external memory 75 loaded at that time, a control command based on the determination result is generated, and the corresponding sub-control unit 63A to sub-control is generated. It is sent to the unit 63D. The sub-control units 63A to 63D control the driving of the corresponding one of the actuators A1 to A14 based on the supplied control command. It is possible to perform an action such as walking, swinging the arm unit 13A, or raising the arm unit 13B, and alternately driving the leg unit 14A and the leg unit 14B. Become.
[0051]
Further, the main control unit 61 outputs a sound based on the sound signal S3 to the outside by giving a predetermined sound signal S3 to the speaker 72 as necessary. Further, the main controller 61 blinks the LED by outputting a drive signal to an LED (not shown) provided at a predetermined position of the head unit 12 and functioning as an apparent "eye".
[0052]
In this way, the robot 1 is capable of acting autonomously based on the surrounding and internal conditions, the presence / absence of a command from the user, and the presence or absence of an action.
[0053]
Next, FIG. 6 shows an example of a functional configuration of the main control unit 61 of FIG. Note that the functional configuration shown in FIG. 6 is realized by the main control unit 61 executing a control program stored in the memory 61A.
[0054]
The main control unit 61 accumulates the sensor input processing unit 101 for recognizing a specific external state, the recognition result of the sensor input processing unit 101, and stores a model of the emotion, instinct, or growth state of the robot 1. Based on a model storage unit 102, a table storage unit 104 for storing a table of voice recognition results and action contents, a recognition result of the sensor input processing unit 101, and a behavior of the robot 1 based on a table stored in the table storage unit 104. , A posture transition mechanism unit 105 that actually causes the robot 1 to perform an action based on the determination result of the behavior determination mechanism unit 103, and a voice synthesis unit 106 that generates a synthetic sound. I have.
[0055]
The sensor input processing unit 101 receives a specific external state, a specific action from a user, a user, based on a sound signal, an image signal, a pressure detection signal, and the like provided from the microphone 82, the CCD camera 81, the touch sensor 51, and the like. , And notifies the model storage unit 102 and the action determination mechanism unit 103 of state recognition information representing the recognition result.
[0056]
That is, the sensor input processing unit 101 has a voice recognition unit 101A, and the voice recognition unit 101A performs voice recognition on a voice signal given from the microphone 82. Then, the voice recognition unit 101A sends, for example, commands such as “walk”, “stop”, and “raise your right hand” and other voice recognition results to the model storage unit 102 and the action determination mechanism unit 103 as state recognition information. Notice.
[0057]
Further, the speech recognition unit 101A can newly recognize an out-of-vocabulary (OOV: Out of Vocabulary), and if necessary, outputs an ID associated with the recognized unknown word to the action determining unit rear unit 103. To supply. Details of the recognition of the unknown word will be described later.
[0058]
Further, the sensor input processing unit 101 has an image recognition unit 101B, and the image recognition unit 101B performs an image recognition process using an image signal given from the CCD camera 81. Then, when the image recognition unit 101B detects, for example, a “red round object” or a “plane that is perpendicular to the ground and equal to or more than a predetermined height” as a result of the processing, “there is a ball” or “ An image recognition result such as “there is a wall” is notified to the model storage unit 102 and the action determination mechanism unit 103 as state recognition information.
[0059]
Further, the sensor input processing unit 101 has a pressure processing unit 101C, and the pressure processing unit 101C processes a pressure detection signal given from the touch sensor 51. Then, as a result of the processing, when the pressure processing section 101C detects a pressure that is equal to or more than a predetermined threshold value and is short-time, the pressure processing section 101C recognizes that “hitting” has been performed, When a long-term pressure is detected, it is recognized as “stroke (praised)”, and the recognition result is notified to the model storage unit 102 and the action determination mechanism unit 103 as state recognition information.
[0060]
The model storage unit 102 stores and manages an emotion model, an instinct model, and a growth model expressing the emotion, instinct, and growth state of the robot 1, respectively.
[0061]
Here, the emotion model indicates, for example, the state (degree) of emotions such as “joy”, “sadness”, “anger”, and “fun” in a predetermined range (for example, −1.0 to 1.. 0, etc.), and the values are changed based on the state recognition information from the sensor input processing unit 101 or the passage of time. The instinct model expresses the state (degree) of instinct such as “appetite”, “sleep desire”, and “exercise desire” by a value in a predetermined range, and the state recognition information from the sensor input processing unit 101. The value is changed based on the time or the passage of time. The growth model represents, for example, a growth state (degree) such as “childhood”, “adolescence”, “mature”, “elderly”, etc., by a value in a predetermined range. The value is changed on the basis of the state recognition information or the passage of time.
[0062]
The model storage unit 102 sends the emotion, instinct, and growth state represented by the values of the emotion model, instinct model, and growth model as described above to the behavior determination mechanism unit 103 as state information.
[0063]
In addition to the state recognition information supplied from the sensor input processing unit 101 to the model storage unit 102, the current or past behavior of the robot 1, specifically, for example, “ The behavior information indicating the content of the behavior such as "walked" is supplied. Even if the same state recognition information is given, the model storage unit 102 responds to the behavior of the robot 1 indicated by the behavior information. , Different status information is generated.
[0064]
That is, for example, when the robot 1 greets the user and strokes the head, the behavior information that the robot 1 greets the user and the state recognition information that the head is stroked are stored in the model storage unit. In this case, in the model storage unit 102, the value of the emotion model representing “joy” is increased.
[0065]
On the other hand, when the robot 1 is stroked on the head while performing any work, the behavior information indicating that the robot 1 is performing the work and state recognition information indicating that the robot has been stroked on the head are given to the model storage unit 102. In this case, the model storage unit 102 does not change the value of the emotion model representing “joy”.
[0066]
As described above, the model storage unit 102 sets the value of the emotion model while referring to not only the state recognition information but also the behavior information indicating the current or past behavior of the robot 1. Thereby, for example, when the user strokes the head with the intention of mischief while performing any task, an unnatural change in emotion such as increasing the value of the emotion model representing “joy” occurs. Can be avoided.
[0067]
Note that the model storage unit 102 also increases and decreases the values of the instinct model and the growth model based on both the state recognition information and the behavior information, as in the case of the emotion model. Further, the model storage unit 102 increases or decreases the values of the emotion model, the instinct model, and the growth model based on the values of other models.
[0068]
The action determination mechanism unit 103 stores the table stored in the table storage unit 104 as necessary based on the state recognition information from the sensor input processing unit 101, the state information from the model storage unit 102, the passage of time, and the like. With reference to the next action, the next action is determined, and the content of the determined action is sent to the attitude transition mechanism unit 105 as action command information.
[0069]
In addition, the action determination mechanism unit 103, from the voice recognition unit 101A of the sensor input processing unit 101, matches a predetermined first rule such as, for example, "This is a <OOV (unknown word)>pause". When a voice input is received, the signals indicating the states of the actuators A1 to A14 supplied from the sub-control units 63A to 63D correspond to the unknown words indicating the pauses obtained as a result of the voice recognition. And stored in the table storage unit 104.
[0070]
Then, the action determining mechanism unit 103 receives, from the voice recognition unit 101A of the sensor input processing unit 101, a voice input that conforms to a predetermined second rule, such as “<OOV>”, for example. When <OOV> is stored in the table storage unit 104, the control information of the actuator corresponding to <OOV> stored in the table storage unit 104 is read and supplied to the posture transition mechanism unit 105.
[0071]
That is, the behavior determining mechanism unit 103 manages a finite state automaton in which the behavior that can be taken by the robot 1 corresponds to a state (state), as a behavior model that defines the behavior of the robot 1. The state in the finite state automaton is changed based on the state recognition information from the sensor input processing unit 101, the value of the emotion model, the instinct model, or the growth model in the model storage unit 102, the elapsed time, and the like, and corresponds to the state after the transition. Is determined as the next action to be taken.
[0072]
Here, when detecting that there is a predetermined trigger, the action determining mechanism unit 103 changes the state. That is, for example, when the time during which the action corresponding to the current state is being executed reaches a predetermined time, or when specific state recognition information is received, the action determining mechanism unit 103 is supplied from the model storage unit 102. The state is changed when the value of the emotion, instinct, or growth state indicated by the state information is equal to or less than a predetermined threshold.
[0073]
Note that, as described above, the action determination mechanism unit 103 performs not only the state recognition information from the sensor input processing unit 101 but also the emotion model in the model storage unit 102, the instinct model, the value of the growth model, and the like. Since the state in the action model is changed, even if the same state recognition information is input, the destination of the state changes depending on the value (state information) of the emotion model, the instinct model, and the growth model.
[0074]
As described above, the action determining mechanism 103 generates action command information for causing the robot 1 to speak, in addition to action command information for operating the head, limbs, and the like of the robot 1. The action command information for causing the robot 1 to speak is supplied to the speech synthesis unit 106, and the action command information supplied to the speech synthesis unit 106 includes the synthesized sound to be generated by the speech synthesis unit 106. The corresponding text and the like are included. Then, upon receiving the action command information from the action determination section 52, the speech synthesis section 106 generates a synthesized sound based on the text included in the action command information, and supplies the synthesized sound to the speaker 18 for output. As a result, from the speaker 18, for example, greeting to the user, such as "Hello", various requests to the user, or other audio output response to the call of the user, such as "What?" Is performed.
[0075]
The posture transition mechanism unit 105 generates posture transition information for transitioning the posture of the robot 1 from the current posture to the next posture based on the behavior command information supplied from the behavior determination mechanism unit 103, and This is sent to the sub-control units 63A to 63D.
[0076]
FIG. 7 is a functional block diagram illustrating functions of the voice recognition unit 101A of the sensor input processing unit 101.
[0077]
A voice signal based on an utterance from a user is input to the voice recognition processing unit 121. The voice recognition processing unit 121 recognizes the input voice signal and outputs a text as a result of the voice recognition. And other accompanying information to the dialog control unit 123 and the word acquisition unit 124 as necessary. Details of the voice recognition processing unit 121 will be described later with reference to FIG.
[0078]
The word acquiring unit 124 automatically stores acoustic features of words (unknown words) not registered in the recognition dictionary included in the speech recognition processing unit 121 so that the speech of the words can be recognized thereafter. I do.
[0079]
That is, the word acquisition unit 124 obtains a pronunciation corresponding to an unknown word portion of the input voice by a phonological typewriter, and classifies it into several clusters. Each cluster has an ID and a representative phoneme sequence, and is managed by the ID. The state of the cluster at this time will be described with reference to FIG.
[0080]
For example, it is assumed that there are three input voices “red”, “blue”, and “guts”. In this case, the word acquiring unit 124 classifies the three times of speech into three clusters of “red” cluster 141, “blue” cluster 142, and “guts” cluster 143, and each cluster has A representative phoneme sequence (“a / k / a,“ a / o ”,“ g / a / t / t / u ”) in the example of FIG. 8 and an ID (“ 1 / , "2", "3").
[0081]
Here, when the voice “Aka” is input again, since the corresponding cluster already exists, the word acquiring unit 124 classifies the input voice into the “Aka” cluster 141 and does not generate a new cluster. On the other hand, when the voice of “kuro” is input, there is no corresponding cluster, so the word acquiring unit 124 newly generates a cluster 144 corresponding to “kuro”, and the cluster includes a representative A typical phoneme sequence (“k / u / r / o” in the example of FIG. 8) and an ID (“4” in the example of FIG. 8) are added.
[0082]
Therefore, whether or not the input speech is an unacquired word can be determined based on whether or not a new cluster has been generated. The details of such a word acquisition process are disclosed in Japanese Patent Application No. 2001-97842 previously proposed by the present applicant.
[0083]
The associative storage unit 122 stores information such as a category such as whether the registered word (unknown word) is a user name or a character name. For example, in the example of FIG. 9, a cluster ID and a category name are stored in association with each other. In the case of the example of FIG. 9, for example, cluster ID “1” and cluster ID “4” correspond to the category of “user name”, and cluster ID “2” corresponds to the category of “character name”. , Cluster ID “3” corresponds to the category of “pause name”.
[0084]
The dialog control unit 123 understands the content of the user's utterance from the output of the speech recognition processing unit 121, and controls the registration of a word (unknown word) based on the result of the understanding. Further, the dialog control unit 123 controls the subsequent dialog based on the information of the registered words stored in the associative storage unit 122 so that the registered words can be recognized.
[0085]
Thus, the speech recognized by the speech recognition unit 101A is supplied to the action determination mechanism unit 103.
[0086]
Here, it is described that the word acquisition unit 124 generates a cluster from the pronunciation obtained by the phoneme typewriter, and performs matching with the cluster when inputting an unknown word thereafter. Without generating a cluster, an ID is added to the phoneme sequence itself (for example, “k / u / r / o”) as an unknown word, and when a new unknown word is input, comparison is performed using the phoneme sequence. Then, it may be determined whether or not the input unknown word matches any of the unknown words to which the ID has already been added.
[0087]
FIG. 10 illustrates a configuration example of the speech recognition processing unit 121.
[0088]
The utterance of the user is input to the microphone 82, where the utterance is converted into an audio signal as an electric signal. This audio signal is supplied to an AD (Analog Digital) converter 171. The AD converter 171 samples and quantizes the audio signal that is an analog signal from the microphone 82, and converts it into audio data that is a digital signal. This audio data is supplied to the feature amount extraction unit 172.
[0089]
The feature amount extraction unit 172 extracts, for each appropriate frame, feature parameters such as a spectrum, a power linear prediction coefficient, a cepstrum coefficient, and a line spectrum pair from the audio data from the AD conversion unit 171, and outputs a matching unit 173 and a phoneme. It is supplied to the typewriter section 174.
[0090]
The matching unit 173 refers to the acoustic model database 181, the dictionary database 182, and the language model database 183 as needed based on the feature parameters from the feature amount extraction unit 172, and refers to the speech (input speech) input to the microphone 82. Find the word string that is closest to ().
[0091]
The acoustic model database 181 stores acoustic models representing acoustic features such as individual phonemes and syllables in the language of the speech to be recognized. As the acoustic model, for example, HMM (Hidden Markov Model) or the like can be used. The dictionary database 182 stores, for each word (phrase) to be recognized, a word dictionary in which information about the pronunciation is described, and a model in which a chain relation between phonemes and syllables is described.
[0092]
Note that the word here is a unit that is more convenient to be treated as one unit in the recognition processing, and does not always match a linguistic word. For example, “Taro-kun” may be treated as one word, or may be treated as two words “Taro” and “Kimi”. In addition, may be dealing with more is a major unit of "Hello Taro" or the like as one word.
[0093]
A phoneme is one that is more conveniently processed acoustically as one unit, and does not always match a phonetic phoneme or phoneme. For example, the "to" portion of "Tokyo" may be represented by three phonetic symbols "t / o / u", or "t:" using the symbol "o:" which is a long sound of "o". / O: ". Alternatively, it can be expressed as “t / o / o”. In addition, a symbol representing silence is prepared, and it is further described as "silence before speech", "short silence section between speech", "silence of speech word", "silence of" tsu "part" And a symbol may be prepared for each.
[0094]
The language model database 183 describes information on how words registered in the word dictionary of the dictionary database 182 are linked (connected).
[0095]
The phoneme typewriter unit 174 acquires a phoneme sequence corresponding to the input speech based on the feature parameters supplied from the feature amount extraction unit 172. The phoneme typewriter unit 174, for example, reads “w / a / t / a / sh / i / n / o / n / a / m / a / e / w / from the voice“ My name is Taro. ” a / t / a / r / o: / d / e / s / u ”is obtained. An existing phoneme typewriter can be used.
[0096]
Note that, instead of the phoneme typewriter unit 174, another configuration that can acquire a phoneme sequence for an arbitrary voice may be used. For example, speech recognition in units of Japanese syllables (a, i, u, ..., k, ..., n) or speech in units of subwords, which are larger than phonemes but smaller than words It is also possible to use recognition or the like.
[0097]
The control unit 175 controls the operations of the AD conversion unit 171, the feature amount extraction unit 172, the matching unit 173, and the phoneme typewriter unit 174.
[0098]
Next, a process when the robot 1 receives a voice input will be described with reference to a flowchart of FIG.
[0099]
In step S1, the voice recognition processing unit 121 of the voice recognition unit 101A determines whether a voice has been input from the microphone 82. If it is determined in step S1 that no voice input has been received, the process of step S1 is repeated until it is determined that a voice input has been received.
[0100]
In step S2, a speech recognition process described later with reference to FIG. 12 is executed.
[0101]
In step S3, the control unit 175 of the speech recognition processing unit 121 determines whether the word string recognized in step S2 includes an unknown word.
[0102]
When it is determined in step S4 that an unknown word is included, in step S4, the control unit 175 controls the word obtaining unit 124 to execute a word obtaining process described later with reference to FIG.
[0103]
In step S5, the dialog control unit 123 executes a template matching process, which will be described later with reference to FIG. 19, using the word obtained in the process of step S4, and the process ends.
[0104]
If it is determined in step S3 that an unknown word is not included, in step S5, the control unit 175 of the voice recognition processing unit 121 outputs the recognized voice to the action determination mechanism unit 103 and, if necessary, , To the model storage unit 102. The action determining mechanism unit 103 executes a predetermined response process based on the supplied speech recognition result.
[0105]
Specifically, for example, when the recognized voice is “forward”, the action determining mechanism unit 103 determines the action of the robot 1 that matches the action model that defines the action of the robot 1. The content of the determined action is sent to the attitude transition mechanism unit 105 as action command information. The posture transition mechanism unit 105 generates necessary control information of the actuators A1 to A14, supplies the control information to the sub-control units 63A to 63D, and drives the corresponding one of the actuators A1 to A14. And causes the robot 1 to execute a “forward” action.
[0106]
Alternatively, if the speech cannot be correctly recognized due to reasons such as not matching the grammar, the action determining mechanism unit 103 generates action command information for causing the robot 1 to utter "What?" The sound is supplied to the synthesizing unit 106 and the speaker 72 outputs the sound “What?”. In addition, the action determining mechanism unit 103 generates the motion command information for tilting the head unit 12 to the side (to make the neck crouch) at the same time as outputting the voice "What?" It is sent to the mechanism unit 105. The posture transition mechanism unit 105 generates control information for causing the actuator A3 and the actuator A4 to tilt the head, supplies the control information to the sub-control unit 63B, drives the actuator A3 and the actuator A4, and causes the robot 1 to “ You may be made to pose "kick your neck".
[0107]
Next, the speech recognition processing executed in step S2 in FIG. 11 will be described with reference to the flowchart in FIG.
[0108]
In step S21, the AD conversion unit 171 converts the analog audio signal supplied from the microphone 82 into audio data that is a digital signal, and supplies the digital audio data to the feature extraction unit 172.
[0109]
The feature amount extraction unit 172 receives the audio data from the AD conversion unit 171 in step S22, and extracts, in step S23, for each appropriate frame, for example, a feature parameter such as a spectrum, power, and a time variation thereof. Then, the data is supplied to the matching unit 173.
[0110]
In step S24, the matching unit 173 connects some of the word models stored in the dictionary database 182.
[0111]
In step S25, a word string generation process described later with reference to FIG. 17 is executed. Note that the words constituting this word string include not only known words registered in the dictionary database 182 but also “<OOV>” which is a symbol representing an unregistered unknown word.
[0112]
In step S26, the phoneme typewriter unit 174 performs recognition in units of phonemes on the feature parameters extracted in the process of step S23, independently of the processes of steps S24 and S25, and converts the phoneme sequence into units. Output. For example, when the voice “My name is Taro (unknown word)” is input, the phoneme typewriter unit 174 outputs “w / a / t / a / sh / i / n / o / n / a”. / M / a / e / w / a / t / a / r / o: / d / e / s / u ".
[0113]
In step S27, the matching unit 173 calculates an acoustic score for each word string generated in step S25. <OOV> A method of calculating an acoustic score for a word string that does not include an (unknown word) is an existing method, that is, by inputting a speech feature parameter for each word string (a word model is linked). A method of calculating degrees is used. On the other hand, with the existing method, the acoustic score of the voice section corresponding to <OOV> cannot be obtained (because there is no word model corresponding to <OOV> in advance). In the calculation of the score, for the speech section, a method of taking out the acoustic score of the same section from the recognition result of the phoneme typewriter and correcting the value is adopted as the acoustic score of <OOV>. Is used. The matching unit 173 further integrates the acoustic score of <OOV> with the acoustic score of another known word part, and sets it as the acoustic score of the word string.
[0114]
In step S28, the matching unit 173 leaves the top m (m ≦ n) word strings with high acoustic scores as candidate word strings. In step S29, the matching unit 173 calculates a language score for each candidate word string with reference to the language model database 183. The language score indicates how appropriate a word string that is a candidate for a recognition result is as a word. Here, a method of calculating the language score will be described in detail.
[0115]
Since the speech recognition processing unit 121 of the present invention can also recognize an unknown word, the language model needs to correspond to the unknown word. For example, a case where a grammar corresponding to an unknown word or a finite state automaton (FSA) is used, and a case where a tri-gram (one of statistical language models) also corresponding to an unknown word is used. Will be described.
[0116]
An example of the grammar will be described with reference to FIG. This grammar is described in BNF (Backus Naur Form). In FIG. 13, ＄ A ”represents“ variable ”, and“ A | B ”represents“ A or B ”. “[A]” means “A can be omitted”, and {A} means “A is repeated 0 or more times”.
[0117]
<OOV> is a symbol representing an unknown word. By describing <OOV> in the grammar, it is possible to deal with a word string including an unknown word. "@ACTION" defines a word corresponding to the motion when the correspondence between the name and the motion content, such as "stand up", "seated", "bow", "greeting", etc., is set in advance. ing.
[0118]
In this grammar, "<start> / Hello / <end>" ( "/" is a separator between words), "<start> / goodbye / <end>", "<start> / I / Roh / name / is / <OOV> / is / <end> ”, a word string that applies to the grammar stored in the database is accepted (parsed with this grammar), but“ <head> / kun / no / < Word strings that do not apply to the grammar stored in the database, such as “OOV> / name / <end>”, are not accepted (not parsed by this grammar). Note that “<head>” and “<end>” are special symbols representing silence before and after the utterance, respectively.
[0119]
A parser (analyzer) is used to calculate a language score using this grammar. The parser divides the word string into a word string that can accept grammar and a word string that cannot be accepted. That is, for example, an acceptable word string is given a language score of 1, and an unacceptable word string is given a language score of 0.
[0120]
Therefore, for example, "<head> / me / name / is / <OOV> (t / a / r / o:) / is / <end>" and "<head> / me / name / When there are two word strings of / <OOV> (j / i / r / o:) / is / <end>, both are “<head> / me / name / ha / <OOV> The language score is calculated after being replaced with // is / <end>, and both are output as language score 1 (acceptance).
[0121]
In addition, to determine whether or not the grammar of the word string can be accepted, the grammar is converted in advance to an equivalent (or approximate) finite state automaton (hereinafter referred to as FSA), and each word string is It can also be realized by determining whether it can be accepted.
[0122]
FIG. 14 shows an example in which the grammar of FIG. 13 is converted into an equivalent FSA. FSA is a directed graph composed of states (nodes) and paths (arcs). As shown in FIG. 14, S1 is a start state, and S20 is an end state. In addition, as in FIG. 13, a word corresponding to an actual operation is registered in “@ACTION”.
[0123]
A word is assigned to a path, and when a transition is made from a predetermined state to the next state, the path consumes this word. However, the path to which “ε” is assigned is a special transition that does not consume a word (hereinafter, referred to as ε transition). For example, in “<head> / me / is / <OOV> / is / <end>”, the state transits from the initial state S1 to the state S2, the <head> is consumed, and the state transits from the state S2 to the state S3. Therefore, "I" is consumed, but the word is not consumed since the transition from the state S3 to the state S5 is an ε transition. That is, it is possible to skip from the state S3 to the state S5 and transition to the next state S6.
[0124]
Whether or not a predetermined word string can be accepted by this FSA is determined by whether or not it can start from the initial state S1 and reach the end state S20.
[0125]
That is, for example, in "<head> / me / name / is / <OOV> / is / <end>", the state transitions from the initial state S1 to the state S2, and the word "<head>" is consumed. You. Next, the state transits from the state S2 to the state S3, and the word “I” is consumed. Hereinafter, similarly, the state sequentially transitions from the state S3 to the state S4, from the state S4 to the state S5, from the state S5 to the state S6, from the state S6 to the state S7, and to “NO”, “NAME”, “HA”, “ <00V> ”is consumed one after another. Further, the state transits from the state S7 to the state S19, "is" is consumed, the state transits from the state S19 to the state S20, "<end>" is consumed, and finally reaches the end state S20. Therefore, "<head> / me / name / is / <OOV> / is / <end>" is accepted by the FSA.
[0126]
However, “<head> / you / no / <OOV> / name / <end>” transits from state S1 to state S2, from state S2 to state S8, from state S8 to state S9, and returns “< Although it is consumed up to "head>", "kun", and "no", it is not possible to transit further, so that it cannot reach the end state S16. Therefore, “<head> / you / no / <OOV> / name / <end>” is not accepted by the FSA (not accepted).
[0127]
Further, in the "<start> / Goodbye / <end>", "<start> / Hi / <end>" are both a transition from the state S1 to the state S2, the word "<start>" is consumed, The transition from the state S2 to the state S19 causes the word “Goodbye” or “Hello” to be consumed, and the transition from the state S19 to the state S20 results in the consumption of “<end>”.
[0128]
"<Head> / this is / <OOV> / pause / dayo / <end>" or a similar word string such as "<head> / this is / <OOV> / is / <end>" Transitions from state S1 to state S2, consumes the word "<head>", transitions from state S2 to state S13, transitions word "this" from state S13 to state S14, “<OOV>” is consumed, the state transitions from the state S13 to the state S14, the word “pause” is consumed, or the state transitions from the state S13 to the state S14, and the state transitions from the state S14 to the state S19. , The word “da” or “is” is consumed, or ε transitions from the state S14 to the state S19, and finally, the state transitions from the state S19 to the state S20, and “<end>” is consumed. .
[0129]
Then, "<head> / <OOV> (character name) / @ ACTION / set / <end>" or a similar word string such as "<head> / <OOV> (character name) / <OOV> In (pose name) / pause / do / <end>, etc., the state transitions from state S1 to state S2, and the word “<head>” is consumed. From state S2 to state S16, the state transitions to state S16. <OOV> ”(character name of the unknown word) is consumed, and the state transits from the state S16 to the state S18, and the word“ $ ACTION ”indicating a predetermined operation is consumed, or the state S16 is changed to the state S17. Then, after the word “<OOV>” (the pose name of the unknown word) is consumed, the state transitions from the state S17 to the state S18, or the word “pause” is consumed. Then, the state transits from the state S18 to the state S19, and the word “shi” is consumed. Finally, the state transits from the state S19 to the state S20, and “<end>” is consumed.
[0130]
Further, an example of calculating a language score when a tri-gram that is one of statistical language models is used as a language model will be described with reference to FIG. The statistical language model is a language model that obtains the generation probability of the word string and uses it as a language score. That is, for example, the language score of “<head> / me / name / is / <OOV> / is / <end>” of the language model shown in FIG. It is represented by the word string generation probability. This is further expressed as a product of the conditional probabilities, as shown in rows 3-6. Note that, for example, “P (no | <head> me)” is based on the condition that the word immediately before “no” is “me” and the word immediately before “me” is “<head>”. , “Of” appear.
[0131]
Further, in the tri-gram, the expressions shown in the third to sixth lines in FIG. 15 are approximated by conditional probabilities of three consecutive words as shown in the seventh to ninth lines. These probability values are obtained by referring to a tri-gram database as shown in FIG. This tri-gram database is obtained by analyzing a large amount of texts in advance.
[0132]
In the example of FIG. 16, the probabilities P (w3 | w1w2) of three consecutive words w1, w2, w3 are shown. For example, if the three words w1, w2, and w3 are respectively “<head>”, “me”, and “no”, the probability value is 0.12, and “me”, “no”, and “name” , The probability value is 0.01, and if it is "<OOV>", "is", or "<end>", the probability value is 0.87.
[0133]
Of course, “P (W)” and “P (w2 | w1)” are similarly obtained in advance.
[0134]
In this way, by performing entry processing on <OOV> in the language model, a language score can be calculated for a word string including <OOV>. Therefore, a symbol <OOV> can be output as a recognition result.
[0135]
Also, when another type of language model is used, a language score can be similarly calculated for a word string including <OOV> by performing entry processing for <OOV>.
[0136]
Further, even when a language model having no <OOV> entry is used, a language score can be calculated by using a mechanism that maps <OOV> to an appropriate word in the language model. For example, even if a tri-gram database in which “P (<OOV> | I)” does not exist is used, accessing the database with “P (Taro | I)” and calculating the probability described there The language score can be calculated by regarding the value as “P (<OOV> | I)”.
[0137]
Returning to the description of the voice recognition processing in FIG. In step S30, the matching unit 173 integrates the acoustic score and the language score. In step S31, the matching unit 173 selects a candidate word string having the best score based on a score obtained by integrating both the acoustic score and the language score obtained in step S30, and outputs it as a recognition result.
[0138]
When the finite state automaton is used as the language model, the integration processing in step S30 is performed by deleting the word string when the language score is 0 and leaving the word string as it is when the language score is not 0. You may.
[0139]
Next, the word string generation processing executed in step S25 in FIG. 12 will be described with reference to the flowchart in FIG.
[0140]
In step S61, the matching unit 173 determines an acoustic score obtained by matching a certain section of the input voice with a known word registered in the dictionary database 182, and a result obtained by the phoneme typewriter unit 174 (in this case, , "W / a / t / a / sh / i / n / o / n / a / m / a / e / w / a / t / a / r / o: / d / e / s / u" The acoustic score of both cases is calculated with the acoustic score of the middle section. The acoustic score indicates how close a word string that is a candidate for a speech recognition result and an input speech are as sounds.
[0141]
Then, an acoustic score obtained by matching a partial section of the input voice with a known word registered in the dictionary database 182 is compared with an acoustic score obtained by the phoneme typewriter unit 174. , Known words are performed in units of words, and matching in the phoneme typewriter unit 174 is performed in units of phonemes, and the scales are different. The acoustic score for each phoneme is larger.)
[0142]
Therefore, in order to enable comparison using the scale, the matching unit 173 corrects the acoustic score of the result obtained by the phoneme typewriter unit 174 in step S62.
[0143]
In step S62, for example, a process of multiplying the acoustic score from the phoneme typewriter unit 174 by a coefficient or reducing a constant value or a value proportional to the frame length is performed. Of course, since this processing is relative, it can be performed on an acoustic score obtained as a result of matching with a known word. The details of this process are disclosed in, for example, “OOV-Detection in Large Vocabulary System Using Automatically Delivered Foreign Documents as a Foreign-Wide-Folded Foreign-Defined-Folded-Folder-Folded-Words-Folding-Folded-Folder” in, for example, “OOV-Detection in Large Vocabulary System Useing Automatically Defined Foreign Document” in the document ““ EUROSPEECH99 Volume 1, Page 49-52 ””.
[0144]
In step S63, the matching unit 173 compares the two acoustic scores, and determines whether the acoustic score obtained as a result of recognition by the phoneme typewriter unit 174 is higher (excellent). If the acoustic score obtained as a result of recognition by the phonemic typewriter unit 174 is higher, the matching unit 173 estimates that section is an unknown word <OOV> in step S64.
[0145]
In step S63, when it is determined that the acoustic score of the result recognized by the phonological typewriter unit 174 is lower than the acoustic score of the result matched with the known word, in step S65, the matching unit 173 It is estimated that the section is a known word.
[0146]
That is, for example, for the section corresponding to “Taro”, the sound score of “t / a / r / o:” output from the phoneme typewriter unit 174 is compared with the sound score when matching is performed with a known word. If the acoustic score of “t / a / r / o:” is higher, “<OOV> (t / a / r / o :)” is output as a word corresponding to the voice section, and a known word is output. If the acoustic score is higher, the known word is output as a word corresponding to the speech section.
[0147]
After the processing of step S64 or step S65 is completed, in step S66, the matching unit 173 preferentially removes n word strings (concatenation of several word models) estimated to have a high acoustic score. After the generation, the process returns to step S26 in FIG.
[0148]
Through such processing, a word string used for calculating an acoustic score is generated in the processing of step S27 in FIG.
[0149]
Next, the word acquisition processing executed in step S4 in FIG. 11 will be described with reference to the flowchart in FIG.
[0150]
In step S <b> 91, the word acquiring unit 124 extracts a feature parameter of an unknown word (<OOV>) from the speech recognition processing unit 121.
[0151]
In step S92, the word acquiring unit 124 determines whether or not the unknown word belongs to the already acquired cluster. If it is determined in step S92 that the unknown word belongs to the already acquired cluster, there is no need to generate a new cluster, and the process proceeds to step S94.
[0152]
If it is determined in step S92 that the unknown word does not belong to the acquired cluster, in step S93, the word acquisition unit 124 generates a new cluster corresponding to the unknown word.
[0153]
If it is determined in step S92 that the unknown word belongs to the already-acquired cluster, or after the process of step S93 ends, in step S94, the word acquisition unit 124 determines the ID of the cluster to which the unknown word belongs by a voice recognition process. Then, the process proceeds to step S5 in FIG.
[0154]
By such processing, in the subsequent step S5 in FIG. 11, an unknown word to be matched with the template is obtained.
[0155]
In FIG. 18, it is determined whether or not an unknown word belongs to an already-acquired cluster from a feature amount parameter of an unknown word. If the unknown word does not belong to an already-acquired cluster, a new cluster is generated. However, without generating a cluster, for example, an ID may be associated with a phoneme sequence, and whether or not an unknown word has already been obtained may be determined by comparing phoneme sequences.
[0156]
Next, the template matching process executed in step S5 in FIG. 11 will be described with reference to the flowchart in FIG.
[0157]
In step S121, the dialog control unit 123 of the speech recognition unit 101A determines whether the supplied speech recognition result matches the template 1 for registering the unknown word as a pose name, that is, what the word string of the recognition result is. It is determined whether or not the pose name is registered.
[0158]
The template 1 will be described with reference to FIG. Note that in FIG. 20 and FIGS. 22, 23, and 24 described later, “/ A /” means “if character string A is included”, and “A | B” means “A or B ". “.” Represents “arbitrary character”, “A +” represents “one or more repetitions of A”, and “(.) +” Represents “arbitrary character string”.
[0159]
If the word string of the recognition result is, for example, a word string of “<head> / this is / <OOV> (g / a / t / t / u:) / pause / is / <end>”, this recognition is performed. Although the character string "This is <OOV>pose" generated from the result, since the regular expression "/ this matches <OOV>(category; pose name) /" of template 1, the corresponding "<OOV> The operation of “registering a cluster ID corresponding to (category; pose name) as a pose name” and the operation of “storing in a table together with the actuator control angle” at that time are executed.
[0160]
Specifically, for example, when the user puts both arms of the robot 1 up and utters “This is a guts (pause)”, an unknown word <guts (pause)> is obtained. Thus, it is detected that it corresponds to template 1.
[0161]
When it is determined in step S121 that the word matches the template 1, in step S122, the associative storage unit 122 stores the cluster ID of the word and the category in association with each other as described with reference to FIG. .
[0162]
In step S123, the voice recognition unit 101A supplies the cluster ID to the action determination mechanism unit 103. Further, the behavior determining mechanism unit 103 acquires the current states (angles) of the actuators A1 to A14 from the sub-control units 63A to 63D. The action determining mechanism unit 103 stores the table as shown in FIG. 21 in the table storage unit 104 in association with the supplied cluster ID and the information on the actuator control angle, and the process ends.
[0163]
Specifically, for example, when the user puts both arms of the robot 1 up and utters “This is a guts (pause)”, it is associated with <guts (pause)> The cluster ID is stored in the table storage unit 104 in association with the states (angles) of the actuators A1 to A14 with both arms of the robot 1 raised.
[0164]
If it is determined in step S121 that the template does not match the template 1, in step S124, the dialogue control unit 123 determines whether the supplied voice recognition result indicates to the robot 1 a pose associated with the unknown word. It is determined whether or not 2 is matched.
[0165]
The template 2 will be described with reference to FIG. If the word string of the recognition result is, for example, a word string of “<head> / <OOV> (b / a / n / z / a / i) / then / <end>”, it is generated from this recognition result. The matched character string “<OOV>” matches the regular expression “/ <OOV>(category; pose name) ++ /” of the template 2, so the corresponding “<OOV>(category; pose) The operation “execute action corresponding to <OOV>” is executed with reference to the table storage unit 104 based on the cluster ID corresponding to (name).
[0166]
If it is determined in step S124 that the word matches the template 2, in step S125, the speech recognition unit 101A supplies the recognized cluster ID of the unknown word to the action determination mechanism unit 103. The action determining mechanism unit 103 refers to the table storage unit 104, extracts information on the actuator control angle from the table described with reference to FIG. 21 based on the supplied cluster ID, and 105.
[0167]
In step S126, the posture transition mechanism unit 105 controls the necessary one of the actuators A1 to A14 to the corresponding one of the sub-control units 63A to 63D based on the information of the actuator control angle. Then, the process ends.
[0168]
Specifically, for example, when the user utters “Banzai” to the robot 1, an unknown word <Banzai> is acquired in the word acquisition unit 124, and the dialog control unit 123 transmits the unknown word <Banzai> to the template 2. Correspondence is detected. Then, the action determining mechanism unit 103 refers to the table storage unit 104, acquires the control information of the actuator corresponding to <Banzai>, and supplies the control information to the corresponding one of the sub-control units 63A to 63D. Thus, the robot 1 can be caused to pose corresponding to Banzai.
[0169]
If it is determined in step S124 that the template does not match the template 2, in step S127, the dialog control unit 123 determines that the supplied voice recognition result indicates that the template 3 instructs the robot 1 to store the character name and the user name. Judge whether or not it matches.
[0170]
The template 3 will be described with reference to FIG. For example, if the recognition result is the word string “<head> / you / no / name / was / <OOV> (t / a / r / o:) / dayo / <end>”, this recognition result Since the character string "Your name is <OOV>" corresponds to the regular expression "/Kimi(.)+<OOV>/", the cluster ID corresponding to <OOV> is a character string. The operation of “register as name” is executed. Also, if the recognition result is a word string “<head> / me // name / is / <OOV> (t / a / r / o:) / is / <end>”, Although the generated character string "My name is <OOV>", the regular expression "(I | I) (.) + Corresponds to <OOV>/", so the cluster ID corresponding to <OOV> The operation of “register as user name” is executed.
[0171]
If it is determined in step S127 that the tag matches the template 3, in step S128, the dialogue control unit 123 stores in the associative storage unit 122 the cluster ID of the word as the speech recognition result and the user name or character name. The process is terminated by storing the categories in association with each other.
[0172]
In some cases, depending on how the robot 1 is used, there is only one type of word to be registered (for example, only “user name”). In that case, the template and the associative storage unit 122 can be simplified. For example, the content of the template may be set as “if the recognition result includes <OOV>, store the ID”, and the associative storage unit 122 may store only the cluster ID.
[0173]
The dialogue control unit 123 reflects the information registered in the associative storage unit 122 in the subsequent dialogue determination process. For example, on the side of the robot 1, “it is determined whether or not“ character name ”is included in the utterance of the user. When it is necessary to perform a process of “execute” or a process of “the robot 1 speaks the user's name”, the dialog control unit 123 refers to the information recorded in the associative storage unit 122, and (An entry whose category name is “character name”) and a word corresponding to the user name (entry whose category name is “user name”) can be obtained.
[0174]
If it is determined in step S127 that the template 3 does not match, in step S129, the dialogue control unit 123 executes a predetermined response process corresponding to the input voice. That is, in this case, the registration process of the unknown word is not performed, and for example, a predetermined process corresponding to the input voice from the user, such as performing a walking operation in response to the command “walk”, is executed, The process ends.
[0175]
In this way, by comparing the acquired audio data with the regular expression, it is determined whether or not the voice data matches a predetermined template. When it is determined that the voice data matches any one of the templates, the template is The operation of the robot 1 is controlled so as to execute a corresponding action.
[0176]
Note that the contents of the templates 1 to 3 described with reference to FIGS. 20, 22 and 23 are not limited to this, and, for example, the regular expression of the template 2 may include “/ <OOV>(category; pose name) + It is also possible to add a pause / "/" or to add the regular expression of the template 2 to "/ <OOV>(category; pose name) (.) + /".
[0177]
When a grammar is used as a language model, a description equivalent to a phoneme typewriter can be incorporated in the grammar. An example of the grammar in this case will be described with reference to FIG. In the grammar shown in FIG. 24, the variable “$ PHONEME” on the first line is replaced with one of the phoneme symbols because all phonemes are connected by “|” meaning “or”. means. The variable “$ OOV” indicates that the variable “$ PHONEME” is repeated 0 times or more. That is, it means "any phoneme symbol is connected 0 or more times" and corresponds to a phoneme typewriter. Therefore, the variable “$ OOV” between “ha” and “is” on the third line can receive any sound.
[0178]
In the recognition result when the grammar shown in FIG. 24 is used, a portion corresponding to the variable “$ OOV” is output as a plurality of symbols. For example, the recognition result of "my name is Taro" is "<head> / me / name / is / t / a / r / o: / is / <end>". When this result is converted into “<head> / me / name / is / <OOV> (t / a / r / o:) / is”, the processing after step S3 in FIG. It can be performed in the same manner as when used.
[0179]
In the above description, a category is registered as information relating to an unknown word, but other information may be registered.
[0180]
Further, the robot 1 can not only execute processing based on a user's command, but also automatically operate even in a state where no operation is performed by the user. The operation may be set in advance, but, for example, the operation described above is performed at random by executing the operation stored in association with the unknown word, and the robot 1 , The speech corresponding to the pause may be automatically performed.
[0181]
The motion recognition utterance process executed by the robot 1 will be described with reference to the flowchart in FIG.
[0182]
In step S151, the voice recognition processing unit 121 of the voice recognition unit 101A determines whether a voice has been input from the microphone 82. If it is determined in step S151 that no voice input has been received, the process of step S151 is repeated until it is determined that a voice input has been received.
[0183]
In step S152, the voice recognition processing described with reference to FIG. 12 is performed.
[0184]
In step S153, the dialogue control unit 123 of the voice recognition unit 101A determines whether or not the supplied voice recognition result is “what is the pause?”.
[0185]
If it is determined in step S153 that the voice "What is the pause?" Has not been recognized, in step S154, the action determining mechanism unit 103 sets the internal timer (not shown) not to receive a command input from the user. In this state, it is determined whether or not notification has been given that a predetermined time has elapsed. If it is determined in step S154 that the timer has not notified the predetermined elapsed time, the process returns to step S151, and the subsequent processes are repeated.
[0186]
If it is determined in step S153 that the voice "What is the pause?" Has been recognized, or if it is determined in step S154 that the timer has notified the predetermined elapsed time, then in step S155, the action determining mechanism unit 103 recognizes the control angles of the actuators A1 to A14 based on the signals supplied from the sub-control units 63A to 63D.
[0187]
In step S156, the action determining mechanism unit 103 refers to the table indicating the correspondence between the pose name and the control angles of the actuators A1 to A14 stored in the table storage unit 104, and determines the actuator A1 recognized in step S155. Or search for a pose name corresponding to the control angle of the actuator A14.
[0188]
In step S157, the action determining mechanism unit 103 determines whether or not the table stored in the table storage unit 104 has a pose name corresponding to the control angles of the actuators A1 to A14 recognized in step S155. If it is determined in step S157 that there is no corresponding pose name, the process returns to step S151, and the subsequent processes are repeated.
[0189]
When it is determined in step S157 that there is a corresponding pose name, in step S158, the action determining mechanism unit 103 refers to the table storage unit 104 and extracts a corresponding unknown word whose category is a pose name. Then, the voice synthesizer 106 is controlled to synthesize a voice saying, for example, “I'm in a <OOV> pause”, and output it from the speaker 72, that is, make it speak, and the process goes to step S151. Return, and the subsequent processing is repeated.
[0190]
With the above-described processing, the robot 1 recognizes not only the response to the user's utterance but also the operation of itself, thereby changing the pose name corresponding to the control angles of the actuators A1 to A14 stored as a table. It can be used to speak automatically.
[0191]
In the processing described above, the case where the control angles of the actuators A1 to A14 in the stationary pose and the obtained unknown words are stored in association with each other is described. Can also be applied to a case where control information of the actuators A1 to A14 when a predetermined operation is performed (for example, a control angle of each actuator is represented in a time series) and an acquired unknown word are associated with each other. Needless to say,
[0192]
Further, in the above description, the case where the operation (driving and utterance) of the robot 1 having the actually drivable parts corresponding to the limbs and the like is described, for example, the (virtual) robot 1 displayed on the display is controlled. In this case, the present invention is also applicable.
[0193]
FIG. 26 illustrates a configuration example of a personal computer 201 that executes the above-described processing. The personal computer 201 has a built-in CPU (Central Processing Unit) 211. An input / output interface 115 is connected to the CPU 211 via a bus 214. The bus 214 is connected to a ROM (Read Only Memory) 212 and a RAM (Random Access Memory) 213.
[0194]
The input / output interface 215 includes an input unit 217 configured by input devices such as a mouse, a keyboard, a microphone, and an AD converter operated by a user, and an output unit configured by output devices such as a display, a speaker, and a DA converter. 216 are connected. Further, the input / output interface 215 is connected to a storage unit 218 including a hard disk drive for storing programs and various data, and a communication unit 219 for communicating data via a network represented by the Internet.
[0195]
A drive 220 that reads and writes data from and to a recording medium such as a magnetic disk 231, an optical disk 232, a magneto-optical disk 233, or a semiconductor memory 234 is connected to the input / output interface 215 as necessary.
[0196]
For example, the personal computer 201 can collect the voice generated by the user with the microphone of the input unit 217, and executes the voice recognition process by the processing of the CPU 211 in the same manner as described above. Further, based on the control of the CPU 211, for example, a character having the appearance of the robot 1 can be displayed on the display of the output unit 216. The CPU 211 recognizes the character displayed on the display of the output unit 216 based on the recognition processing result of the sound collected by the microphone of the input unit 217 or based on an operation performed by the user using the mouse of the input unit 217. It controls the movement (movement on the display) (that is, controls the display on the display of the output unit 216), generates voice data in response to the user's call, and outputs it from the speaker of the output unit 216.
[0197]
Then, when the user utters “This is a <happy> pose”, the CPU 211 recognizes the unknown word “happy”, associates the unique ID as the unknown word of the pose name category, and outputs The movement of the character displayed on the display of the unit 216 (that is, the image data displayed on the display) can be recorded, for example, in the RAM 213 together with the corresponding ID. Then, when the user instructs to “pause”, the display on the output unit 216 is changed to a display corresponding to the unknown word “happy” based on the information recorded in the RAM 213. By the processing of the CPU 211, the display on the output unit 216 is automatically changed to a display corresponding to the unknown word <happy>, and the speaker of the output unit 216 outputs a voice saying “This is a <happy> pause”. be able to.
[0198]
Further, the series of processes described above can be executed by software. The software is a computer in which a program constituting the software is built in dedicated hardware, or a general-purpose personal computer that can execute various functions by installing various programs. For example, it is installed from a recording medium.
[0199]
As shown in FIG. 26, this recording medium is a magnetic disk 231 (including a flexible disk) on which the program is recorded, and an optical disk 232 (CD-ROM), which are distributed in order to provide the user with the program, separately from the computer. It is composed of a package medium such as a ROM (Compact Disk-Read Only Memory), a DVD (including a Digital Versatile Disk), a magneto-optical disk 233 (including an MD (Mini-Disk) (trademark)), or a semiconductor memory 234. Is done.
[0200]
For example, the robot 1 control processing program that causes the personal computer 201 to execute the operation as the robot 1 control device to which the present invention is applied includes a magnetic disk 231 (including a floppy disk), an optical disk 232 (CD-ROM (Compact Disc-Read)). It is supplied to the personal computer 201 in a state where it is stored in the Only Memory (DVD) (including a Digital Versatile Disc), a magneto-optical disk 233 (including an MD (Mini Disc)), or a semiconductor memory 234, and is read out by the drive 220. Then, it is installed in a hard disk drive built in the storage unit 218. The voice processing program installed in the storage unit 218 is loaded from the storage unit 218 to the RAM 213 and executed by a command from the CPU 211 corresponding to a command from the user input to the input unit 217.
[0201]
In this specification, the step of describing a program to be recorded on a recording medium may be performed in a chronological order in the order described, but is not necessarily performed in a chronological order. This also includes processes executed individually.
[0202]
【The invention's effect】
Thus, according to the present invention, speech can be recognized. In particular, the recognized voice and information for controlling the operation of the robot can be registered in association with each other.
Further, according to another aspect of the present invention, in addition to detecting the state of the robot, a voice including a word corresponding to the state of the robot can be synthesized based on the detection result of the state of the robot.
[Brief description of the drawings]
FIG. 1 is a perspective view showing an external configuration of a robot to which the present invention is applied.
FIG. 2 is a rear perspective view showing the external configuration of the robot shown in FIG. 1;
FIG. 3 is a schematic diagram for explaining the robot of FIG. 1;
FIG. 4 is a block diagram showing an internal configuration of the robot shown in FIG. 1;
FIG. 5 is a block diagram for mainly explaining a portion related to control of the robot in FIG. 1;
FIG. 6 is a block diagram illustrating a configuration of a main control unit in FIG. 5;
FIG. 7 is a block diagram illustrating a configuration of a voice recognition unit in FIG. 6;
FIG. 8 is a diagram illustrating a state of a cluster.
FIG. 9 is a diagram for explaining word registration.
FIG. 10 is a block diagram illustrating a configuration of a voice recognition processing unit in FIG. 7;
FIG. 11 is a flowchart illustrating processing executed by a robot to which the present invention is applied.
FIG. 12 is a flowchart illustrating a speech recognition process.
FIG. 13 is a diagram illustrating an example of a grammar used in a language model database.
FIG. 14 is a diagram illustrating an example of a language model using a finite state automaton.
FIG. 15 is a diagram illustrating an example of calculating a language score using tri-gram.
FIG. 16 is a diagram illustrating an example of a tri-gram database.
FIG. 17 is a flowchart illustrating a word string generation process.
FIG. 18 is a flowchart illustrating a word acquisition process.
FIG. 19 is a flowchart illustrating a template matching process.
FIG. 20 is a diagram illustrating a template.
FIG. 21 is a diagram illustrating an actuator control angle registered in association with a cluster ID.
FIG. 22 is a diagram illustrating a template.
FIG. 23 is a diagram illustrating a template.
FIG. 24 is a diagram showing an example of a grammar incorporating a phoneme typewriter.
FIG. 25 is a flowchart illustrating a motion recognition utterance process.
FIG. 26 is a block diagram illustrating a configuration of a computer to which the present invention has been applied.
[Explanation of symbols]
1 robot, 61 main control unit, 63 sub control unit, 72 speakers, 82 microphone, 101 sensor input processing unit, 101A graduate student recognition unit, 103 action decision mechanism unit, 104 table storage unit, 105 posture transition mechanism unit, 106 voice synthesis , 121 Speech Recognition Processing Unit, 122 Associative Memory Unit, 123 Dialogue Control Unit, 124 Word Acquisition Unit, 172 Feature Extraction Unit, 173 Matching Unit, 174 Phoneme Typewriter Unit, 175 Control Unit, 181 Acoustic Model Database, 182 Dictionary Database, 183 language model database

Claims (15)

In a robot controller for controlling the operation of the robot,
A recognition means for recognizing a continuous input voice;
Acquisition means for acquiring a word corresponding to the unknown word, when the recognition result recognized by the recognition means is determined to include an unknown word,
A registration unit for registering the word acquired by the acquisition unit in association with information for controlling the operation of the robot. Further comprising a pattern determination means for determining whether the recognition result recognized by the recognition means matches a specific pattern,
When the pattern determination unit determines that the recognition result matches the specific pattern, the registration unit registers the word in association with information for controlling the operation of the robot. The robot control device according to claim 1, wherein Further comprising a detecting means for detecting the state of the robot,
The detecting means detects the state of the robot at the time when the recognition result is determined to match the specific pattern,
3. The robot according to claim 2, wherein the registration unit registers the word in association with information that controls an operation of the robot so as to be in a state of the robot detected by the detection unit. 4. Control device. Control means for controlling the driving of the robot,
Pattern determining means for determining whether the recognition result recognized by the recognition means matches a specific pattern,
When the pattern determination unit determines that the recognition result matches the specific pattern, the control unit controls the operation of the robot registered in association with the word by the registration unit. The robot control device according to claim 1, wherein driving of the robot is controlled based on information to be performed. A storage unit that stores the words acquired by the acquisition unit in a plurality of categories;
The robot control device according to claim 1, wherein the registration unit registers the word stored in a predetermined category in the storage unit in association with information for controlling an operation of the robot. In a robot control method of a robot control device that controls the operation of a robot,
A recognition step of recognizing a continuous input voice;
A determination step of determining whether an unknown word is included in the recognition result recognized by the processing of the recognition step,
An acquiring step of acquiring a word corresponding to the unknown word when the recognition result includes the unknown word, by the processing of the determining step;
A registration step of registering the word acquired by the processing of the acquisition step in association with information for controlling the operation of the robot. A program that causes a computer to execute processing for controlling the operation of the robot,
A recognition step of recognizing a continuous input voice;
A determination step of determining whether an unknown word is included in the recognition result recognized by the processing of the recognition step,
An acquiring step of acquiring a word corresponding to the unknown word when the recognition result includes the unknown word, by the processing of the determining step;
A registration step of registering the word acquired by the processing of the acquisition step in association with information for controlling the operation of the robot, and recording the computer-readable program. A program that causes a computer to execute processing for controlling the operation of the robot,
A recognition step of recognizing a continuous input voice;
A determination step of determining whether an unknown word is included in the recognition result recognized by the processing of the recognition step,
An acquiring step of acquiring a word corresponding to the unknown word when the recognition result includes the unknown word, by the processing of the determining step;
A registration step of registering the word acquired by the processing of the acquisition step in association with information for controlling the operation of the robot. In a robot controller for controlling the operation of the robot,
Control means for controlling the driving of the robot,
Registration means for registering information indicating the state of the robot in association with a corresponding word,
Detecting means for detecting a state of the robot whose driving is controlled by the control means;
Voice synthesis means for synthesizing voice;
Output means for outputting the voice synthesized by the voice synthesis means,
When the state of the robot detected by the detecting unit matches information indicating the state of the robot registered by the registering unit, the voice synthesizing unit determines the state of the robot registered by the registering unit. A robot control device for synthesizing a voice including the word associated with information indicating a state. Further comprising input means for receiving a command from a user,
The robot control device according to claim 9, wherein the detection unit detects a state of the robot when no operation input is received by the input unit for a predetermined time. A recognition means for recognizing a continuous input voice;
Pattern determining means for determining whether or not the recognition result by the recognition means matches a specific pattern,
The robot control device according to claim 9, wherein when the determination unit determines that the recognition result matches the specific pattern, the detection unit detects a state of the robot. The recognition result recognized by the recognition means, when it is determined that an unknown word is included, further comprising an obtaining means for obtaining a word corresponding to the unknown word,
10. The robot control device according to claim 9, wherein the registration unit associates and stores a word corresponding to the unknown word acquired by the acquisition unit and information indicating a state of the robot. Using registration information in which information indicating a state of the robot and a corresponding word are registered in association with each other, a robot control method of a robot control device that controls an operation of the robot,
A detecting step of detecting a state of the robot,
A determining step of determining whether the state of the robot detected by the processing of the detecting step matches the registered information registered in the registered information;
When it is determined that the state of the robot matches the information indicating the state of the robot registered in the registration information, the state of the robot is associated with the information indicating the state of the robot. A voice synthesizing step of synthesizing a voice including the word. A program that causes a computer to execute a process of controlling the operation of the robot by using registration information in which information indicating a state of the robot and a corresponding word are registered in association with each other,
A detecting step of detecting a state of the robot,
A determining step of determining whether the state of the robot detected by the processing of the detecting step matches the registered information registered in the registered information;
When it is determined that the state of the robot matches the information indicating the state of the robot registered in the registration information, the state of the robot is associated with the information indicating the state of the robot. A voice synthesizing step of synthesizing voice including the word, wherein a computer readable program is recorded. A program that causes a computer to execute a process of controlling the operation of the robot by using registration information in which information indicating a state of the robot and a corresponding word are registered in association with each other,
A detecting step of detecting a state of the robot,
A determining step of determining whether the state of the robot detected by the processing of the detecting step matches the registered information registered in the registered information;
When it is determined that the state of the robot matches the information indicating the state of the robot registered in the registration information, the state of the robot is associated with the information indicating the state of the robot. A voice synthesizing step of synthesizing a voice including the word.

Images