JP2009222969A - Speech recognition robot and control method for speech recognition robot - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a speech recognition robot and a control method for the speech recognition robot, capable of making a speaker understand a cause of failure of the speech recognition, when the speech recognition fails. <P>SOLUTION: The speech recognition robot includes: a receiving section for receiving speech spoken by the speaker; a speech recognition section for recognizing contents of the received speech; an imaging section for taking an image in a direction which the speech is received, to obtain the taken image as an image data; a face detecting section for detecting a face of the speaker who exists in the taken image; an extracting section for extracting motion of a specified part from the detected face; a determining section for determining a speech receiving status based on the motion of the extracted specified part; and an output section for outputting a warning signal based on the determined speech receiving status, when the speech recognition fails. The warning signal which makes the speaker understand the cause of failure of the speech recognition is output, by assuring that the received speech is surely spoken by the speaker. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a conversation-responsive speech recognition robot that recognizes the content of speech uttered by a speaker and responds to the speaker, and a method for controlling such a speech recognition robot.

  In recent years, it has received conversational content for humans (speakers) as speech data, recognized the content, and output a response sentence corresponding to the content as speech. Speech recognition systems are being developed. Such a speech recognition system stores a large amount of speech data to be uttered as a response sentence in a storage area inside the system, selects speech data most related to the content of the recognized speech, and selects the selected speech data as speech. Speak. Such a speech recognition system acquires the spoken content as speech data, and divides the speech data into clauses of a predetermined language, so that the response most suitable for the content is based on the order and proximity of each clause. Select a sentence. (For example, Patent Document 1)

  On the other hand, robots having a voice recognition function using such a voice recognition system are also being used for customer service. Such a speech recognition robot does not always understand the content spoken by humans, so that if speech recognition fails, the spoken humans can understand that speech recognition has failed. The robot is caused to perform a specific operation (for example, Patent Document 2).

JP 2004-109323 A JP 2002-116792 A

  However, the reason why a robot having such a voice recognition function fails in voice recognition is that it inputs voice in addition to those caused by human voice strength (volume) and voice content problems. Some are due to software problems such as hardware (microphone) failures and driver failures. Of these reasons for failing speech recognition, speech recognition cannot be performed correctly even if the input speech is corrected for reasons other than those caused by the strength or content of the speech uttered by humans. Therefore, as described above, when the robot informs the human (speaker) that the speech recognition has not been successful, the speech recognition fails again even if the speech on the speaker side is adjusted.

  The present invention has been made to solve such a problem. When the speech recognition is not successful, the speech recognition robot capable of allowing the speaker to understand the cause of the speech recognition failure. And it aims at providing the control method of a speech recognition robot.

  A voice recognition robot according to the present invention is a voice recognition robot including a receiving unit that receives voice uttered by a speaker, and a voice recognition unit that recognizes the content of the received voice. An imaging unit that captures an image and acquires the captured image as image data, a face detection unit that detects a face of a speaker existing in the captured image, and an extraction unit that extracts a motion of a specific part from the detected face A determination unit that determines the reception state of the voice based on the extracted movement of the specific part; and an output unit that outputs a warning signal based on the determined reception state of the voice when the voice recognition is not successful. It is characterized by providing.

  Such a voice recognition robot can confirm that the received voice is surely uttered by the speaker when the voice recognition of the received voice is not successful. Therefore, when speech is not recognized despite the voice being spoken by the speaker, it is not due to the strength or content of the speaker's speech but to the hardware or software reason inside the robot. Judgment can be made. Therefore, by including in the warning signal to be output that the reason why the speech recognition failed is not due to the strength or content of the utterance of the speaker, but to the reason for hardware or software inside the robot, On the other hand, it is possible to understand the cause of the voice recognition failure.

  The specific part extracted by the extraction unit may be any part as long as it can be determined that the speaker has uttered voice, but is preferably the lip included in the detected face. . Extracts the movement of the speaker's lips and determines whether the received voice is uttered from the speaker by determining whether the received voice is uttered by the speaker. It becomes possible to do.

  Moreover, when the extracted specific site | part is a lip, the said determination part may determine the receiving state of an audio | voice based on the open / closed degree of the extracted lip. In this way, it is possible to easily confirm that the received voice is surely uttered from the speaker by determining whether or not the speaker's lips are opened and closed when the voice is received. .

  The face detection unit may detect the speaker's face by any means, but identifies the positions of the eyes and lips contained in the speaker's face and based on these positions. Thus, the entire face may be detected by estimating the contour of the face. In this way, even if there is another person or object near the face of the person (speaker) and the outline of the person's face cannot be specified by simply measuring the distance from the robot, It is possible to specify the contour of the face only by detecting the positions of the eyes and mouth.

  In addition, such a speech recognition robot preferably further includes a direction recognition unit that recognizes the detected face direction based on the positions of both eyes included in the face detected by the face detection unit. In this way, it is possible to easily determine the orientation of the speaker's face without considering the position of the entire face and the entire body of the speaker, and a plurality of human beings in the direction in which the voice is received. If it exists, the speaker can be identified by recognizing the direction of each detected face. Note that one or a plurality of microphones having directivity may be used as means for specifying the direction in which the voice is received. The voice recognition robot configured as described above is preferable because it can easily and accurately detect the direction in which the voice is received.

  Further, such a voice recognition robot is configured to change the imaging direction so that the imaging unit continues to be positioned substantially at the center in the image obtained by imaging the face detected by the face detection unit. And more preferred. In this way, even if the speaker moves while speaking to the robot and changes the relative position with the robot, the face of the speaker can be detected following the movement. Further, when such a speech recognition robot is configured as a humanoid type, if the imaging unit is provided at a position corresponding to the face of the robot, a person who is always a conversation target (speaker) ), It is possible to make it appear as if the conversation is continued while looking at the face of the speaker.

  Note that such a voice recognition robot may be fixed to a floor surface or the like, but may be configured to include moving means and be movable within a predetermined area. Since such a speech recognition robot can move while changing its own position in a state where the speaker is specified, for example, it is used for a guidance robot that performs a guidance operation for performing guidance while performing movement in a venue or the like. It becomes possible.

  The present invention also provides a robot control method for controlling a voice recognition robot that receives voice uttered by a speaker and recognizes the contents thereof. Specifically, such a robot control method captures an image in the direction in which the voice is received, acquires an image of the captured image as image data, and detects a face of a speaker present in the captured image. A detection step, an extraction step for extracting the movement of the specific part from the detected face, a determination step for determining the state of voice input based on the extracted movement of the specific part, and when the voice recognition is not successful And an output step of outputting a warning signal based on the determined result.

  By using such a robot control method, it is possible to confirm that the received voice is surely uttered by the speaker when the voice recognition of the received voice is not successful. For this reason, when speech is not recognized despite the voice being spoken by the speaker, it is not due to the strength or content of the speaker's utterance but to the hardware or software reason inside the robot. I understand. Therefore, by including in the warning signal to be output that the reason why the speech recognition failed is not due to the strength or content of the utterance of the speaker, but to the reason for hardware or software inside the robot, On the other hand, it is possible to understand the cause of the voice recognition failure.

  The specific part extracted in the extraction step may be any part as long as it can be determined that the speaker uttered the voice, but is preferably the lip included in the detected face. . Extracts the movement of the speaker's lips and determines whether the received voice is uttered from the speaker by determining whether the received voice is uttered by the speaker. It becomes possible to do.

  Further, in the determination step, the sound reception state may be determined based on the opening / closing degree of the lips. In this way, it is possible to easily confirm that the received voice is surely uttered from the speaker by determining whether or not the speaker's lips are opened and closed when the voice is received. .

  As described above, according to the present invention, a speaker who talks to a speech recognition robot can understand the cause of the failure of speech recognition of the robot.

Embodiment 1 of the Invention
Hereinafter, a voice recognition robot and a method for controlling the voice recognition robot according to the first embodiment of the present invention will be described with reference to FIGS. 1 to 5. In this embodiment, a voice recognition robot (hereinafter simply referred to as a robot) will be described using an example in which the robot is movable by wheel driving and the upper body is a humanoid type robot.

  A robot 10 shown in FIG. 1 is a robot whose upper body is configured as a humanoid type, and includes a head 11, a body 12, a right arm 13, a left arm 14, a waist 15, and a wheel drive unit 20 as a moving means. ing.

  The head 11 is provided with cameras 111 and 112 as imaging units on the left and right sides of the front surface thereof, and is provided with an antenna 113 on the side surface thereof and a speaker 114 as an output unit on the lower side of the front surface. Microphones 115 and 116 are provided as receiving units for inputting signals. Further, the head 11 is connected to the body 12 so as to be able to rotate in the left-right direction within a plane horizontal to the floor surface, and the imaging range of the cameras 111 and 112 can be changed by rotating the head 11. It can be changed according to the situation and the surrounding environment can be grasped.

  Cameras 111 and 112 provided on the head 11 visually recognize the surrounding environment, and output image data obtained as a result of the visual recognition to a control computer described later. As such a camera, for example, a digital camera that images the surrounding environment and acquires the captured image as a digital signal can be used.

  The antenna 113 is used to receive a position signal for recognizing the absolute position of the robot 10 and to transmit a signal indicating the current self position and the self state. Transmission / reception of these pieces of information is performed with a robot monitoring system (not shown). A control computer described later recognizes its absolute position in real time based on a position signal received by the antenna 113. The position signal obtained in this way is used when determining a moving path and direction.

  The speaker 114 is provided below the front surface of the head 11 and outputs sound data appropriately selected from the sound data file included in the storage area of the control computer. The audio file output from the speaker 114 is composed of a plurality of files whose contents respond when spoken to a speaker, as described later, in addition to information such as a guide. Then, an appropriate one of these audio files is appropriately selected for the speaker and output to the outside through the speaker 114.

  The microphones 115 and 116 each have a plurality of so-called directional microphones that can acquire sound from a certain direction in the horizontal direction, and the sounds uttered in the surroundings are relative to the speech recognition robot 1. It is possible to roughly specify from which direction the signal is transmitted. These microphones 115, 116 are provided on the left and right side surfaces of the head 11, acquire voices emitted around the voice recognition robot 1, take them as voice data, and output them to the control computer 120.

  The body 12 includes the control computer 120 as described above and includes a battery (not shown) for supplying power to each component of the robot.

  As shown in FIG. 2, the body 12 recognizes the contents of digital signals as image data input from the cameras 111 and 112 and the contents of audio signals input from the microphones 115 and 116. A control computer 120 as a control unit for performing various operations and a battery (not shown) for supplying power for operating each component including the computer 120 are provided. An arithmetic processing unit (not shown) included in the control computer 120 detects the face of the speaker who uttered the voice signal from the digital signal as the image data input from the cameras 111 and 112 by image processing, and further detects the face. Eyes and lips from the selected face. Detailed procedures regarding these image processes will be described later.

  Moreover, the right arm part 13 and the left arm part 14 are attached to the left and right side surfaces of the body part 12, and include a plurality of joint parts that are driven by a motor part (not shown) at various points such as the elbow part, the wrist part, and the finger part. Then, the posture can be changed by changing the joint drive amount of these joints by a signal from the control computer 120, and desired operations such as object gripping and direction instruction can be performed. The shape of each arm driven by the joint is stored in the control computer in advance, and when the operation is performed by driving the joint, the space occupied by the arm by the operation is calculated by the arithmetic processing unit. Shall be.

  The waist 15 is fixed above the wheel drive unit 20 and is rotatably attached to the bottom surface of the body unit 12 by a driving force such as a motor. The wheel drive unit 20 and the body unit 12 are relatively connected to each other. The posture can be changed.

  Further, as shown in FIG. 2, the wheel drive unit 20 is composed of a pair of opposed wheels 21, 21 and an opposed two-wheel vehicle including a caster 22 in front thereof. The robot 10 can move while these postures are supported horizontally by the wheels 21 and 21 and the caster 22. Further, the wheel drive unit 20 includes motors 23 and 23 for driving the wheels 21 and 21, respectively, and counters 24 and 24 for detecting the rotation speed of each wheel. The wheel drive unit configured as described above is controlled by the control computer 120 independently of the drive amounts of the wheels 21, 21, and travels straight, curves (turns), moves backwards, rotates on the spot (the middle point of both wheels). It is possible to perform a movement operation such as turning around the center, and autonomously determine a moving speed and a moving direction.

  As shown in FIG. 3, the control computer 120 exists in an image captured by the voice recognition unit 121 that recognizes the content from the input voice data and the cameras 111 and 112 provided in the head 10. A face detection unit 122 for detecting a person's face, a direction recognition unit 123 for recognizing the orientation of the face detected by the face detection unit 122, an extraction unit 124 for extracting a lip portion as a specific part from the detected face, and extraction A determination unit 125 that determines the state of voice reception based on the movement of the specific part, a voice synthesis unit 126 that generates response sentence data to be output as a warning signal, and a response composed of a predetermined program and a plurality of response sentence data A control unit 127 having a storage area 127a for storing a sentence database and the like is provided.

  The voice recognition unit 121 converts the voice acquired from the microphones 115 and 116 into voice data such as a WAVE file, divides the voice data into speech sections, and stores a word database in which each syllable is stored in the storage area 127a. Use to replace with a word. Then, the words included in each utterance section and the word order thereof are analyzed, and the sentence closest to the analyzed voice data is selected from the many sentences stored in the storage area. If the degree of approximation between the selected text and the voice data is greater than or equal to a predetermined value, the analyzed voice data is recognized as the same content as the selected text, and indicates that the acquired voice is equal to the selected text Output a signal. If the closest sentence is less than the predetermined degree of approximation, it is determined that the corresponding sentence is not stored in the storage area, and a signal indicating that the content of the acquired voice has not been recognized is output.

  In addition, the process for dividing | segmenting audio | voice data for every speech area can use the audio | voice feature-value represented, for example by the MFCC (Mel-frequency Cepstrum Coefficient) parameter. As an example of the MFCC parameter, for example, voice data of 16 [bit], 16 [Hz] is framed at a predetermined minute time interval (for example, 20 [ms]), a static feature MFCC 12 dimension, a dynamic feature MFCC 12 dimension, What is represented by a total of 25 dimensional feature values of 1 dimensional power can be used. Such MFCC parameters are calculated from the input voice data, and voice segment discrimination data including 25 normal distributions is calculated from the MFCC parameters. Then, the degree of similarity (for example, using Mahalanobis distance, etc.) is calculated for each dimension of the speech segment determination data and the MFCC parameter, and the average value obtained as a result of repeating this calculation for a predetermined time is used as a predetermined threshold value. To determine the utterance interval.

  The voice recognition unit 121 has a function of determining whether or not the input voice data is a voice input. Arbitrary information processing can be used as a method for determining whether or not the voice is input. For example, the volume of the input voice intersects the zero volume line per certain time (the number of zero crossings). , And a method of discriminating voice input when the number of zero crossings exceeds a predetermined threshold, or a mean square of amplitude values of input voice data is calculated for a certain time, and the value is a threshold A method of discriminating from voice input when exceeding the threshold can be used.

  The face detection unit 122 detects only the face portion of a person from image data obtained by imaging with the cameras 111 and 112. The face detection unit 122 first extracts the eyes and mouth (lips) included in the face of a person, and estimates an edge corresponding to the contour of the face based on these positions. Then, an area surrounded by the estimated face contour is detected as a human face.

  The direction recognizing unit 123 determines in which direction the detected face is viewed from the robot based on the position of the eyes included in the face of the person detected by the face detecting unit 122, that is, the relative distance and relative direction from the robot. Can be estimated. Specifically, the center positions of the right eye and the left eye included in the face of the person are specified, and the midpoint on the line segment connecting the center positions is specified. Then, a direction perpendicular to the line segment connecting each center position is obtained from this midpoint in a plane parallel to the floor including the line segment connecting each center position, and this direction is determined as the gaze direction, that is, the speaker's direction. The direction the face is facing.

  Furthermore, the direction recognizing unit 123 directs the detected face toward the robot itself from the relative position and the direction of each face included in the captured image detected by the face detecting unit 122. Recognize whether or not Specifically, based on the position of the robot itself (for example, the center point of the head 11), a predetermined width is given to the direction in which each face faces, and in detail, each direction is centered. It is assumed that a width is given by a minute angle (for example, 5 degrees) in the horizontal direction to the floor surface. In this way, it is determined whether or not each face is facing the robot itself, and those not facing the robot itself are considered not to be speakers.

  The extraction unit 124 extracts the lips as a specific part from the face in the image determined to be facing the robot by the direction recognition unit 123. Any process can be used as the process for extracting the lip portion. For example, a large number of lip shape data is stored in advance, and the shape represented by the shape data in the recognized face is stored. For example, a process of extracting an approximated portion as a lip is used. Note that the extraction unit 124 acquires the extracted lip image data as time-sequential continuous frames and stores them in the storage area 127.

  The determination unit 125 determines whether or not the lips are opened and closed from the continuous frames represented by the image data indicating the shape of the lips extracted by the extraction unit 124. Such a determination method is based on the average of correlation values with the images represented in the frames from the current frame to the Nth previous frame, for N frames obtained continuously as shown in FIG. What is determined is used. Such an average V (t) is represented by the following (formula 1) where p (t) is the current frame, R (ti) is the i-th previous frame, and S is the function for calculating the correlation value. Is represented by:

  Such calculation of the average V (t) is repeated for each predetermined number of frames (M times), and the average of correlation values of M frames is compared with a predetermined threshold value. Is determined to be open / closed. Note that the determination result as to whether or not the lips are open / closed is transmitted to the voice synthesizer 126 and is used for determination when selecting voice data to be output.

  The speech synthesizer 126 reads out the most appropriate response sentence data from among a large number of response sentence data groups stored in advance in the storage area corresponding to the acquired speech content recognized by the speech recognition unit 121. The sound file is converted and output to the outside through the speaker 114. At this time, if necessary, an operation such as a gesture may be performed together with the sound output using the arm portions (the right arm 12 and the left arm 13). In addition, when the speech synthesis unit 126 fails to recognize the received speech, the speech synthesis unit 126 selects appropriate response text data based on the determination result transmitted from the determination unit 125 and outputs it as speech information.

  The control unit 127 performs processing such as storage, processing, and output on the above-described image data and sound data based on the stored program, and also the driving amount of the motor that drives the arm unit (right arm unit and left arm unit) The movement of the wheel drive unit 20 is controlled. In particular, based on a program for determining a movement route stored in the storage area 127a, the movement route is appropriately determined according to the surrounding environment. Although details are omitted, the control unit 127 autonomously selects a moving direction, a route plan, or an action that can be taken based on information about a recognized external object when performing a moving operation. can do.

  Next, after the voice recognition robot 10 configured as described above recognizes the voice uttered by the speaker, determines the voice input state, and then outputs a warning signal based on the determined result. This will be described with reference to the flowchart shown in FIG.

  As shown in FIG. 5, first, the voice recognition robot is in a state of waiting for a sound to be input. When a sound emitted from the outside toward the voice recognition robot is received in this state (STEP 101), the microphone 115 and 116 specify the direction in which the received sound was emitted (the direction in which the robot received the sound) (STEP 102). Then, the head 11 is rotated so that the front surface of the head 11 is positioned in the specified direction, and the direction in which the sound is emitted is captured by the cameras 111 and 112 to acquire image data (STEP 103). Image data obtained by the cameras 111 and 112 is transmitted to the control unit 127 and stored in the storage area 127a.

  Next, it is determined whether or not the face detection unit 122 can detect a person's face from the stored image data (STEP 104). Here, if even one human face can be detected in the image data, the direction of the detected face is recognized by the direction recognition unit 123, and whether or not there is a face facing the voice recognition robot 1 is present. Judgment is made (STEP 105). If a face cannot be detected from the image data, the process returns to the initial state waiting for sound input.

  If there are faces facing the voice recognition robot 1, the distances of the faces from the voice recognition robot 1 are obtained, and the face existing at the closest position is selected (STEP 106). In this way, it is possible to specify a person to whom he / she should respond to a plurality of persons included in the captured screen. On the other hand, if the face facing the robot cannot be detected in STEP 105, or if it is determined in STEP 106 that there is no face facing the robot, there is no speaker who has spoken to the speech recognition robot 1. It returns to the initial state which waited for sound input.

  Next, only the lip portion is extracted from the selected face (STEP 107), and it is determined whether or not the opening / closing operation of the extracted lip is being performed (STEP 108). If the opening / closing operation is not performed, it is determined that no voice is being emitted to the voice recognition robot 1, and the process returns to STEP 101 and waits until the voice is received. If the opening / closing operation of the lips is performed, it is determined whether or not the received sound is a voice as a result of the voice recognition by the voice recognition unit 121 (STEP 109). If it is determined that the received sound is a voice, the voice recognition unit 121 subsequently performs voice recognition of the received voice (STEP 110). If it is determined that the received sound is not a sound, the process returns to the initial state where the sound input is waited.

  Next, it is determined whether or not the voice recognition by the voice recognition unit 121 is successful (STEP 111). If the speech recognition is successful, an appropriate response sentence for the recognized speech content is selected from the stored response unit database and output (STEP 112). On the other hand, if the voice recognition fails, it is assumed that the received voice cannot be recognized as a problem in the microphone (hardware) or software for receiving the voice, and a response message to inform that fact. (For example, “Please check the microphone”) is selected and output (STEP 212).

  Then, after the output of the response sentence is completed, it is determined again whether or not the voice reception is continued (STEP 113), and when it is continued, the process returns to the initial state waiting for the sound input. When the reception of sound is not continued, the reception of the sound is ended after performing a predetermined end process.

  As described above, according to the above-described embodiment, the voice recognition robot can determine that the received sound is the voice from the speaker. Therefore, the reason why the voice cannot be recognized is the hardware for receiving or It can be determined that a software problem is the cause. Therefore, by outputting a warning signal indicating that such a cause is pointed out, it becomes possible to understand the cause of the failure of speech recognition of the robot to the speaker.

  Note that the output of such a warning signal may be to generate voice data including the specific reason why voice recognition has failed, but in addition to this, gestures by driving joints such as arms For example, the reason why the speech recognition failed may be communicated to the speaker. Further, a light emitting element such as an LED may be provided in the robot, and the light emitting element may be blinked to notify the speaker that the microphone on the robot side is malfunctioning. In addition, when a speaker has a controller for remotely controlling the robot, and a signal is transmitted to the robot via the controller, a light emitting element, a vibration element, etc. provided in the controller are installed. You may make it operate | move and notify that the microphone etc. of a robot are out of order.

  In the above-described embodiment, the voice recognition robot is controlled so as to capture the direction in which the received sound is uttered and detect the speaker's face from the captured image data. Thus, even after the face is detected, control may be performed so as to correct the direction of the camera so that the detected face is positioned substantially in the center of the image obtained by imaging. In this way, while responding to the speaker, the part corresponding to the robot's “face” is always facing the speaker, so that the conversation is continued while looking at the speaker's face. The effect of being able to show is also obtained. In addition, even if the speaker moves during reception of the robot, the face of the speaker can be detected following the movement of the speaker. Such an operation (tracking) to follow the movement of the speaker is preferably continued until the speaker (the detected speaker's face) moves to a position more than a predetermined distance away from the voice recognition robot. is there. The distance for continuing such an operation of following the movement of the speaker may be determined based on the resolution of the imaging unit, the brightness of the surroundings, and the like.

  In the embodiment described above, as a method for recognizing the detected face orientation, a recognition method based on the center position of each face determined based on the position of the speaker's eyes is used. Thus, a method of recognizing the features of the speaker's eyes and mouth shape (such as the eyes and the mouth edge) by extracting with a neural network may be used. Further, the orientation of the face model may be obtained by comparing the three-dimensional position of the feature point in the image captured by the camera and the face model obtained from these feature points.

  Further, the moving means of the voice recognition robot is not limited to the combination of wheels and casters as described above, and may be an inverted pendulum type moving means composed of only wheels, and drives the legs. It may be a walking type moving means that moves.

  Also, as a means for recognizing the surrounding external environment provided in the robot, not only the imaging unit but also an optical camera such as a laser range sensor or CCD, or such a base station provided outside the robot is used. Environmental information may be transmitted to the robot.

  In the above-described embodiment, a signal from a robot monitoring system (not shown) is received by the antenna 113 and the absolute position of the robot is recognized. Instead, the distance moved by the number of rotations of the wheels, etc. Alternatively, the self-position may be calculated by calculating the azimuth and direction. Further, a more accurate self-position may be calculated by combining information from the robot monitoring system and the self-position calculated by the odometry method.

  In the present embodiment, the voice recognition unit uses a technique of converting the acquired voice into voice data, dividing the voice data into syllables, and replacing the divided syllables with words. The present invention is not limited to this, and many speech recognition methods currently used can be used. Furthermore, the method of selecting a response sentence that responds to the recognized voice content is not limited to the above embodiment, and other methods can be applied.

1 is an overall schematic diagram showing an appearance of a voice recognition robot according to a first embodiment of the present invention. It is the schematic which shows schematically the internal structure of the wheel drive part with which the speech recognition robot shown in FIG. 1 was equipped. FIG. 2 is a block diagram conceptually showing internal functions of a control unit provided in the voice recognition robot shown in FIG. 1. It is a figure which shows notionally a mode that it determines based on the average of a correlation value with the image represented by the flame | frame represented from the present frame to the frame before N about N frames acquired continuously. FIG. 3 is a flowchart showing a procedure from when the voice recognition robot shown in FIG. 1 recognizes a voice uttered by a speaker, determines a voice input state, and outputs a warning signal based on the determined result.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Voice recognition robot 100 ... Control part 121 ... Voice recognition part 122 ... Face detection part 123 ... Direction recognition part 124 ... Extraction part 125 ... Determination part 126 ... Speech synthesis unit 127... Control unit 127 a... Storage areas 111 and 112.
114 ... Output unit (speaker)
115, 116... Receiver (microphone)
20 ... Moving means (wheel drive unit)

Claims (10)

A voice recognition robot comprising: a receiving unit that receives voice uttered by a speaker; and a voice recognition unit that recognizes the content of the received voice,
An imaging unit that captures an image in the direction in which the sound is received and acquires the captured image as image data;
A face detection unit for detecting the face of the speaker present in the captured image;
An extraction unit for extracting the movement of a specific part from the detected face;
A determination unit that determines the reception state of the sound based on the extracted movement of the specific part;
A voice recognition robot comprising: an output unit that outputs a warning signal based on the determined voice reception state when voice recognition is not successful.   The voice recognition robot according to claim 1, wherein the specific part extracted by the extraction unit is a lip included in the detected face.   The voice recognition robot according to claim 2, wherein the determination unit determines a voice reception state based on a degree of opening / closing of the lips.   2. The face detection unit detects an entire face by specifying positions of eyes and lips included in a speaker's face and estimating a face outline based on the positions. 4. The speech recognition robot according to any one of items 1 to 3.   The voice recognition robot according to claim 1, further comprising a direction recognition unit that recognizes the direction of the detected face based on the positions of both eyes included in the detected face.   The voice recognition according to any one of claims 1 to 5, wherein the imaging unit is capable of changing the imaging direction so that the detected face continues to be positioned substantially at the center in the captured image. robot.   The voice recognition robot according to any one of claims 1 to 6, wherein the voice recognition robot further includes a moving unit, and is configured to be movable within a predetermined area. A control method for controlling a voice recognition robot that receives a voice uttered by a speaker and recognizes its contents,
An imaging step of capturing in the direction in which the sound is received and acquiring the captured image as image data;
A face detection step for detecting the face of the speaker present in the captured image;
An extraction step of extracting the movement of a specific part from the detected face;
A determination step of determining a voice reception state based on the extracted movement of the specific part;
An output step of outputting a warning signal based on the determined result when the speech recognition is not successful;
A method for controlling a speech recognition robot, comprising:   The method for controlling a voice recognition robot according to claim 8, wherein the specific part extracted in the extraction step is a lip included in the detected face.   The method for controlling a voice recognition robot according to claim 9, wherein in the determination step, a voice reception state is determined based on a degree of opening / closing of the lips.