JP2007241304A - Device and method for recognizing voice, and program and recording medium therefor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a device and method for recognizing voice capable of raising user's recognition accuracy of voice uttered at a position part from a microphone almost without increasing computational complexity required for the voice recognition processing, and to provide a program and a recording medium therefor. <P>SOLUTION: A distance calculating part 47 finds the distance from an uttering user up to the microphone 21, and supplies the distance to a voice recognition part 41B. The voice recognition part 41B stores a set of acoustic models which takes into account acoustic environments produced from each voice data of collected voices uttered at two or more positions apart by two or more different distances. Then, the voice recognition part 41B selects a set of acoustic models closest to the distance supplied from the distance calculating part 47, and performs voice recognition using the set of acoustic models. <P>COPYRIGHT: (C)2007,JPO&INPIT

Description

  The present invention relates to a voice recognition device, a voice recognition method, a program, and a recording medium, and performs voice recognition processing using a set of acoustic models corresponding to the distance from a speaker to the voice recognition device, for example. The present invention relates to a speech recognition apparatus and speech recognition method, a program, and a recording medium that can improve speech recognition accuracy.

  Targeting tens of thousands of vocabulary words using statistical modeling techniques using large amounts of speech data and text data as CPU (Central Processing Unit) speeds up and memory capacity increase in recent years. A large vocabulary speech recognition system has been realized.

  In a speech recognition system including such a large-scale speech recognition system, highly accurate speech recognition is realized for speech uttered at a position close to a microphone (microphone) to which recognition target speech is input.

  However, for speech uttered at a position distant from the microphone, the voice recognition accuracy deteriorates as the distance from the microphone increases, due to noise mixing or the like.

  So, for example, Miki, Nishiura, Nakamura, Shikano, "Speech recognition under noise and reverberation by microphone array and HMM decomposition / synthesis", IEICE Transactions D-II, Vol.J83-DII No.11 pp. 2206-2214, Nov. 2000 (hereinafter referred to as “Document 1” as appropriate), as a first method, using a microphone array improves the SN (Signal Noise) ratio of speech uttered at a position away from the microphone. A voice recognition method is proposed.

  Also, for example, Shimizu, Iwata, Takeda, Itakura, “Space Diversity Robust Speech Recognition Considering Spatial Acoustic Characteristics”, IEICE Transactions D-II, Vol.J83-DII No.11 pp.2448-2456 , Nov. 2000 (hereinafter referred to as “Document 2” as appropriate), as a second method, a plurality of microphones are distributed in a room and an impulse response corresponding to the distance from the sound source to each microphone is used for learning. HMM (Hidden Markov Model) that takes into account the impulse response of each distance is prepared by learning using the voice data obtained by convolution with the voice data, and for each voice input to multiple microphones, each distance A speech recognition method has been proposed to calculate the likelihood of HMM considering the impulse response.

  However, in the first and second methods described above, there are restrictions on the installation of the microphone, and the application thereof may be difficult.

  That is, for example, in recent years, as a toy or the like, a robot that recognizes a voice uttered by a user and autonomously performs actions such as performing a certain gesture or outputting a synthesized sound based on the voice recognition result (In the present specification, including a stuffed animal) has been commercialized, but when the speech recognition device according to the first method is mounted on such a robot, the installation intervals of a plurality of microphones constituting a microphone array Physical constraints such as these become obstacles to robot miniaturization and design freedom.

  Further, when the speech recognition apparatus according to the second method is mounted on the robot, it is necessary to install a plurality of microphones for each room where the robot is used, which is not realistic. Furthermore, in the case of the second method, the likelihood of the HMM considering the impulse response of each distance must be calculated for the speech input from each of the plurality of microphones, which is large for speech recognition processing. A calculation amount is required.

  The present invention has been made in view of such a situation, and improves the recognition accuracy of a user's voice uttered at a position away from the microphone without substantially increasing the amount of calculation required for voice recognition processing. It is to be able to be made.

  The first speech recognition apparatus according to the present invention includes a distance calculation unit that obtains a distance to a sound source of the sound, and a plurality of different distances that are generated using a sound emitted from each of the sound sources separated by a plurality of different distances. Storage means storing a set of acoustic models in consideration of the acoustic environment, and a set of acoustic models corresponding to the distance obtained by the distance calculation means for each of a plurality of different distances stored in the storage means, An acquisition unit that acquires by selecting from a set of acoustic models in consideration of the acoustic environment; and a voice recognition unit that recognizes speech using the set of acoustic models acquired by the acquisition unit. To do.

  The first speech recognition method of the present invention includes a distance calculation step for obtaining a distance to a sound source of sound, and a set of acoustic models corresponding to the distance obtained in the distance calculation step, each of sound sources separated by a plurality of different distances. Considering the acoustic environment for each of a plurality of different distances stored in a storage means storing a set of acoustic models considering the acoustic environment for each of a plurality of different distances generated using the sound emitted from An acquisition step of acquiring by selecting from among the set of acoustic models, and a speech recognition step of recognizing speech using the set of acoustic models acquired in the acquisition step.

  The first program of the present invention generates a distance calculation step for obtaining a distance to a sound source of sound and a set of acoustic models corresponding to the distance obtained in the distance calculation step from each of a plurality of sound sources separated by a plurality of different distances. Acoustics taking into account the acoustic environment for each of a plurality of different distances stored in a storage means storing a set of acoustic models taking into account the acoustic environment for each of a plurality of different distances generated using the obtained speech A computer that performs a process including an acquisition step acquired by selecting from a set of models, and a speech recognition step of recognizing speech using the set of acoustic models acquired in the acquisition step; To do.

  The first recording medium of the present invention includes a distance calculation step for obtaining a distance to a sound source of sound, and a set of acoustic models corresponding to the distance obtained in the distance calculation step from a plurality of sound sources separated by a plurality of different distances. Considering the acoustic environment for each of a plurality of different distances stored in the storage means storing a set of acoustic models considering the acoustic environment for each of a plurality of different distances generated using the emitted speech A program for causing a computer to perform processing including an acquisition step acquired by selecting from a set of acoustic models and a speech recognition step of recognizing speech using the set of acoustic models acquired in the acquisition step is recorded It is characterized by being.

  The second speech recognition apparatus according to the present invention obtains a tap coefficient for realizing a distance calculating means for obtaining a distance to a sound source of speech and an inverse filter having a frequency characteristic corresponding to the distance obtained by the distance calculating means. A plurality of voices generated from sound sources emitted from a plurality of sound sources separated by a plurality of different distances, and a filter means for filtering voice using the tap coefficient acquired in the first acquisition means. Storage means storing a set of acoustic models in consideration of the acoustic environment for each different distance, and a plurality of different acoustic models stored in the storage means corresponding to the distance obtained by the distance calculation means A second acquisition means for acquiring by selecting from a set of acoustic models in consideration of the acoustic environment for each distance; Rings have been a voice, characterized in that it comprises a speech recognition means for recognizing with a set of acoustic models acquired in the second acquisition means.

  According to a second speech recognition method of the present invention, a distance calculating step for obtaining a distance to a sound source of speech, and a first coefficient for obtaining a tap coefficient for realizing an inverse filter having a frequency characteristic corresponding to the distance obtained in the distance calculating step. A set of acoustic models corresponding to the distance obtained in the distance calculating step and the filter step for filtering the voice using the tap coefficient acquired in the first acquiring step, and a plurality of different distances. For each of a plurality of different distances stored in a storage means storing a set of acoustic models taking into account the acoustic environment, generated for each of a plurality of different distances, generated using sound emitted from each remote sound source, A second acquisition step of acquiring by selecting from a set of acoustic models in consideration of the acoustic environment; Filtered speech in-up, characterized in that it comprises a speech recognition step recognizes using a set of acoustic models acquired in the second acquisition step.

  The second program of the present invention is a distance acquisition step for obtaining a distance to a sound source of sound, and a first acquisition for obtaining a tap coefficient for realizing an inverse filter of a frequency characteristic corresponding to the distance obtained in the distance calculation step. A set of acoustic models corresponding to the distance obtained in the step, a filter step for filtering speech using the tap coefficient obtained in the first obtaining step, and a distance obtained in the distance calculating step are separated by a plurality of different distances. Acoustic environment for each of a plurality of different distances stored in a storage means storing a set of acoustic models taking into account the acoustic environment for each of a plurality of different distances generated using sound emitted from each sound source A second acquisition step by selecting from a set of acoustic models taking into account The speech filtered in-flop, characterized in that to perform processing including a voice recognition step recognizes the computer using the set of acoustic models acquired in the second acquisition step

  The second recording medium of the present invention is a distance calculation step for obtaining a distance to a sound source of sound, and a first coefficient for obtaining a tap coefficient for realizing an inverse filter of a frequency characteristic corresponding to the distance obtained in the distance calculation step. A set of acoustic models corresponding to the distance obtained in the obtaining step, the filter step for filtering the sound using the tap coefficient obtained in the first obtaining step, and the distance calculating step are separated by a plurality of different distances. Acoustics for each of a plurality of different distances stored in a storage means storing a set of acoustic models in consideration of the acoustic environment, generated for each of a plurality of different distances, generated using the sound emitted from each sound source. A second acquisition step by selecting from a set of acoustic models taking into account the environment; and a filter step. In the filtered speech, and to perform processing including a second set speech recognition step recognizes using acoustic models acquired by the acquisition step to the computer, characterized in that it is recorded.

  In the first speech recognition apparatus, speech recognition method, and program of the present invention, the distance to the sound source of the sound is obtained, and a set of acoustic models corresponding to the distance and considering the acoustic environment is acquired. Then, speech is recognized using the acquired set of acoustic models.

  In the second speech recognition apparatus, speech recognition method, and program of the present invention, the distance to the sound source of the sound is obtained, and the tap coefficient that realizes the inverse filter of the frequency characteristic corresponding to the distance is obtained. Then, the acquired tap coefficient is used to filter the voice, and the filtered voice is recognized using a set of acoustic models that considers the acoustic environment corresponding to the distance of the voice to the sound source.

  According to the first speech recognition apparatus, speech recognition method, and program of the present invention, the distance to the sound source of the sound is obtained, and a set of acoustic models corresponding to the distance and considering the acoustic environment is acquired. Then, speech is recognized using the acquired set of acoustic models. Therefore, the voice recognition accuracy can be improved.

  According to the second speech recognition apparatus, speech recognition method, and program of the present invention, the distance to the sound source of the sound is obtained, and the tap coefficient that realizes the inverse filter of the frequency characteristic corresponding to the distance is obtained. Then, the acquired tap coefficient is used to filter the voice, and the filtered voice is recognized using a set of acoustic models that considers the acoustic environment corresponding to the distance of the voice to the sound source. Therefore, the voice recognition accuracy can be improved.

  FIG. 1 is a perspective view showing an example of the external configuration of a pet robot to which the present invention is applied, and FIG. 2 is a block diagram showing an example of the internal configuration thereof.

  In the embodiment of FIG. 1, the pet-type robot is a four-legged animal type robot, and mainly includes a torso unit 1, leg units 2A, 2B, 2C, 2D, a head unit 3, And the tail unit 4.

  Leg units 2A, 2B, 2C, 2D corresponding to the legs are connected to the front, rear, left and right of the body unit 1 corresponding to the body, respectively, and the front end portion and the rear end portion of the body unit 1 are respectively connected to A head unit 3 corresponding to the head and a tail unit 4 corresponding to the tail are connected.

  A back sensor 1 </ b> A is provided on the upper surface of the body unit 1. The head unit 3 is provided with a head sensor 3A at the top and a chin sensor 3B at the bottom. Note that each of the back sensor 1A, the head sensor 3A, and the chin sensor 3B is configured by a pressure sensor, and detects the pressure applied to the part.

  The tail unit 4 is attached to the body unit 1 so as to be swingable in the horizontal direction and the vertical direction.

  As shown in FIG. 2, the body unit 1 stores a controller 11, an A / D conversion unit 12, a D / A conversion unit 13, a communication unit 14, a semiconductor memory 15, a back sensor 1A, and the like.

  The controller 11 includes a CPU 11A that controls the overall operation of the controller 11, a memory 11B that stores an OS (Operating System), application programs, and other necessary data that the CPU 11A executes to control each unit. is doing.

  The A / D (Analog / Digital) converter 12 converts the analog signals output from the microphone 21, the CCD cameras 22L and 22R, the back sensor 1A, the head sensor 3A, and the chin sensor 3B into digital signals, Supply to the controller 11. A D / A (Digital / Analog) converter 13 converts the digital signal supplied from the controller 11 into an analog signal by D / A conversion, and supplies the analog signal to the speaker 23.

  The communication unit 14 performs communication control when communicating with the outside wirelessly or by wire. Accordingly, when the OS or application program is upgraded, the upgraded OS or application program is downloaded via the communication unit 14 and stored in the memory 11B, or a predetermined command is It can be received by the communication unit 14 and given to the CPU 11A.

  The semiconductor memory 15 is composed of, for example, an EEPROM (Electrically Erasable Programmable Read-only Memory) or the like, and is detachable from a slot (not shown) provided in the body unit 1. The semiconductor memory 15 stores, for example, an emotion model as described later.

  The back sensor 1A is provided in a part corresponding to the back of the pet-type robot in the torso unit 1. The back sensor 1A detects a pressure from the user applied thereto, and outputs a pressure detection signal corresponding to the pressure to the A / A. The data is output to the controller 11 via the D conversion unit 12.

  The body unit 1 also stores, for example, a battery (not shown) serving as a power source for the pet-type robot, a circuit for detecting the remaining battery capacity, and the like.

  In the head unit 3, as shown in FIG. 2, a microphone 21 corresponding to an "ear" that senses sound as a sensor that senses an external stimulus, and a "left eye" and "right eye" that sense light Corresponding CCD (Charge Coupled Device) cameras 22L and 22R, and a head sensor 3A and a jaw sensor 3B corresponding to a tactile sensation that senses a pressure applied by the user touching, for example, are provided at corresponding portions, respectively Yes. Further, the head unit 3 is provided with a speaker 23 corresponding to the “mouth” of the pet-type robot, for example, at a corresponding part.

  The joint parts of the leg units 2A to 2D, the connection parts of the leg units 2A to 2D and the body unit 1, the connection parts of the head unit 3 and the body unit 1, and the tail unit 4 and the body part An actuator is installed at a connecting portion of the unit 1 or the like. The actuator operates each unit based on an instruction from the controller 11. That is, for example, the leg units 2A to 2D are moved by the actuator, whereby the robot walks.

  The microphone 21 installed in the head unit 3 collects surrounding sounds (sounds) including utterances from the user and outputs the obtained sound signals to the controller 11 via the A / D conversion unit 12. To do. The CCD cameras 22 </ b> L and 22 </ b> R capture the surrounding situation and output the obtained image signal to the controller 11 via the A / D conversion unit 12. The head sensor 3A provided at the upper part of the head unit 3 and the chin sensor 3B provided at the lower part of the head unit 3 are subjected to physical actions such as “blow” and “slap” from the user, for example. The pressure is detected, and the detection result is output as a pressure detection signal to the controller 11 via the A / D converter 12.

  The controller 11 is based on an audio signal, an image signal, and a pressure detection signal given from the microphone 21, CCD cameras 22L and 22R, the back sensor 1A, the head sensor 3A, and the chin sensor 3B via the A / D conversion unit 12. Then, the surrounding situation, the command from the user, the presence / absence of the action from the user, and the like are determined, and the action to be taken next by the pet robot is determined based on the determination result. Based on the determination, the controller 11 drives the necessary actuators, thereby swinging the head unit 3 up and down, left and right, moving the tail unit 4, and moving the leg units 2A to 2D. Drive to take action such as walking a pet-type robot.

  Further, the controller 11 generates a synthesized sound as needed, and supplies it to the speaker 23 via the D / A converter 13 for output, or at the “eye” position of the pet robot. A provided LED (Light Emitting Diode) (not shown) is turned on, turned off, or blinked.

  As described above, the pet-type robot takes actions autonomously based on the surrounding situation and the user who comes into contact.

  Next, FIG. 3 shows a functional configuration example of the controller 11 of FIG. The functional configuration shown in FIG. 3 is realized by the CPU 11A executing the OS and application program stored in the memory 11B. In FIG. 3, the A / D converter 12 and the D / A converter 13 are not shown.

  The sensor input processing unit 41 of the controller 11 is based on a pressure detection signal, an audio signal, an image signal, and the like given from the back sensor 1A, the head sensor 3A, the chin sensor 3B, the microphone 21, the CCD cameras 22L and 22R, respectively. A specific external state, a specific action from the user, an instruction from the user, and the like are recognized, and state recognition information representing the recognition result is notified to the model storage unit 42 and the action determination mechanism unit 43.

  That is, the sensor input processing unit 41 includes a pressure processing unit 41A, a voice recognition unit 41B, and an image processing unit 41C.

  The pressure processing unit 41A processes a pressure detection signal given from the back sensor 1A, the head sensor 3A, or the chin sensor 3B. Then, for example, when the pressure processing unit 41A detects a pressure that is equal to or higher than a predetermined threshold value and for a short time as a result of the processing, the pressure processing unit 41A recognizes that the pressure processing unit 41A has been struck and is below the predetermined threshold value. When a long-time pressure is detected, it is recognized as “struck (praised)”, and the recognition result is notified to the model storage unit 42 and the action determination mechanism unit 43 as state recognition information. .

  The voice recognition unit 41B performs voice recognition on the voice signal given from the microphone 21. Then, the voice recognition unit 41B uses, as state recognition information, a command storage unit 42 and an action determination mechanism unit, for example, commands such as “walk”, “turn down”, and “follow the ball” as the voice recognition result. 43 is notified. Note that the distance from the sound source of the user or the like to the microphone 21 is supplied to the voice recognition unit 41B from a distance calculation unit 47 described later, and the voice recognition unit 41B performs voice recognition based on this distance. Is supposed to do.

  The image processing unit 41C performs image recognition processing using image signals given from the CCD cameras 22L and 22R. Then, when the image processing unit 41C detects, for example, “a red round object”, “a plane perpendicular to the ground and higher than a predetermined height” as a result of the processing, An image recognition result such as “There is a wall” is notified to the model storage unit 42 and the action determination mechanism unit 43 as state recognition information.

  The model storage unit 42 stores and manages an emotion model, an instinct model, and a growth model that express the emotion, instinct, and growth state of the robot.

  Here, the emotion model includes, for example, emotion states (degrees) such as “joyfulness”, “sadness”, “anger”, “fun”, etc. within a predetermined range (for example, −1.0 to 1.. 0), and the value is changed based on the state recognition information from the sensor input processing unit 41, the passage of time, and the like.

  The instinct model represents, for example, the state (degree) of desire by instinct such as “appetite”, “sleep desire”, “exercise desire”, etc., by a predetermined range of values. The value is changed based on the passage of time or the like.

  The growth model represents, for example, growth states (degrees) such as “childhood”, “adolescence”, “mature age”, “old age”, and the like by values in a predetermined range. The value is changed based on the state recognition information and the passage of time.

  The model storage unit 42 sends the emotion, instinct, and growth states represented by the values of the emotion model, instinct model, and growth model as described above to the behavior determination mechanism unit 43.

  In addition to the state recognition information supplied from the sensor input processing unit 41, the model storage unit 42 receives the current or past behavior of the pet robot from the behavior determination mechanism unit 43, specifically, for example, “ The behavior information indicating the content of the behavior such as “walked for a long time” is supplied, and the model storage unit 42 is provided with the behavior of the pet-type robot indicated by the behavior information even if the same state recognition information is given. Depending on the situation, different state information is generated.

  For example, when the pet-type robot greets the user and strokes the head, the model storage unit 42 includes behavior information indicating that the user has been greeted and state recognition information indicating that the head has been stroked. In this case, the value of the emotion model representing “joy” is increased in the model storage unit 42.

  The action determination mechanism unit 43 determines the next action based on the state recognition information from the sensor input processing unit 41, the state information from the model storage unit 42, the passage of time, etc., and the content of the determined action is The action command information is output to the posture transition mechanism unit 44.

  That is, the behavior determination mechanism unit 43 manages a finite automaton that associates the behavior that the pet robot can take with the state as a behavior model that defines the behavior of the pet robot. Then, the behavior determination mechanism unit 43 uses the state recognition information from the sensor input processing unit 41, the value of the emotion model, instinct model, or growth model in the model storage unit 42, the time in the finite automaton as the behavior model. Transition is made based on the progress and the like, and the action corresponding to the state after the transition is determined as the action to be taken next.

  Here, the behavior determination mechanism unit 43 transitions the state when it detects that a predetermined trigger (trigger) has occurred. That is, the behavior determination mechanism unit 43 is supplied from the model storage unit 42 when, for example, the time during which the behavior corresponding to the current state is executed reaches a predetermined time or when specific state recognition information is received. The state is changed when the emotion, instinct, and growth state values indicated by the state information are below or above a predetermined threshold.

  In addition, as described above, the behavior determination mechanism unit 43 is based not only on the state recognition information from the sensor input processing unit 41 but also on the emotion model, instinct model, growth model value, and the like in the model storage unit 42. Since the state in the behavior model is transitioned, even if the same state recognition information is input, the state transition destination differs depending on the value (state information) of the emotion model, instinct model, and growth model.

  As a result, for example, when the state information represents “not angry” and “not hungry”, the behavior determination mechanism unit 43 indicates that the state recognition information is “the palm in front of the eyes”. Is generated, action command information for taking the action of “hand” is generated in response to the palm being presented in front of the eyes. To the unit 44.

  In addition, for example, when the state information indicates “not angry” and “hungry”, the behavior determination mechanism unit 43 indicates that the state recognition information indicates that the palm is in front of the eyes. When it indicates that it has been `` submitted, '' action command information is generated to perform an action such as `` flipping the palm '' in response to the palm being presented in front of the eyes. And sent to the posture transition mechanism 44.

  Note that the behavior determination mechanism unit 43 uses, for example, walking as a behavior parameter corresponding to the transition destination state based on the emotion, instinct, and growth state indicated by the state information supplied from the model storage unit 42. , The magnitude and speed of movement when moving the limb, and in this case, action command information including those parameters is sent to the posture transition mechanism unit 44.

  In addition, as described above, the behavior determination mechanism unit 43 generates behavior command information for causing the pet robot to speak in addition to behavior command information for operating the head, limbs, and the like of the pet robot. Is done. Then, the action command information to be uttered by the pet type robot is supplied to the voice synthesis unit 46. When the voice synthesis unit 46 receives the behavior command information, the voice synthesis unit 46 performs voice synthesis according to the behavior command information, and outputs the obtained synthesized sound from the speaker 23.

  The posture transition mechanism unit 44 generates posture transition information for shifting the posture of the pet-type robot from the current posture to the next posture based on the behavior command information supplied from the behavior determination mechanism unit 43. Is sent to the control mechanism unit 45.

  Here, the postures that can be transitioned from the current posture are, for example, the physical shape of the pet-type robot such as the shape and weight of the torso, hands and feet, and the connected state of each part, and the direction and angle at which the joint bends. And the mechanism of the actuator.

  Further, as the next posture, there are a posture that can be directly changed from the current posture and a posture that cannot be directly changed. For example, a four-legged pet-type robot can make a direct transition from a lying position with its limbs thrown down to a lying state, but cannot make a direct transition to a standing state. A two-step movement is required, which is a close-up posture by pulling close to the torso and then standing up. There are also postures that cannot be executed safely. For example, a four-legged pet-type robot will easily fall if it tries to banzai with both front legs raised from its four-legged posture.

  For this reason, the posture transition mechanism unit 44 registers in advance a posture that can be directly transitioned, and when the behavior command information supplied from the behavior determination mechanism unit 43 indicates a posture that can be transitioned directly, the behavior command Information is sent to the control mechanism unit 45.

  On the other hand, when the action command information indicates a posture that cannot be directly transitioned, the posture transition mechanism unit 44 provides posture transition information that makes a transition to a target posture after once transitioning to another posture that can be transitioned. It is generated and sent to the control mechanism unit 45. As a result, it is possible to avoid situations where the robot forcibly executes a posture incapable of transition or a situation where the robot falls over.

  The control mechanism unit 45 generates a control signal for driving the actuator in accordance with the posture transition information from the posture transition mechanism unit 44, and sends this to the actuator of each unit.

  The voice synthesis unit 46 receives the behavior command information from the behavior determination mechanism unit 43, performs, for example, regular voice synthesis according to the behavior command information, and supplies the obtained synthesized sound to the speaker 23 for output.

  The distance calculation unit 47 is supplied with image signals output from the CCD cameras 22L and 22R. The distance calculation unit 47 performs stereo processing (processing by a stereo matching method) using the image signals from the CCD cameras 22L and 22R, thereby generating a sound source such as a user displayed on the images captured by the CCD cameras 22L and 22R. From this, the distance to the microphone 21 is obtained and supplied to the voice recognition unit 41B.

  Here, the stereo processing performed by the distance calculation unit 47 corresponds by associating pixels between a plurality of images obtained by photographing the same object with a camera from two or more directions (different line-of-sight directions). It calculates parallax information between pixels and the distance from the camera to the object.

  That is, the CCD cameras 22L and 22R are now referred to as the reference camera 22L and the detection camera 22R, respectively, and the images output from them are referred to as the reference camera image and the detection camera image, for example, as shown in FIG. When the camera 22L and the detection camera 22R photograph a user as an imaging object, a reference camera image including the user's projection image is obtained from the reference camera 22L, and a detection camera image including the user's projection image from the detection camera 22R. Is obtained. And now, for example, if a certain point P on the mouth of the user is displayed in both the reference camera image and the detected camera image, the position on the reference camera image where the point P is displayed, The disparity information can be obtained from the position on the detected camera image, that is, the corresponding point (corresponding pixel), and further, the position of the point P in the three-dimensional space (three-dimensional position) is obtained using the principle of triangulation. be able to.

  Therefore, in stereo processing, it is first necessary to detect corresponding points. As a detection method, for example, there is an area-based matching method using an epipolar line.

That is, as shown in FIG. 5, in the reference camera 22L, the point P on the user is on the straight line L connecting the point P and the optical center (lens center) O _{1 of the} reference camera 22L. Projected to the intersection n _a with the imaging surface S ₁ .

Further, in the detection camera 22R is P point on the user, on a straight line connecting the optical center (lens center) O ₂ and the point P detected camera 22R, intersection n of the imaging surface S ₂ of the detection camera 22R Projected on _b .

In this case, the straight line L is an intersection line L ₂ between the plane passing through the three points of the optical centers O ₁ and O ₂ and the point n _a (or the point P) and the imaging surface S _{2 on} which the detection camera image is formed. and it is projected onto the imaging surface S _2. Point P is a point on the straight line L, thus, the imaging surface S _2, the n _b point obtained through projection of the point P, and lies on the straight line L ₂ obtained by projecting the straight line L, the straight line L ₂ is epipolar lines Called. That is, there is a possibility that the corresponding point n _b of the point n _a exists on the epipolar line L _2. Therefore, the search for the corresponding point n _b may be performed on the epipolar line L ₂ .

Here, the epipolar line, for example, can be considered for each pixel constituting the reference camera image formed on the imaging surface S _1, the positional relationship of the reference camera 22L and the detection camera 22R is if known, the pixel The epipolar line which exists every time can be calculated | required by calculation, for example.

Detection of corresponding points n _b from a point on the epipolar line L ₂ is, for example, can be carried out by the following area based matching.

That is, in area-based matching, as shown in FIG. 6A, for example, a rectangular small block (hereinafter referred to as a reference block as appropriate) having _a point n _a on the reference camera image as _a center (for example, an intersection of diagonal lines). Is extracted from the reference camera image and, as shown in FIG. 6B, has the same size as the reference block centered on a certain point on the epipolar line L ₂ projected onto the detected camera image. A small block (hereinafter referred to as a detection block as appropriate) is extracted from the detection camera image.

Here, in the embodiment of FIG. 6 (B), six points n _{b1 to} n _b6 are provided on the epipolar line L ₂ as points serving as the center of the detection block. These six points n _{b1 to} n _b6 are points at which the straight line L in the three-dimensional space shown in FIG. 5 is _{divided at} predetermined constant distances, that is, the distance from the reference camera 22L is, for example, 1 m, 2 m, 3 m , 4m, 5 m, respectively point 6m, a point obtained by projecting the imaging surface S ₂ of the detection cameras 22R, therefore, each corresponding distance from the reference camera 22L is 1m, 2m, 3m, 4m, 5m, to the point of 6m is doing.

In area-based matching, detection blocks centered on the points n _{b1 to} n _b6 provided on the epipolar line L ₂ are extracted from the detection camera image, and the correlation between each detection block and the reference block is Calculation is performed using a predetermined evaluation function. Then, the point n _b of the central correlation highest detection block and the reference block centered on the point n _a is obtained as the corresponding point of the point n _a.

That is, for example, when a function that takes a smaller value as the correlation is higher is used as the evaluation function, for example, evaluation values as shown in FIG. 7 for the points n _{b1 to} n _b6 on the epipolar line L ₂ , for example. Assume that (value of evaluation function) is obtained. In this case, the smallest evaluation value (the correlation is the highest) is point n _b3, is detected as a corresponding point of the point n _a. In FIG. 7, interpolation is performed using evaluation values near the minimum value among the evaluation values (indicated by ● in FIG. 7) obtained for each of the points n _{b1 to} n _b6 , and the evaluation value becomes smaller. It is also possible to obtain (represented by x in FIG. 7) and detect that point as the final corresponding point.

In the embodiment of FIG. 6, as described above, the point that divides the straight line L at predetermined equidistant in a three-dimensional space, but a point obtained by projecting the imaging surface S ₂ of the detection camera 22R is set, This setting can be performed at the time of calibration of the reference camera 22L and the detection camera 22R, for example. Then, such a setting is performed for each epipolar line existing for each pixel constituting the imaging surface S ₁ of the reference camera 22L, and as shown in FIG. If a set point / distance table that associates the set point with the distance from the reference camera 22L is created in advance, the set point that becomes the corresponding point is detected and the set point / distance table is referred to. Thus, the distance from the reference camera 22L (the distance to the user) can be obtained immediately. In other words, the distance can be obtained directly from the corresponding points.

On the other hand, for the points n _a on the reference camera image, by detecting the corresponding points n _b on the detection camera image can be obtained parallax (disparity information) between the two points n _a and n _b. Further, if the positional relationship between the reference camera 22L and the detection camera 22R is known, from the disparity between the two points n _a and n _b, the principle of triangulation, can determine the distance to the user. The calculation of the distance from the parallax can be performed by performing a predetermined calculation. However, as shown in FIG. 8B, a parallax / distance table for associating the parallax ζ with the distance is calculated. If created in advance, the distance from the reference camera 22L can be immediately obtained by detecting the corresponding point, obtaining the parallax, and referring to the parallax / distance table.

  Here, the parallax and the distance to the user have a one-to-one correspondence. Therefore, obtaining the parallax and obtaining the distance to the user are equivalent to each other.

Further, the detection of the corresponding point, to use a reference block and a detection block comprised of a plurality of pixels such blocks is to reduce the effects of noise, the characteristics of the pattern of pixels around the pixel (point) n _a of the reference camera image When, by determining clarifies a correlation between the feature of the pattern of peripheral pixels of the corresponding point (pixel) n _b on the detection camera image, it is for the sake of certainty of the corresponding point detection, in particular, changes For a reference camera image and a detection camera image with a small amount, the correlation between the images increases the certainty of detection of corresponding points as the block size increases.

  In area-based matching, the evaluation function for evaluating the correlation between the reference block and the detection block is the difference between the pixel values of the pixels constituting the reference block and the pixels constituting the detection block corresponding to each pixel. The sum of absolute values of the values, the square sum of the differences, normalized cross correlation, and the like can be used.

  The stereo processing has been briefly described above. However, the stereo processing (stereo matching method) is also described in, for example, Yakuin, Nagao, “Introduction to Image Processing in C Language”, Shosodo pp. 127, etc. ing.

  Next, FIG. 9 shows a configuration example of the voice recognition unit 41B of FIG.

  The voice data input to the voice recognition unit 41B via the microphone 21 and the A / D conversion unit 12 in FIG. 2 is supplied to the feature extraction unit 101 and the voice section detection unit 107.

  The feature extraction unit 101 performs acoustic analysis processing on the audio data from the A / D conversion unit 12 for each appropriate frame, and thereby, for example, a feature vector as a feature quantity such as MFCC (Mel Frequency Cepstrum Coefficient) is obtained. Extract. In addition, the feature extraction unit 101 can extract other feature vectors (feature parameters) such as a spectrum, a linear prediction coefficient, a cepstrum coefficient, and a line spectrum pair.

  The feature vectors obtained for each frame in the feature extraction unit 101 are sequentially supplied to and stored in the feature vector buffer 102. Therefore, the feature vector buffer 102 stores feature vectors for each frame in time series.

  Note that the feature vector buffer 102 stores, for example, a time-series feature vector obtained from the start to the end of a certain utterance (voice section).

The matching unit 103 uses the feature vectors stored in the feature vector buffer 102 to generate an acoustic model database 104 _n (n = 1, 2,..., N (N is an integer of 2 or more), a dictionary database 105, The speech (input speech) input to the microphone 21 is recognized based on, for example, the continuous distribution HMM method while referring to the grammar database 106 as necessary.

That is, the acoustic model database 104 _n stores a set of acoustic models representing acoustic features for each predetermined unit (PLU (Phonetic-Linguistic-Units)) such as individual phonemes and syllables in the speech language for speech recognition. is doing. Here, since speech recognition is performed based on the continuous distribution HMM method, an HMM (Hidden Markov Model) using a probability density function such as a Gaussian distribution is used as the acoustic model. The dictionary database 105 stores a word dictionary in which information about pronunciation (phoneme information) is described for each word (vocabulary) to be recognized. The grammar database 106 stores grammatical rules (language model) describing how each word registered in the word dictionary of the dictionary database 105 is linked (connected). Here, as the grammar rule, for example, a rule based on context free grammar (CFG), regular grammar (RG), statistical word chain probability (N-gram), or the like can be used.

The matching unit 103 refers to the word dictionary in the dictionary database 105 to connect an acoustic model stored in the acoustic model database 104 _n to configure a word acoustic model (word model). Further, the matching unit 103 connects several word models by referring to the grammar rules stored in the grammar database 106, and uses the word models connected in this way, Are matched by the continuous distribution HMM method, and the voice input to the microphone 21 is recognized. That is, the matching unit 103 calculates a score representing the likelihood that a time-series feature vector stored in the feature vector buffer 102 is observed from each word model sequence configured as described above. Then, for example, the matching unit 103 detects a series of word models having the highest score, and outputs a word string corresponding to the series of word models as a speech recognition result.

  Here, since speech recognition is performed by the HMM method, the matching unit 103 acoustically accumulates the appearance probability of each feature vector for the word string corresponding to the connected word model, and the accumulated value. Is a score.

  That is, the score calculation in the matching unit 103 is given by an acoustic score given by an acoustic model stored in the acoustic model database 104 (hereinafter referred to as an acoustic score as appropriate) and a grammar rule stored in the grammar database 106. This is performed by comprehensively evaluating a linguistic score (hereinafter referred to as a language score as appropriate).

  Specifically, for example, in the case of the HMM method, the acoustic score is based on a probability (probability of appearance) that a series of feature vectors output from the feature extraction unit 101 is observed from an acoustic model constituting a word model. Calculated for each word. Further, for example, in the case of bigram, the language score is obtained based on the probability that the word of interest and the word immediately preceding the word are linked (connected). Then, a speech recognition result is determined based on a final score (hereinafter, referred to as a final score as appropriate) obtained by comprehensively evaluating the acoustic score and the language score for each word.

  Here, the speech recognition unit 41B can be configured without providing the grammar database 106. However, according to the rules stored in the grammar database 106, the word models to be connected are limited. As a result, the number of words for which the acoustic score is calculated in the matching unit 103 is limited. The amount can be reduced and the processing speed can be improved.

In the embodiment shown in FIG. 9, N acoustic model databases 104 ₁ , 104 ₂ ,..., 104 _N are provided, and these N acoustic model databases 104 _{1 to} 104 _N are included in the _N acoustic model databases 104 _{1 to} 104 _N. A set of acoustic models for each of a plurality of different distances generated using sounds emitted from a plurality of sound sources separated from the microphone by a plurality of different distances is stored.

That is, for example, the distance from the microphone to the speaker of the learning voice that is the sound source is represented by D ₁ , D ₂ ,..., D _N (where D ₁ <D ₂ <... <D _N )), the voice of the speaker who spoke from a distance D ₁ , D ₂ ,..., D _N from the microphone is recorded with the microphone, and for each recorded distance. obtained by performing learning using the speech data, the distance D _1, D _2, · · ·, a set of acoustic models of each D _N (here HMM), an acoustic model database 104 _1, 104 _2, .., 104 _N are stored respectively.

Therefore, the acoustic model database 104 _n stores a set of acoustic models generated from the voice data of a speaker who has spoken from a position away from the microphone by a distance D _n .

As the minimum value D ₁ of the distances D _{1 to} D _N , for example, 0 (actually the state where the user's mouth and the microphone are close to each other) can be adopted, and the maximum value D _N is here. Then, for example, the statistical value of the maximum value of the distance that the user is expected to talk to the robot (for example, a questionnaire to determine how far away the maximum number of users should talk to the robot) Can be used as an average of distances to be answered. Furthermore, as the other distances D ₂ , D ₃ ,..., D _N−1 , for example, a distance that equally divides the distance D _N can be adopted.

In addition, the acoustic model set of the distance D _n to be stored in the acoustic model database 104 _n can be generated from the voice data of the actually performed speech at a position away from the microphone by the distance D _n. For audio data that records utterances made in close proximity (with the distance to the microphone set to 0) (for example, audio data recorded using a headset microphone), the microphone and the distance D _n away from it. It is also possible to generate from voice data obtained by convolving the impulse response between the two positions (space). In addition, it is described in the above-mentioned document 2, for example, to obtain voice data in which a voice uttered at a position separated by a predetermined distance using an impulse response is recorded with a microphone.

  The voice segment detection unit 107 detects a voice segment based on the output of the A / D conversion unit 12 and supplies a message representing the detection result to the selection control unit 108. Here, as a method for detecting a speech section, for example, a method of calculating the output power of the A / D conversion unit 12 for each predetermined frame and determining whether the power is equal to or higher than a predetermined threshold. is there.

  When the selection control unit 108 receives a message indicating that it is a voice segment from the voice segment detection unit 107, the selection control unit 108 requests the distance calculation unit 47 (FIG. 3) to calculate the distance from the microphone 21 to the user who is speaking. In response to the request, the distance supplied from the distance calculation unit 47 is received. Further, the selection control unit 108 controls the selector 109 based on the distance received from the distance calculation unit 47.

  Here, in the distance calculation unit 47 in the embodiment of FIG. 3, the distance from the CCD camera 22L or 22R to the user who is speaking is calculated by the above-described stereo processing, but the CCD camera 22L or 22R Therefore, the distance from the CCD camera 22L or 22R to the user who is speaking is regarded as the distance from the microphone 21 to the user who is speaking. To do. However, when the positional relationship between the CCD cameras 22L and 22R and the microphone 21 is known, the distance from the microphone 21 to the user can be obtained based on the distance from the CCD camera 22L or 22R to the user. is there.

The selector 109 selects the acoustic model database 104 _n that is one of the _N acoustic model databases 104 _{1 to} 104 _N under the control of the selection control unit 108. Further, the selector 109 acquires a set of acoustic models of the distance D _n stored in the selected acoustic model database 104 _n and provides it to the matching unit 103. Accordingly, the matching unit 103 calculates an acoustic score using the acoustic model of the distance D _n acquired by the selector 109.

  Next, the speech recognition processing by the speech recognition unit 41B in FIG. 9 will be described with reference to the flowchart in FIG.

  First, in step S1, the speech segment detection unit 107 determines whether or not there is a speech input from the user. That is, the voice section detection unit 107 determines whether or not the voice section, and if the voice section is determined to be a voice section, the voice section detection unit 107 determines that there is a voice input from the user. It is determined that there was no voice input from.

  If it is determined in step S1 that there is no voice input, steps S2 to S5 are skipped and the process proceeds to step S6.

  If it is determined in step S1 that there is a voice input, that is, the voice segment detection unit 107 detects a voice segment, a message to that effect is supplied to the selection control unit 108, and the feature extraction unit 101 When the extraction of the feature vector of the speech data of the speech section is started and the storage of the feature vector is started in the feature vector buffer 102, the process proceeds to step S2, and the selection control unit 108 includes the distance calculation unit 47. (Fig. 3) is requested to calculate the distance to the user who is speaking. Thereby, the distance calculation part 47 calculates the distance to the user who is speaking in step S2, and supplies the distance to the selection control part 108.

  Here, since it is generally expected that the user often talks from the front direction of the robot, the CCD cameras 22L and 22R that image the user to calculate the distance to the user have the imaging direction of the robot It is assumed that the head unit 3 (FIG. 2) is installed so as to be in the front direction.

  In this case, when the user talks from the front direction of the robot, for example, from the side or back direction, the CCD cameras 22L and 22R cannot capture the user. Therefore, for example, a microphone having directivity in the same direction as the imaging direction of the CCD cameras 22L and 22R is adopted as the microphone 21, and the head unit 3 is moved in a direction in which the sound level input to the microphone 21 is maximized. As a result, the CCD cameras 22L and 22R can image the user.

  In addition, the robot is provided with a plurality of microphones, the direction of the sound source is estimated from the power difference and phase difference of the audio signals reaching the plurality of microphones, and the maximum sound level of the plurality of microphones in the direction is estimated. By moving the head unit 3 in the direction in which the image is obtained, the CCD cameras 22L and 22R can capture the user. When the robot is provided with a plurality of microphones, for example, a microphone that can obtain the maximum sound level (if the robot faces the direction of the user, the microphone that is basically provided in the front direction is used. ) Is output as a speech recognition target.

  Here, in the distance calculation unit 47 of FIG. 3, in order to calculate the distance to the user by performing stereo processing using the images obtained from the CCD cameras 22L and 22R, the images output from the CCD cameras 22L and 22R Therefore, it is necessary for the user to detect a pixel displayed (hereinafter referred to as a user pixel as appropriate). For example, a pixel displaying a predetermined color such as a so-called skin color is detected as a user pixel. Is possible. Alternatively, for example, it is also possible to detect user pixels by imaging a user's face in advance using a CCD camera 22L or 22R and performing image recognition using the face image as a standard pattern.

When the selection control unit 108 receives the distance to the user from the distance calculation unit 47 (FIG. 3), the selection control unit 108 proceeds to step S3, and selects the most distance from the _N distances D _{1 to} D _N described above to the user. The selector 109 is controlled to detect the close distance D _n and select the acoustic model database 104 _n storing the set of acoustic models of the distance D _n . Thereby, in step S3, the selector 109 selects the acoustic model database 104 _n according to the control of the selection control unit 108, acquires a set of acoustic models having the distance D _n closest to the distance to the user, and performs matching. Supplied to the unit 103 and proceeds to step S4.

In step S4, the matching unit 103, stored in the feature vector buffer 102, using the feature vectors extracted from the speech data of the speech interval, the set of acoustic models of the distance D _n supplied from the selector 109, the dictionary database 105 The language score and the acoustic score for the word string (word) as a speech recognition result candidate are calculated by referring to the word dictionary stored in FIG. The word string (word) having the highest final score is determined as the speech recognition result.

  In step S5, the matching unit 103 outputs the voice recognition result determined in step S4, and then proceeds to step S6.

  In step S6, it is determined whether or not the voice recognition process is to be terminated. If it is determined not to be terminated, the process returns to step S1 and the same process is repeated thereafter.

  In step S6, when it is determined that the voice recognition process is to be ended, that is, for example, when the power of the robot is turned off by the user, the voice recognition process is ended.

As described above, to calculate the distance to the user who made the utterance, the closest distance D _n apart sets of acoustic models generated from speech data was recorded speech uttered by the position on the distance (distance D _Since the speech recognition is performed using a set of _n acoustic models, the recognition accuracy of the user's speech uttered at a position away from the microphone can be improved.

  That is, voice recognition is performed using a set of acoustic models learned using voice data recorded in an acoustic environment close to the acoustic environment where the user uttered, thereby improving voice recognition accuracy. Can do.

  Further, in this case, since speech recognition is performed by selecting one set of N acoustic models, the amount of calculation required for speech recognition processing is not (almost) increased.

  Note that the acoustic environment changes depending on the noise level, reverberation characteristics, and other factors in addition to the distance from the microphone 21 to the user. Therefore, the use of a set of acoustic models that takes them into consideration further improves speech recognition accuracy. It becomes possible.

Further, in the embodiment of FIG. 9, from the acoustic model database 104 ₁ through 104 _N which stores the set of acoustic models for each distance D ₁ to D _N, to the acoustic model set (user corresponding to the distance to the user The set of acoustic models closest to the distance) is selected, but the acoustic model set corresponding to the distance to the user can be acquired via a network, for example. is there.

  Next, FIG. 11 is a block diagram showing another internal configuration example of the pet-type robot of FIG. In the figure, portions corresponding to those in FIG. 2 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the pet type robot shown in FIG. 11 is basically configured in the same manner as in FIG. 2 except that the ultrasonic sensor 111 is newly provided in the head unit 3.

  The ultrasonic sensor 111 has a sound source and a microphone (not shown), and emits ultrasonic pulses from the sound source as shown in FIG. Further, the ultrasonic sensor 111 receives the reflected wave that is reflected by the obstacle and is returned by the microphone and emits the ultrasonic pulse, and then receives the reflected wave (hereinafter, referred to as the reflected wave). (Referred to as lag time) and supplies the controller 11 with it.

  Next, FIG. 13 shows a functional configuration example of the controller 11 of FIG. In the figure, portions corresponding to those in FIG. 3 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate. That is, the controller 11 of FIG. 13 is supplied with the output of the ultrasonic sensor 111 instead of the outputs of the CCD cameras 22L and 22R to the distance calculation unit 47, except in the case of FIG. It is configured in the same way.

  In the embodiment of FIG. 13, the distance calculation unit 47 calculates the distance to the user based on the output of the ultrasonic sensor 111.

  That is, in the embodiment of FIG. 13, for example, the utterance is made by the method using the directional microphone 21 as described above, the method using a plurality of microphones, the method using image recognition, or the like. The direction of the user who is performing is recognized, and the head unit 3 is moved so that the sound source of the ultrasonic sensor 111 faces the direction of the user. Then, the ultrasonic sensor 111 emits an ultrasonic pulse toward the user, receives the reflected wave, obtains a lag time, and supplies the lag time to the distance calculation unit 47. The distance calculation unit 47 calculates the distance to the user based on the lag time supplied from the ultrasonic sensor 111 and supplies the calculated distance to the voice recognition unit 41B. Thereafter, the voice recognition unit 41B performs the same processing as that described with reference to FIGS.

  Next, FIG. 14 shows another configuration example of the voice recognition unit 41B of FIG. 3 or FIG. In the figure, portions corresponding to those in FIG. 9 are denoted by the same reference numerals, and description thereof will be omitted below as appropriate.

In the embodiment of FIG. 9, a set of acoustic models closest to the distance to the user is selected from a set of acoustic models with distances D _{1 to DN} , and speech recognition is performed using the set of acoustic models. In the embodiment of FIG. 14, the audio data output from the microphone 21 is filtered using an inverse filter having a frequency characteristic corresponding to the distance to the user, and the filtered audio is output. For data, speech recognition is performed using a set of predetermined acoustic models.

  In other words, in the embodiment of FIG. 14, the acoustic model database 104 includes, for example, a set of acoustic models generated from audio data that records utterances performed close to the microphone (the distance from the microphone is 0). I remember it.

  The filter unit 121 is a digital filter that operates using the tap coefficient supplied from the tap coefficient selection unit 122 (hereinafter referred to as a selected tap coefficient as appropriate) as a tap coefficient, and filters the audio data output from the A / D conversion unit 12. , And supplied to the feature extraction unit 101.

  When the tap coefficient selection unit 122 receives a message indicating that the voice section has been detected from the voice section detection unit 107, the tap coefficient selection unit 122 requests the distance calculation unit 47 of FIG. 3 or FIG. 13 to calculate the distance to the user. In response, the distance to the user supplied from the distance calculation unit 47 is received. Furthermore, the tap coefficient selection unit 122 reads out from the tap coefficient storage unit 123 a tap coefficient set that realizes an inverse filter having a frequency characteristic corresponding to the distance closest to the user. Then, the tap coefficient selection unit 122 supplies the tap coefficient read from the tap coefficient storage unit 123 to the filter unit 121 as the selected tap coefficient and sets it as the tap coefficient.

The tap coefficient storage unit 123 stores, for example, a set of tap coefficients that realizes an inverse filter as a digital filter having characteristics opposite to the frequency characteristics corresponding to the _N distances D ₁ to DN described above. .

  In the speech recognition unit 41B configured as described above, when the speech segment detection unit 107 detects that the speech segment is a speech segment, the tap coefficient selection unit 122 causes the distance calculation unit 47 (FIG. 3 or FIG. 13) to Then, it requests the calculation of the distance to the user, and receives the distance to the user supplied from the distance calculation unit 47 in response to the request. Furthermore, the tap coefficient selection unit 122 reads out a tap coefficient set that realizes an inverse filter having a frequency characteristic corresponding to the distance closest to the distance to the user from the tap coefficient storage unit 123, and uses the tap coefficient set as the selected tap coefficient. To supply.

  The filter unit 121 filters the audio data output from the A / D conversion unit 12 using the selected tap coefficient as the tap coefficient, thereby corresponding to the distance from the frequency component of the audio data output from the microphone 21 to the user. The voice data from which the frequency characteristics are removed, that is, the voice data in which an utterance made close to the microphone 21 is equivalently obtained, is obtained and supplied to the feature extraction unit 101.

Here, for example, now, the same was done at the position (the t time (representing time)) of audio data x (t) was recorded utterances made in proximity to the microphone and, at a distance D _n The voice data recording the utterance is represented by y (t), the impulse response of the space from the microphone 21 to the position separated by the distance D _{n is} represented by h _n (t), and each frequency is represented by ω and x ( If each Fourier transform of t), y (t), h _n (t) is expressed as X (ω), Y (ω), H _n (ω), the following equation is established.

Y (ω) = X (ω) H _n (ω) (1)

  The following equation holds from equation (1).

X (ω) = Y (ω) / H _n (ω) (2)

H _n (omega), since it represents the frequency characteristic corresponding to the distance D _n, the equation (2), the distance D corresponding frequency characteristic _n H _n (omega) opposite characteristics 1 / H _n of the (omega) If the speech data Y (ω) containing utterances performed at a position away from the microphone 21 by the distance D _n is filtered by an inverse filter, which is a filter having the same, it is equivalently performed close to the microphone 21 Audio data X (ω) containing utterances can be obtained.

In the embodiment of FIG. 14, the tap coefficient storage unit 123 has characteristics 1 / H ₁ (ω) through inverse of frequency characteristics H ₁ (ω) through H _N (ω) corresponding to the distances D ₁ through D _N. The tap coefficient for realizing the inverse filter having 1 / H _N (ω) is stored, and the tap coefficient for realizing the inverse filter of the frequency characteristic corresponding to the distance closest to the distance to the user in the tap coefficient selection unit 122 is stored. Are read from the tap coefficient storage unit 123 and supplied to the filter unit 121 as selected tap coefficients.

  In the filter unit 121, the selected tap coefficient is used as the tap coefficient (digital filter tap coefficient), and the audio data output from the A / D conversion unit 12 is filtered, so that the selected tap coefficient is close to the microphone 21. The voice data recording the uttered speech is obtained equivalently and supplied to the feature extraction unit 101.

  Accordingly, the matching unit 103 equivalently sets a set of acoustic models generated from the sound data recorded of the utterances performed close to the microphone for the sound data recorded of the utterances performed close to the microphone 21. Since speech recognition is performed using the same, the speech recognition accuracy can be improved without increasing the calculation amount of the matching unit 103 as in the embodiment of FIG.

Here, the distance D corresponding frequency characteristic _n H _n (ω) opposite characteristics 1 / H _n of (omega) from the position at a distance D _n from the microphone, ideally, impulse [delta] (t) And observing the voice data s (t) output from the microphone recording the impulse δ (t), and in practice, using the TSP (Time Stretched Pulse) signal, the equation (1) ) Or (2).

  In addition, it is desirable to use the microphone 21 and the microphone that records the audio data for learning that have the same frequency characteristics.

  As described above, the case where the present invention is applied to a substantial robot having a voice recognition function has been described. However, the present invention is applied to, for example, a virtual robot displayed on a computer or any other device. Is possible.

  The series of voice recognition processes described above can be performed by, for example, a general-purpose computer. In this case, the voice recognition apparatus is installed by installing a program for performing the series of voice recognition processes on the general-purpose computer. Realized.

  Further, in this case, the program can be recorded in advance on a hard disk or ROM (Read Only Memory) as a recording medium built in the computer.

  Alternatively, the program is temporarily or permanently stored on a removable recording medium such as a flexible disk, CD-ROM (Compact Disc Read Only Memory), MO (Magneto optical) disk, DVD (Digital Versatile Disc), magnetic disk, and semiconductor memory. Can be stored (recorded). Such a removable recording medium can be provided as so-called package software.

  In addition to installing the program from a removable recording medium as described above to the computer, the program can be wirelessly transferred from a download site to a computer via an artificial satellite for digital satellite broadcasting, a LAN (Local Area Network), the Internet Such a program can be transferred to a computer via a network, and the computer can receive the program transferred in this way and install it on a built-in hard disk.

  In the present specification, the processing steps for describing a program for causing a computer (CPU) to perform various processes do not necessarily have to be processed in time series according to the order described in the flowchart. It includes processing executed individually (for example, parallel processing or object processing).

  Further, the program may be processed by a single computer, or may be processed in a distributed manner by a plurality of computers. Furthermore, the program may be transferred to a remote computer and executed.

  Furthermore, the present invention is also applicable to speech recognition using algorithms other than the HMM method.

  In the present embodiment, the distance to the user is obtained by stereo processing or an ultrasonic sensor, but the distance to the user can be obtained by any other method. That is, for example, the distance to the user can be obtained, for example, by having the user utter the distance and recognizing the utterance. Furthermore, it is possible to obtain a distance to the user in the robot by providing the remote commander with a button for inputting a distance and having the user operate the button.

It is a perspective view which shows the example of an external appearance structure of the pet type robot to which this invention is applied. It is a block diagram which shows the hardware structural example of a pet-type robot. 3 is a block diagram illustrating a functional configuration example of a controller 11. FIG. It is a figure which shows the state which is image | photographing a user with the reference camera 22L and the detection camera 22R. It is a figure for demonstrating an epipolar line. It is a figure which shows a reference | standard camera image and a detection camera image. It is a figure which shows transition of an evaluation value. It is a figure which shows a set point / distance table and a parallax / distance table. It is a block diagram which shows the structural example of the speech recognition part 41B. It is a flowchart explaining the process of the speech recognition part 41B. It is a block diagram which shows the other hardware structural example of a pet-type robot. It is a figure explaining the process of the ultrasonic sensor. 4 is a block diagram illustrating another functional configuration example of the controller 11. FIG. It is a block diagram which shows the other structural example of the speech recognition part.

Explanation of symbols

1 body unit, 1A back sensor, 2A to 2D leg unit, 3 head unit, 3A head sensor, 3B chin sensor, 4 tail unit, 11 controller, 11A CPU, 11B memory, 12 A / D conversion unit, 13 D / A conversion unit, 14 communication unit, 15 semiconductor memory, 21 microphone, 22L, 22R CCD camera, 23 speaker, 41 sensor input processing unit, 41A pressure processing unit, 41B speech recognition unit, 41C image processing unit, 42 model Storage unit, 43 action determination mechanism unit, 44 posture transition mechanism unit, 45 control mechanism unit, 46 speech synthesis unit, 47 distance calculation unit, 101 feature extraction unit, 102 feature vector buffer, 103 matching unit, 104, 104 _{1 to} 104 _N acoustic model database, 105 dictionary database, 106 grammar de Database, 107 voice section detection unit, 108 a selection control unit, 109 a selector, 111 ultrasonic sensor, 121 filter unit, 122 the tap coefficient selection unit, 123 the tap coefficient storage unit

Claims (10)

A speech recognition device that recognizes input speech,
A distance calculating means for calculating a distance to the sound source;
Storage means for storing a set of acoustic models taking into account the acoustic environment for each of the plurality of different distances, generated using sounds emitted from a plurality of sound sources separated by a plurality of different distances;
Acquired by selecting a set of acoustic models corresponding to the distance obtained by the distance calculating means from among a set of acoustic models considering the acoustic environment for each of the plurality of different distances stored in the storage means. Acquisition means to
A voice recognition device comprising: voice recognition means for recognizing the voice using the set of acoustic models acquired by the acquisition means. The speech recognition apparatus according to claim 1, wherein the distance calculation unit obtains a distance to the sound source by performing stereo processing using images output from a plurality of imaging units that capture images. The speech recognition apparatus according to claim 1, wherein the distance calculation unit obtains a distance to the sound source using an output of an ultrasonic sensor. A speech recognition method for recognizing input speech,
A distance calculating step for obtaining a distance to the sound source of the sound;
A set of acoustic models corresponding to the distance obtained in the distance calculating step is generated using sounds emitted from sound sources separated by a plurality of different distances. An acquisition step of acquiring by selecting from among a set of acoustic models considering an acoustic environment for each of the plurality of different distances stored in a storage means storing a set of considered acoustic models;
A voice recognition method comprising: a voice recognition step of recognizing the voice using the set of acoustic models acquired in the acquisition step. A program for causing a computer to perform speech recognition processing for recognizing input speech,
A distance calculating step for obtaining a distance to the sound source of the sound;
A set of acoustic models corresponding to the distance obtained in the distance calculating step is generated using sounds emitted from sound sources separated by a plurality of different distances. An acquisition step of acquiring by selecting from among a set of acoustic models considering an acoustic environment for each of the plurality of different distances stored in a storage means storing a set of considered acoustic models;
A program that causes a computer to perform a process including a voice recognition step of recognizing the voice using the set of acoustic models acquired in the acquisition step. A recording medium on which a program for causing a computer to perform speech recognition processing for recognizing input speech is recorded,
A distance calculating step for obtaining a distance to the sound source of the sound;
A set of acoustic models corresponding to the distance obtained in the distance calculating step is generated using sounds emitted from sound sources separated by a plurality of different distances. An acquisition step of acquiring by selecting from among a set of acoustic models considering an acoustic environment for each of the plurality of different distances stored in a storage means storing a set of considered acoustic models;
A recording medium in which a program for causing a computer to perform a process including a voice recognition step of recognizing the voice using the set of acoustic models acquired in the acquisition step is recorded. A speech recognition device that recognizes input speech,
A distance calculating means for calculating a distance to the sound source;
First acquisition means for acquiring a tap coefficient for realizing an inverse filter having a frequency characteristic corresponding to the distance obtained by the distance calculation means;
Filter means for filtering the voice using the tap coefficient acquired in the acquisition means;
Storage means for storing a set of acoustic models taking into account the acoustic environment for each of the plurality of different distances, generated using sounds emitted from a plurality of sound sources separated by a plurality of different distances;
Acquired by selecting a set of acoustic models corresponding to the distance obtained by the distance calculating means from among a set of acoustic models considering the acoustic environment for each of the plurality of different distances stored in the storage means. A second acquisition means for:
A voice recognition device comprising: voice recognition means for recognizing the voice filtered by the filter means using the set of acoustic models acquired by the acquisition means. A speech recognition method for recognizing input speech,
A distance calculating step for obtaining a distance to the sound source of the sound;
A first acquisition step of acquiring a tap coefficient for realizing an inverse filter of a frequency characteristic corresponding to the distance obtained in the distance calculation step;
A filter step of filtering the voice using the tap coefficient acquired in the first acquisition step;
A set of acoustic models corresponding to the distance obtained in the distance calculating step is generated using sounds emitted from sound sources separated by a plurality of different distances. A second acquisition step of acquiring by selecting from among a set of acoustic models considering an acoustic environment for each of the plurality of different distances stored in a storage means storing a set of considered acoustic models;
A speech recognition method comprising: recognizing the speech filtered in the filtering step using the set of acoustic models acquired in the second acquisition step. A program for causing a computer to perform speech recognition processing for recognizing input speech,
A distance calculating step for obtaining a distance to the sound source of the sound;
A first acquisition step of acquiring a tap coefficient for realizing an inverse filter of a frequency characteristic corresponding to the distance obtained in the distance calculation step;
A filter step of filtering the voice using the tap coefficient acquired in the first acquisition step;
A set of acoustic models corresponding to the distance obtained in the distance calculating step is generated using sounds emitted from sound sources separated by a plurality of different distances. A second acquisition step of acquiring by selecting from among a set of acoustic models considering an acoustic environment for each of the plurality of different distances stored in a storage means storing a set of considered acoustic models;
A program for causing a computer to perform a process including a voice recognition step of recognizing the voice filtered in the filtering step using the acoustic model set acquired in the second acquisition step. A recording medium on which a program for causing a computer to perform speech recognition processing for recognizing input speech is recorded,
A distance calculating step for obtaining a distance to the sound source of the sound;
A first acquisition step of acquiring a tap coefficient for realizing an inverse filter of a frequency characteristic corresponding to the distance obtained in the distance calculation step;
A filter step of filtering the voice using the tap coefficient acquired in the first acquisition step;
A set of acoustic models corresponding to the distance obtained in the distance calculating step is generated using sounds emitted from sound sources separated by a plurality of different distances. A second acquisition step of acquiring by selecting from among a set of acoustic models considering an acoustic environment for each of the plurality of different distances stored in a storage means storing a set of considered acoustic models;
A program that causes a computer to perform a process including a voice recognition step of recognizing the voice filtered in the filtering step using the acoustic model set acquired in the second acquisition step is recorded. A recording medium characterized by the above.

Images