JP5595112B2 - robot - Google Patents

Description

  The present invention relates to a robot, and more particularly, to a robot in which a plurality of microphones are installed in a head and a sound source direction is estimated by acquiring sound from all directions.

It is disclosed that a conventional robot includes a rotatable head and a plurality of microphones on the head so that sound can be acquired from all directions of 360 degrees, and estimates the direction in which the speaker is present.
However, in the case where the conventional robot itself has a device that becomes a noise generation source that outputs noise (noise sound), or when there is a noise generation source in the vicinity of the place where the robot is installed, In addition to the voice of the speaker, the noise sound output from the noise source is acquired. Therefore, when the noise level of the noise sound is high, the robot estimates the direction in which the noise is generated as the direction in which the speaker is present, and cannot estimate the direction in which the original speaker is present (the sound source direction to be estimated). There was a problem.
In particular, a robot has a cooling fan for cooling a power device or a computer for driving itself, and the noise level of the sound of rotation of the cooling fan is high, so that the sound source direction to be estimated cannot be estimated. was there.

  For this reason, the mobile robot of Patent Document 1 reduces the number of rotations of the cooling fan based on the voice recognition result so that the noise level of the noise sound output from the noise generation source (cooling fan) included in the mobile robot is lowered. In addition, a technique for controlling a device itself as a noise generation source and a technique for requesting a speaker to speak again are disclosed.

JP 2006-95635 A

  However, a conventional robot having a voice input unit on a rotatable head is a noise source that outputs a noise sound in order to estimate the direction in which the speaker is present (the sound source direction to be estimated). There was a problem that had to be controlled.

  The present invention has been made to solve the above-described problems, and noise sound is output from a sound input from 360 degrees in all directions to a sound input unit provided in a rotatable head. It is an object of the present invention to provide a robot that estimates a direction in which a speaker is present (a sound source direction to be estimated) by excluding a direction in which the speaker is present.

  In order to solve the above-mentioned problem, a robot according to a first aspect of the present invention includes a torso, a head supported by a support shaft so as to be rotatable on an upper surface of the torso, and the head is turned. Three or more voice input units arranged on the head and spaced apart at a predetermined angle around the support shaft in the rotating direction to convert the input voice to voice data, and the voice input Audio data processing for generating omnidirectional sound pressure component data indicating sound pressure values from 360 degrees in all directions with reference to the direction of the head, using the signal decomposition algorithm for the sound data input from each of the units A rotation angle measuring unit that measures the rotation angle of the head with respect to the direction of the body, and a range of directions that are not estimated as a sound source based on the direction of the body A storage unit storing the excluded angle range and a sound source method for estimating the sound source direction In the robot including the estimation unit, the sound source direction estimation unit removes data within the excluded angle range from the omnidirectional sound pressure component data by using the measured rotation angle, and an effective direction sound pressure component A filter unit for generating data is provided.

According to such a configuration, the robot can generate sound from the direction indicated by the exclusion angle range based on the direction of the torso (for example, on the back of the robot) regardless of the rotation angle of the head on which the voice input unit is installed. Noise noise output by the rotation of the fan can be removed.
Also, the robot can acquire effective direction sound pressure component data input from outside the excluded angle range.

The robot according to claim 2 is the robot according to claim 1, wherein the storage unit stores a voice input angle indicating a direction of each of the voice input units around the support shaft, and the sound source. The direction estimation unit extracts at least one or more angles of sound pressure values larger than both the sound pressure value at the immediately preceding angle and the sound pressure value at the immediately following angle from the effective direction sound pressure component data generated by the filter unit. An estimation unit that separates and extracts audio data input from directions of estimated sound source angles extracted by the angle extraction unit from audio data output by the audio input unit based on the audio input angle. Speech recognition is performed on the estimated sound source direction speech data extracted by the sound source separation unit and the estimated sound source separation unit, a plurality of hypothesis phrases are generated, and a speech recognition likelihood indicating the correctness of each hypothesis phrase is calculated. The speech recognition unit related to the hypothesis phrase with a speech input direction suitability indicating the correctness of the relationship between the speech recognition unit that performs the hypothesis phrase and the direction in which the estimated sound source direction speech data related to the hypothesis phrase is input A phrase certainty factor calculating unit that weights likelihoods, calculates a phrase certainty factor of each hypothesis phrase, and estimates a sound source direction based on the calculated phrase certainty factor, is provided.

According to this configuration, the robot determines the correctness of the relationship between the speech recognition result performed by the speech recognition unit and the direction in which the estimated sound source direction speech data separated and extracted by the estimated sound source separation unit is input. Can be shown.
In addition, the robot weights the speech recognition likelihood indicating the correctness of the speech recognition result in the speech recognition unit by the speech input direction suitability, so that the correctness of the speech recognition result and the direction in which the speech is input It is possible to calculate a value that evaluates both the correctness and the correctness.

The robot according to claim 3, in the robot according to claim 2, wherein the phrase confidence factor computing unit, from the hypothesis phrases each phrase confidence, and extracts the maximum phrases confidence, the phrase belief The estimated sound source angle related to the estimated sound source direction sound data that generated the hypothesis phrase related to the degree is estimated in the sound source direction .

According to such a configuration, the robot can set the estimated sound source angle related to the speech recognition result that maximizes the phrase certainty as the sound source direction.

According to a fourth aspect of the present invention, there is provided a robot according to the third aspect, wherein the robot according to the third aspect includes a phrase storage unit and a phrase determination unit, and the phrase storage unit collects a plurality of phrase patterns in which phrases that can be input in advance are collected. And a direction angle value pattern indicating a direction centered on the support shaft and based on the direction of the head, and the voice input direction adaptation indicating the correctness of the phrase pattern being input from the direction of the direction angle value pattern Are stored in association with each other, and the phrase certainty calculation unit acquires a plurality of hypothesis phrases generated by the speech recognition unit and the speech recognition likelihood thereof, and the hypothesis phrase and the using the estimated sound source angle angle extracting unit has extracted, associated with the phrase pattern and the direction angle value patterns stored in the phrase storage unit Serial extract audio input direction fit, by weighting the speech recognition likelihoods that the speech input direction fitness, calculates the phrase confidence, the phrase determiner, the phrase confidence factor computing unit has extracted the the hypothesis phrase according to the maximum phrases confidence, characterized by the phrases and to Turkey recognized speech sound from the sound source direction in which the phrase confidence factor computing unit is estimated according to the hypothesis phrase.

According to such a configuration, the robot can indicate the correctness of the relationship between the estimated speech angle acquired by the angle extraction unit and the hypothesis phrase generated by speech recognition by the speech recognition unit as the speech input direction suitability.
In addition, the robot weights the speech recognition likelihood indicating the correctness of the hypothetical phrase generated by the speech recognition unit by the correctness of the direction in which the hypothetical phrase is input, and the speech input direction suitability, so that It is possible to calculate a phrase certainty factor that serves as a determination criterion indicating the correctness of the recognition result.
Then, the robot can set the hypothesis phrase having the maximum phrase certainty as a correct phrase.
Furthermore, even if the value of the speech recognition likelihood is large, the robot can remove a hypothesis phrase having a low speech input direction suitability.

  Further, the robot according to claim 5 is the robot according to claim 4, wherein the phrase storage unit is a two-dimensional model composed of the direction angle value pattern and the voice input direction matching degree for each phrase pattern. A histogram is stored.

  According to this configuration, the robot can extract the phrase certainty factor from a two-dimensional histogram composed of the direction angle value pattern and the speech recognition likelihood for each phrase pattern.

  A robot according to a sixth aspect of the present invention is the robot according to any one of the second to fifth aspects, wherein the robot rotates to the head until the estimated sound source angle extracted by the angle extraction unit is reached. The behavior control part to be provided is provided.

  With this configuration, the robot can turn the head in the called direction.

  According to the first aspect of the present invention, the range of the direction not to be estimated as the sound source (excluded angle range) is stored in the storage unit in advance, whereby the rotating head is provided with the voice input unit, and the noise generation source Even if the relative position to the voice input unit changes, the robot can estimate the direction in which a speaker (sound source) located outside the excluded angle range uttered from the sound pressure component value of the effective direction sound pressure component data. it can.

  Further, according to the invention of claim 1, since the sound from the noise generation source can be removed by setting the exclusion angle range based on the position of the noise generation source, the device that becomes the noise generation source There is no need to control.

  According to the second aspect of the invention, the robot can obtain a value for evaluating both the correctness of the speech recognition result and the correctness of the direction in which the speech is input, and the calculated value is large. The higher the probability that the speech recognition result is correct, the higher the recognition rate of speech recognition can be reduced. For example, by providing a reference value based on whether the recognition rate is correct or incorrect, it is possible to estimate that the direction of the calculated value that is equal to or greater than the reference value is the sound source direction.

  According to the third aspect of the present invention, the robot can acquire the voice recognition result with the highest probability that the voice recognition result is correct by extracting the maximum value from the plurality of calculated values. The recognition error rate of recognition can be reduced.

  According to the fourth and fifth aspects of the present invention, the robot recognizes the hypothesis phrase having the maximum phrase certainty as a determination criterion indicating the correctness of the speech recognition result, and the speech from the sound source direction. By using the phrase, the recognition error rate can be reduced.

  According to the sixth aspect of the present invention, the robot can turn its face in the direction of the calling speaker.

1 is an overall configuration diagram of a robot system including a robot according to the present invention. It is a block diagram which shows the whole structure of the robot which concerns on this invention. (A) It is a front view of head R1 of the robot which concerns on this invention. (B) It is AA sectional drawing of head R1 of the robot which concerns on this invention. It is a block diagram which shows the structure which concerns on the sound source direction estimation process of the robot which concerns on this invention. It is a figure which shows an example of audio | voice input angle DB of the robot which concerns on this invention. It is a figure which shows an example of phrase DB of the robot which concerns on this invention. It is a figure which shows an example of the B-axis of the robot which concerns on this invention, an H-axis, and an exclusion angle range. It is a figure which shows an example of the MUSIC spectrum produced | generated in the audio | voice data processing part of the robot which concerns on this invention. It is a flowchart of the sound source direction estimation process of the robot according to the present invention. FIG. 10 is a flowchart of a sound source direction estimation process of the robot following FIG. 9. FIG. It is a flowchart of the sound source direction estimation process of the robot following FIG.

  Hereinafter, a robot according to an embodiment of the present invention (hereinafter referred to as “this embodiment”) will be described with reference to the drawings. Each figure is only schematically shown so that the present invention can be fully understood. Therefore, the present invention is not limited to the illustrated example. Moreover, in each figure, the same code | symbol is attached | subjected about the common component and the same component, and those overlapping description is abbreviate | omitted.

[Robot system configuration]
First, an overall configuration of a robot system including a robot according to the present embodiment will be described with reference to FIG. Here, an autonomous mobile biped robot will be described as an example.
As shown in FIG. 1, the robot system includes a robot R, a base station 1 connected to the robot R by wireless communication, and a management computer 3 connected to the base station 1 via a robot dedicated network 2. The terminal 5 is connected to the management computer 3 via the network 4. In this robot system, although not shown, a plurality of robots R are assumed.

As shown in FIG. 1, the robot R according to the present embodiment has a head R1, an arm R2, a leg R3, a trunk R4, and a back housing R5, and is connected to the trunk R4. The head R1, arm R2, and leg R3 are each driven by an actuator (driving means), and bipedal walking is controlled by an autonomous movement controller 50 (see FIG. 2) described later. Details of this bipedal walking are disclosed in, for example, Japanese Patent Application Laid-Open No. 2001-62760.
Here, the robot R includes a back surface storage portion R5 on the back surface of the body portion R4. In the rear storage portion R5, a control device (such as the main control portion 40) that controls the operations of the head portion R1, the arm portion R2, the leg portion R3, and the torso portion R4, and heat processed by the control device are exhausted. A fan F (noise generation source), a battery 70 described later, and the like are stored. As the fan F rotates, a noise sound is output.

  The robot dedicated network 2 connects the base station 1, the management computer 3, and the network 4, and is realized by a LAN (Local Area Network) or the like.

  The management computer 3 manages a plurality of robots R, and performs various controls such as movement and speech of the robot R via the base station 1 and the robot dedicated network 2 and information necessary for the robot R. I will provide a. Here, the necessary information corresponds to voice data for speaking, the name of the detected person, a map around the robot R, and the like. These pieces of information are provided in the management computer 3. It is stored in a storage unit (not shown).

  The terminal 5 is connected to the management computer 3 via the network 4 and registers information about a person in the storage means (not shown) provided in the management computer 3 or corrects the registered information. To do. The terminal 5 is used for registering tasks to be executed by the robot R, changing a task schedule set in the management computer 3, inputting an operation command for the robot R, and the like.

Hereinafter, the robot R and the management computer 3 will be described in detail.
[robot]
FIG. 2 is a block diagram showing the overall configuration of the robot. As shown in FIG. 2, the robot R includes a camera C, a speaker S, a voice input unit MC, and an image processing unit in addition to a head R1, an arm R2, a leg R3, a trunk R4, and a rear storage unit R5. 10, a voice processing unit 20, a storage unit 30, a main control unit 40, an autonomous movement control unit 50, a wireless communication unit 60, and a target detection unit 80.

The head R1 is rotated left and right about the support shaft O on the trunk R4 by a head rotation unit R11 described later controlled by the autonomous movement control unit 50 (behavior control unit). As a result, the direction (H axis) in which the head R1 (face) faces is shifted from the direction (B axis) in which the front of the torso R4 faces by the rotation angle θ. Here, the autonomous movement control unit 50 (rotation angle measurement unit) holds the angle (rotation angle θ) rotated by the head rotation unit R11. That is, the autonomous movement control unit 50 holds the rotation angle θ that is a measurement value of the rotation angle of the head R1.
Further, the robot R has a gyro sensor SR1 and a GPS receiver SR2 in order to detect the current position of the robot R.
The camera C, the speaker S, and the voice input unit MC are all disposed inside the head R1.

[camera]
The camera C (C1, C2) is capable of capturing video as digital data, and for example, a color CCD (Charge-Coupled Device) camera is used. The camera C1 and the camera C2 are arranged side by side in parallel on the left and right, and the captured image is output to the image processing unit 10.

[Image processing unit]
The image processing unit 10 is a part for recognizing surrounding obstacles and persons in order to process the image captured by the camera C and grasp the situation around the robot R from the captured image. The image processing unit 10 includes a stereo processing unit 11a, a moving body extraction unit 11b, and a face recognition unit 11c. Each component will be briefly described below.

  The stereo processing unit 11a performs pattern matching using one of the two images taken by the left and right cameras C as a reference, calculates the parallax of each corresponding pixel in the left and right images, and generates a parallax image. The parallax image and the original image are output to the moving object extraction unit 11b. This parallax represents the distance from the robot R to the photographed object.

The moving body extraction unit 11b extracts a moving body in the photographed image based on the data output from the stereo processing unit 11a. The reason for extracting the moving object (moving body) is to recognize the person by estimating that the moving object is a person.
In order to extract the moving object, the moving object extraction unit 11b stores images of several past frames (frames), compares the newest frame (image) with the past frames (images), and determines the pattern. Matching is performed, the movement amount of each pixel is calculated, and a movement amount image is generated. Then, from the parallax image and the movement amount image, when there is a pixel with a large movement amount within a predetermined distance range from the camera C, it is estimated that there is a person, and the movement is performed as a parallax image only in the predetermined distance range. The body is extracted and an image of the moving body is output to the face recognition unit 11c.

The face recognition unit 11c extracts a skin color portion from the extracted moving body, and recognizes the face position from the size, shape, and the like. Similarly, the position of the hand is also recognized from the skin color area, size, shape, and the like.
The recognized face position is output to the main control unit 40 as information when the robot R moves and to communicate with the person.

[Voice input section]
The voice input unit MC includes a microphone that extracts sound around the robot R as an electrical signal (acoustic signal), and an analog-digital converter (A) that digitizes the acoustic signal and generates acoustic signal data (including voice data). / D converter) and output to the speech processing unit 20 (speech data processing unit 22 and estimated sound source separation unit 24 described later).
Fig.3 (a) is a front view of head R1, and FIG.3 (b) is AA sectional drawing of head R1. FIG. 3B shows the voice input unit MC arranged inside the head R1.
The voice input parts MC1, MC2,..., MC8 are arranged at angles ∠H _mc (∠H _mc1 (= H axis), ∠H from the direction (H axis) in which the face faces around the support axis O, respectively. _mc2, ∠H _mc3, ···) only at a distance, is disposed so as to collect the sound from the head R1 outside.

Here, the disposition angle ∠H _mc may be any angle that allows the sound input unit MC to be disposed so that sound from all directions of 360 degrees can be collected, but as shown in FIG. When there are eight input parts MC, it is desirable that they are equiangular (45 degrees).
Also, outside the exclusion angular range r to be described later, so that the voice input unit MC is present two or more at any time, the number and disposed angle ∠H _mc speech input unit MC is set. Here, in the present embodiment, in order to obtain the sound source direction based on the acoustic signal data input from a plurality of directions, it is necessary that two units exist outside the excluded angle range r.

[Speaker]
The speaker S outputs sound based on the sound data generated by the sound processing unit 20 (speech synthesis unit 21 described later).

[Audio processor]
The speech processing unit 20 includes a speech synthesis unit 21, a speech data processing unit 22, a sound source direction estimation unit 23, an estimated sound source separation unit 24, a speech recognition unit 25, and a sound source localization unit 26.
The voice synthesizer 21 is a part that generates utterance voice data from the character information (text data) based on the utterance action command determined by the main controller 40 and outputs the voice to the speaker S. For the generation of speech voice data, the correspondence between the character information (text data) stored in advance in the storage unit 30 and the voice data from which the character information is read out is used. The character information and the voice data are acquired from the management computer 3 and stored in the storage unit 30.

The processing related to the sound source direction estimation process of the present embodiment includes a sound data processing unit 22, a sound source direction estimating unit 23, an estimated sound source separating unit 24, a sound recognizing unit 25, and a sound source localization unit 26 included in the sound processing unit 20. And done. Since a detailed description of the sound source direction estimation process performed by these components will be described later, here, the sound source direction estimation process performed by the sound processing unit 20 will be briefly described.
The sound processing unit 20 uses a signal decomposition algorithm (for example, MUSIC (MUltiple SIgnal Classification) method) on the acoustic signal data input from the sound input unit MC (MC1, MC2, MC3,...) Omnidirectional sound pressure component data (for example, MUSIC spectrum) indicating sound pressure values from 360 degrees in all directions with reference to the direction is generated. Then, the sound processing unit 20 estimates the sound source direction from the omnidirectional sound pressure component data. Next, the speech processing unit 20 separates and extracts the acoustic signal data input from the estimated sound source direction from the acoustic signal data input from the speech input unit MC, and performs speech recognition on the separated and extracted acoustic signal data. I do.

[Storage unit]
The storage unit 30 is composed of, for example, a general hard disk, etc., and character information and speech data used when the speech synthesizer 21 transmitted from the management computer 3 generates speech speech data, an object detection unit 80 to be described later. In addition to the necessary information used by the person (person name, local map data, conversation data, etc.), the identification number and position information of the person identified by the robot R, data related to the sound source direction estimation processing of this embodiment is stored. Is.
Data related to the sound source direction estimation processing is stored in the voice input angle DB 31, the excluded angle range DB 32, the phrase DB 33 (phrase storage unit), and the transfer function DB 34, and detailed description thereof will be described later. .

[Main control section]
The main control unit 40 performs overall control of the image processing unit 10, the sound processing unit 20, the storage unit 30, the autonomous movement control unit 50, the wireless communication unit 60, and the target detection unit 80.

[Autonomous Movement Control Unit]
The autonomous movement control unit (autonomous movement control means) 50 drives the head R1, the arm R2, and the leg R3 in accordance with instructions from the main control unit 40.
The data detected by the gyro sensor SR1 and the GPS receiver SR2 is output to the main control unit 40 and used to determine the behavior of the robot R.

[Wireless communication part]
The wireless communication unit 60 is a communication device that transmits and receives data to and from the management computer 3. The wireless communication unit 60 includes a public line communication device 61a and a wireless communication device 61b.
The public line communication device 61a is a wireless communication means using a public line such as a mobile phone line or a PHS (Personal Handyphone System) line. On the other hand, the wireless communication device 61b is a wireless communication unit using short-range wireless communication such as a wireless LAN conforming to the IEEE802.11b standard.
The wireless communication unit 60 performs data communication with the management computer 3 by selecting the public line communication device 61 a or the wireless communication device 61 b in accordance with a connection request from the management computer 3.

[Battery]
The battery 70 is used for the operation of each part of the robot R (head R1, arm R2, leg R3, and torso R4), rotational drive of the fan F, processing of the control device stored in the rear storage part R5, and the like. It is a source of necessary power. The battery 70 has a rechargeable configuration.

[Target detection unit]
The target detection unit 80 detects whether or not there is a person with the tag T around the robot R. For example, it includes a plurality of light emitting units that are configured by LEDs and are arranged in front, rear, left, and right along the outer periphery of the head R1 of the robot R (not shown).
The object detection unit 80 transmits infrared light including a signal indicating a light emitting unit ID for identifying each light emitting unit from the light emitting unit, and receives a reception report signal from the tag T that has received the infrared light. The tag T that has received any infrared light generates a reception report signal based on the light emitting unit ID included in the infrared light, so that the robot R determines the light emitting unit ID included in the reception report signal. By referencing, it is possible to specify in which direction the tag T is present when viewed from the robot R. Further, the target detection unit 80 has a function of specifying the distance to the tag T based on the radio wave intensity of the reception report signal acquired from the tag T. Therefore, the target detection unit 80 can specify the position (distance and direction) of the tag T as the position of the person based on the reception report signal.
Furthermore, the object detection unit 80 not only emits infrared light from the light emitting unit, but also transmits a radio wave including a signal indicating the robot ID from an antenna (not shown). Thus, the tag T that has received the radio wave can correctly identify the robot R that has transmitted infrared light. Details of the target detection unit 80 and the tag T are disclosed in, for example, Japanese Patent Application Laid-Open No. 2006-192563.

Next, a configuration related to the sound source direction estimation process of the robot R according to the present embodiment will be described with reference to FIG.
[Configuration of storage unit]
As shown in FIG. 4, the storage unit 30 includes a voice input angle DB 31, an excluded angle range DB 32, and a phrase DB 33.

(Audio input angle DB)
The voice input angle DB 31 includes an identification number of the voice input unit MC and an arrangement angle ∠H _mc (∠H _mc1 , ∠H _mc2 , ∠H) formed from the direction (H axis) in which the voice input unit MC faces. _mc3 ,...) in association with each other and stored.
FIG. 5 is a diagram illustrating an example of the voice input angle DB when the voice input unit MC is disposed inside the head R1, as shown in FIG. In the voice input angle DB, an arrangement angle (∠H _mc1 ) = 0 degrees is recorded in an identification number “MC1” corresponding to the voice input unit MC1, and an identification number “MC2” corresponding to the voice input unit MC2 is recorded. In addition, an arrangement angle (∠H _mc2 ) = 45 degrees is recorded.

(Transfer function DB)
The transfer function DB 34 has a sound source in the θ direction, which is obtained by simulation in advance to generate the MUSIC spectrum P _avg (θ) in the audio data processing unit 22 to be described later, and is based on the head direction. The transfer function vector v (θ) is recorded. This transfer function vector v (θ) is recorded in the number of directions performed at the time of simulation. Details of the transfer function vector v (θ) will be described in the description of the audio data processing unit 22 described later.
In the present embodiment, a simulation is performed in advance when there is a sound source in 72 directions obtained by equally dividing 360 degrees in all directions at 5 degrees, and 72 transfer function vectors v (θ) (v ₁ , v ₂ ,. ·, V ₇₂ ) is calculated in advance.

(Exclusion angle range DB)
The exclusion angle range DB 32 is a storage unit that stores an exclusion angle range r that is input from the terminal 5 by the administrator of the robot R and set in the robot R via the management computer 3.
The excluded angle range r is a range in a direction not estimated as a sound source, which is formed from a direction (B axis, see FIG. 7) in which the front surface of the trunk portion R4 faces around the support axis O. For example, it is stored as 。B = + 120 to +180 to +240 degrees as the excluded angle range r.

This exclusion angle range r is a range set in advance based on the voice input unit MC and the operating environment of the robot R. For example, in order to remove the noise sound output from the fan F (noise generation source) stored in the back surface storage unit R5 of the robot R, the noise of the noise sound by the fan F input from the audio input unit MC in advance is removed. For the signal data, the audio data processing unit 22 is caused to generate omnidirectional sound pressure component data (for example, MUSIC spectrum). The sound pressure component of the noise sound is extracted from the omnidirectional sound pressure component data, and the range of influence of the noise sound is considered.
Here, the influence given by the noise sound is that the voice recognition unit 25 described later does not recognize a voice from other than the fan F (noise generation source) as a voice by the noise sound.

(Phrase DB)
The phrase DB 33 is a database that stores the phrase pattern 331, the direction angle value pattern 332, and the phrase input direction suitability 333 (speech input direction suitability) in association with each other.
FIG. 6 is a diagram illustrating an example of the phrase DB.
The phrase pattern 331 is a phrase uttered by a person, and in particular, there are a plurality of phrases that are uttered when a person (speaker Y, see FIG. 1) calls to another person (robot R, see FIG. 1) to turn around. It is remembered. For example, “Look here”, “Look here”, “Oi”, “I'm sorry”, etc.

The direction angle value pattern 332 is an angle formed from the direction in which the face is facing (H axis) (see FIG. 3B), and 360 degree directional angle values are stored.
In FIG. 6, the direction angle value is a value every 20 degrees, but may be a value every 1 degree.

The phrase input direction suitability 333 (speech input direction suitability) is a value indicating from which angle a phrase uttered by a human is likely to be spoken (input) to the robot R, that is, the phrase and the direction angle value. Is a value indicating the correctness of the relationship (the probability that the relationship between the phrase and the direction angle value is correct).
For example, the phrase “Look here” is not a phrase that is said to be from the direction in which the face of the robot R is facing (on the H axis, the direction angle value = 0 degree), but the direction in which the face of the robot R is not facing (horizontal Paying attention to the phrase that is said to be from the back), the phrase input direction suitability 333 is a value preset by the administrator of the robot R.

In FIG. 6, when the phrase is “Looking over here”, the value of the phrase input direction suitability 333 when the direction angle value is “0 degree” is set low (= 0.00), while the direction angle value is “180”. The value of the phrase input direction suitability 333 in “degree” is set high (= 0.10).
When the phrase is “What is your name?”, It is a phrase that is said from the direction in which the face of the robot R is facing (on the H axis, the direction angle value = 0 degrees). Therefore, the value of the phrase input direction fitness 333 when the direction angle value is “0 degree” is set high (= 0.30), while the value of the phrase input direction fitness 333 when the direction angle value is “180 degrees” is It is set low (= 0.00).
Here, the phrase DB 33 of FIG. 6 has a value of the phrase input direction suitability 333 for each direction angle value so that the sum of the values of the phrase input direction suitability 333 of the omnidirectional angle values is 1 for each phrase. Is set.

[Configuration of audio processing unit]
As shown in FIG. 4, the speech processing unit 20 includes a speech data processing unit 22, a sound source direction estimation unit 23, an estimated sound source separation unit 24, a speech recognition unit 25, and a sound source localization unit 26. The

(Audio data processing unit)
In the present embodiment, the audio data processing unit 22 uses the MUSIC method as a signal decomposition algorithm. The audio data processing unit 22 estimates the noise signal space from the acoustic signal data input from the audio input unit MC (MC1, MC2, MC3,...) Using the MUSIC method, and prepares in advance. This is a processing unit that generates MUSIC spectrum P _avg (θ) (omnidirectional sound pressure component data) by projecting each 72-direction transfer function vector v (θ) to the space of the noise signal.

The transfer function vector v (θ) is a vector value obtained by simulation in advance.
In the simulation, an impulse is output from a sound source in the θ direction with reference to the head direction. The voice data processing unit 22 acquires an impulse response output from each voice input unit MC (MC1, MC2, MC3,...) To which the impulse is input. Then, the audio data processing unit 22 performs a discrete Fourier transform on the impulse response and converts the impulse response into the frequency domain, thereby obtaining a transfer function vector v (θ).

The simulation is performed as follows. When calculating the transfer function vector v ₁ corresponding to the sound source located in the direction of 0 degrees (θ = 0) with respect to the head direction, an impulse is output from the direction of 0 degrees. The voice data processing unit 22 calculates a transfer function vector v ₁ by performing discrete Fourier transform on the impulse response acquired from each voice input unit MC (MC1, MC2, MC3,...) And converting the impulse response to the frequency domain. can do.
In the present embodiment, a simulation is performed in advance when there is a sound source in 72 directions obtained by equally dividing 360 degrees in all directions at 5 degrees, and 72 transfer function vectors v (θ) (v ₁ , v ₂ ,. ·, V ₇₂ ) is calculated in advance.

(MUSIC spectrum calculation process)
Then, the audio data processing unit 22 performs a MUSIC spectrum calculation process.
First, the audio data processing unit 22 acquires acoustic signal data from the audio input unit MC, and 72 transfer function vectors v (θ) (v ₁ , v ₂ , v ₃ ) from the transfer function DB 34 of the storage unit 30. ,..., V ₇₂ ).
Then, the audio data processing unit 22 calculates the MUSIC spectrum P (θ) using Expression (1). e is an eigenvector, which is a value calculated from acoustic signal data. The calculation means will be described later.
Here, M is the number of voice input units MC. N indicates the maximum number that can be recognized as a sound source, and can be set to a predetermined value. In this embodiment, “N = 3” is set. T indicates a transposed matrix.

(Calculation of eigenvector e)
Here, the eigenvector e is calculated from the acoustic signal data as follows.
First, the audio data processing unit 22 performs discrete Fourier transform on the acoustic signal data, converts it into the frequency domain, and calculates the spectrum x.
Then, the correlation matrix R _xx can be expressed by Expression (2) indicated by the next expected value E.

An eigenvalue λ and an eigenvector e such that the correlation matrix R _xx satisfies the equation (3) are calculated. Thereby, the eigenvector e which is a projection matrix of the noise signal onto the space can be acquired.

The audio data processing unit 22 holds a set of all eigenvalues λ and eigenvectors e that satisfy Expression (3). Let K be the number of sets held at this time.
Then, the audio data processing unit 22 acquires a set of the (N + 1) th to Kth eigenvalues λ and the eigenvector e from the set having the largest eigenvalue λ. The MUSIC spectrum P (θ) shown in Expression (1) is calculated using the K−N eigenvectors e.

  Here, normally, there are as many pairs (M) as the number of voice input units MC that satisfy the formula (3). Therefore, the value of N set as the maximum number that can be recognized as a sound source is preferably N <M.

As described above, the audio data processing unit 22 can perform the MUSIC spectrum calculation process and acquire the MUSIC spectrum P (θ) of the frequency ω at time t.
Then, the audio data processing unit 22 performs a calculation process of the MUSIC spectrum P (θ) for each frequency, and acquires the MUSIC spectrum P (θ) in a predetermined frequency band.
Here, the predetermined frequency band is a frequency band in which the sound pressure of the voice uttered by the speaker is high, and a frequency band in which the sound pressure of the noise is low, for example, may be 0.5 to 2.8 kHz.

Then, the audio data processing unit 22 extends the MUSIC spectrum P (θ) of each frequency band to a wideband signal.
When the audio data processing unit 22 extracts a frequency band ω having a good S / N ratio (low noise) from the acoustic signal data and expanding the frequency band ω to a wideband signal, the MUSIC spectrum P (θ) of the frequency band ω is strong. As shown in Equation (4), the MUSIC spectrum P (θ) is weighted using the largest eigenvalue λ _max obtained from Equation (3) from the acoustic signal data in the frequency band ω. To calculate the sum. As a result, a broadband MUSIC spectrum P _avg (θ) is acquired.

Ω：周波数帯域の集合、|Ω|：集合Ωの要素数

Ω: Set of frequency bands, | Ω |: Number of elements in set Ω

As described above, the audio data processing unit 22 can generate the MUSIC spectrum P _avg (θ) (omnidirectional sound pressure component data).

(Sound source direction estimation unit)
The sound source direction estimation unit 23 includes a filter unit 231 and an angle extraction unit 232.
The filter unit 231 acquires the MUSIC spectrum P _avg (θ) (omnidirectional sound pressure component data) from the audio data processing unit 22, acquires the excluded angle range r from the excluded angle range DB 32, and the autonomous movement control unit 50 To obtain the rotation angle θ of the head rotation unit R11.
The filter unit 231 corresponds to the excluded angle range r with respect to the B axis in consideration of the rotation angle θ with respect to the B axis from the MUSIC spectrum P _avg (θ) with the H axis as a reference. Data within the range is removed, and an estimated range spectrum (effective direction sound pressure component data) is extracted.

The angle extraction unit 232 extracts all peak sound pressure values (Peak1, Peak2,...) And the angles (estimated sound source angles) of these peak sound pressure values from the estimated range spectrum. The direction formed by the estimated sound source angle is the estimated sound source direction. Here, the estimated sound source angle in this embodiment is an angle ∠φ _H (∠φ _H1 , ∠φ _H2 ,...) With the H axis as a reference.
Here, the peak sound pressure value will be described. In the estimated range spectrum, when the sound pressure value at the angle θ is larger than the sound pressure value at the immediately preceding angle and the sound pressure value at the immediately following angle, the sound pressure value at the angle θ is the peak sound pressure value. is there. The angle θ is the estimated sound source angle ∠φ _H.
Then, the angle extraction unit 232 outputs the estimated sound source angles ∠φ _H (∠φ _H1 , ∠φ _H2 ,...) Of all the extracted peak sound pressure values to the estimated sound source separation unit 24.
This estimated sound source angle ∠φ _H is also a rotation angle necessary until the front surface (face) of the head R1 faces the estimated sound source direction.

Here, the MUSIC spectrum P _avg (θ) (omnidirectional sound pressure component data) generated by the sound data processing unit 22 is acquired from the sound signal data input from the sound input unit MC, and the sound source direction estimation unit 23 (filter) The processing until the unit 231 and the angle extracting unit 232) estimate the sound source direction will be described with reference to FIGS. FIG. 7 is a diagram illustrating an example of the B axis, the H axis, and the excluded angle range r, and FIG. 8 is a diagram illustrating an example of the MUSIC spectrum generated by the audio data processing unit 22.

In FIG. 7, the direction in which the face of the head R1 faces (H axis) is the direction (θ =) 30 degrees to the right from the direction in which the front of the torso R4 faces (B axis). FIG. At this time, the H axis (∠H = 0 degree) is represented by ∠B = + 30 degrees when the B axis is converted with reference (0 degree).
In order to remove the noise sound output from the fan F (noise generation source) stored in the rear storage unit R5, the excluded angle range r (r = 120 degrees) is ∠B = + 120 to +180 to +240 degrees. It is memorized in the range. This is a range of ∠H = + 80 to +140 to +200 degrees in terms of the H axis.

FIG. 8 shows the 360 generated by the audio data processing unit 22 using the MUSIC method for the acoustic signal data input to the audio input units MC (MC1 to MC8) of the head R1 in the state shown in FIG. MUSIC spectrum P _avg (θ) (omnidirectional sound pressure component data) indicating sound pressure values (spectrum intensity) from all directions. The vertical axis is the sound pressure [dB] indicating the spectral intensity, and the horizontal axis is the angle [degree] indicating the direction with respect to the H axis.

The filter unit 231 removes data in a range corresponding to the excluded angle range r in the MUSIC spectrum P _avg (θ), and acquires a spectrum (estimated range spectrum). The estimated range spectrum in FIG. 8 is spectrum data represented by ∠H = −160 to ± 0 to +80 degrees other than the excluded angle range r (∠H = + 80 to +140 to +200) (indicated by a thick line in FIG. 8). .

From this estimated range spectrum, the angle extraction unit 232 estimates the estimated sound source angle ∠φ _H1 (∠φ _H ) (∠H = −70 degrees) of the peak sound pressure value Peak1 and the estimated sound source angle ∠φ of the peak sound pressure value Peak2. _H2 (∠φ _H ) (∠H = + 10 degrees) is extracted. Then, the estimated sound source angles ∠φ _H1 and ∠φ _H2 are output to the estimated sound source separation unit 24.

  As described above, the maximum sound pressure value (MAX) (see FIG. 8) in the MUSIC spectrum is within the excluded angle range r (∠H = + 80 to +140 to +200) by the processing of the filter unit 231 and the angle extraction unit 232. To be deleted.

(Estimated sound source separation unit)
The estimated sound source separation unit 24 acquires all estimated sound source angles ∠φ _H (∠φ _H1 , ∠φ _H2 ,...) Indicating the estimated sound source direction from the angle extraction unit 232, and the voice input units MC (MC1, MC2). , MC3,...) Acoustic signal data is acquired. Then, the estimated sound source separation unit 24 separates and extracts the sound signal data input from the direction of each estimated sound source angle ∠φ _H from the sound signal data from all 360 degrees obtained from the plurality of acquired sound signal data. To do. Finally, the estimated sound source separation unit 24 recognizes all estimated sound source angles ∠φ _H (∠φ _H1 , Hφ _H2 ,...) And the estimated sound source direction speech data separated and extracted based on the respective angle directions. To the unit 25.

Here, a process for separating and extracting specific acoustic signal data from acoustic signal data input from a plurality of sound input units MC performed by the estimated sound source separation unit 24 is disclosed in Japanese Patent Application Laid-Open No. 2008-306712. This is a process using a known technique.
The process will be briefly described.

Each of the plurality of sound input units MC receives a sound signal on which individual sound source signals are superimposed from each of the plurality of sound sources. The estimated sound source separation unit 24 acquires acoustic signal data from the voice input unit MC. And the estimation sound source separation part 24 isolate | separates the sound source signal (acoustic signal data) from each sound source from each sound signal data using a blind sound source separation method (BSS: Blind Source Separation). Then, the estimated sound source separation unit 24 extracts acoustic signal data (estimated sound source direction sound data) input from the direction of the estimated sound source angle ∠φ _H.
As BSS, DSS (Decorrelation based Source Separation), ICA (Independent Component Analysis), HDSS (Higher-order DSS) based sound source separation method, GSS (Geometric constrained Source Separation with geometric information added to each of these methods) ), GICA (Geometric constrained ICA), or GHDSS (Geometric constrained HDSS) may be used.

(Voice recognition unit)
The speech recognition unit 25 acquires all the estimated sound source angles ∠φ _H (∠φ _H1 , ∠φ _H2 ,...) And the estimated sound source direction speech data of each angular direction from the estimated sound source separation unit 24. .
The speech recognition unit 25 has a function of performing speech recognition on each estimated sound source direction speech data and generating character information (text data), and generates a plurality of hypothesis phrases from one estimated sound source direction speech data. . This hypothesis phrase is generated as text data.

Furthermore, the speech recognition unit 25 has a function of calculating speech recognition likelihood indicating the correctness of the hypothesis phrase (the probability that the hypothesis phrase is correct). This speech recognition likelihood indicates whether or not the hypothesis phrase is correct sentence content from the contents of the hypothesis phrase and the sentence contents before and after (preceding phrase before and after).
<Voice Recognition Likelihood Calculation Processing of Voice Recognition Unit 25>
A speech recognition likelihood calculation process performed by the speech recognition unit 25 will be described.
The speech recognition likelihood (P (W | X)) in general speech recognition can be expressed by the following equation (5), where X is a speech signal and W is a phrase (word string).

Ｐ（Ｗ｜Ｘ）：音声認識尤度、Ｐ（Ｘ｜Ｗ）：音響尤度、Ｐ（Ｗ）：言語確率

P (W | X): speech recognition likelihood, P (X | W): acoustic likelihood, P (W): language probability

Here, when calculating the language probability P (W), a grammar may be used, or a statistical language model may be used.
When calculating the language probability P (W) using the grammar, P (W) = 1 if the phrase W is defined in the grammar, and P (W) = 0 if not defined. Then, according to the definition of the probability, finally, normalization is performed so that the sum of P (W) becomes 1.
On the other hand, when the language probability P (W) is calculated using a statistical language model, an N-gram learned from a large amount of sentences for the ease of connection between words is used. In general, N-gram is a learning of the ease of connection of N words.

Here, a case where N = 3, which is used in many cases, will be described using a statistical language model. Generally, when learning using a 3-gram statistical language model, 2-gram is also learned at the same time.
For example, in the case of “crossing the bridge (crossing the bridge)”, it is divided into four words such as “bridge | Then, after the two words “bridge | o”, learning the probability P (cross | bridge, o) how easily the word “cross” appears, and database (3-gram database, 2-gram) As a database).
Thus, the probability P (the bridge | is crossed) that the phrase W “bridge | cross | cross” can be expressed by the following equation (6).

The right side is obtained from the 3-gram and 2-gram databases, and the left side is the language probability P (W).
From the above, the speech recognition unit 25 calculates the speech recognition likelihood (P (W | X)) of the hypothesis phrase “cross the bridge” with a large value (the probability that the hypothesis phrase is correct is high), The speech recognition likelihood (P (W | X)) of the hypothesis phrase “cross chopsticks” is calculated with a small value (the hypothesis phrase has a low probability of being correct).
As described above, the speech recognition likelihood calculation process is performed in the speech recognition unit 25, and the speech recognition likelihood (P (W | X)) is calculated.

The speech recognition unit 25 then generates all hypothesis phrases generated from the estimated sound source direction data in the angular direction indicated by the estimated sound source angle ∠φ _H , and the speech recognition likelihood (P (W | X)) of each hypothesis phrase. Is associated with the estimated sound source angle ∠φ _H and output to the sound source localization unit 26 (phrase certainty calculation unit 261).
Note that the correspondence between the voice data and the text data is stored in the storage unit 30 in advance.

(Sound source localization part)
The sound source localization unit 26 includes a phrase certainty factor calculation unit 261 and a phrase determination unit 262.
(Phrase certainty calculator)
The phrase certainty factor calculation unit 261 is a hypothetical phrase associated with each of the estimated sound source angles ∠φ _H (1φ _H1 , ∠φ _H2 ,...) And the estimated sound source angle ∠φ _H from the speech recognition unit 25. And the speech recognition likelihood (P (W | X)) of each hypothesis phrase.
Then, the phrase certainty calculation unit 261 extracts one estimated sound source angle ∠φ _H from all the estimated sound source angles ∠φ _H and, further, from all hypothetical phrases associated with the estimated sound source angle ∠φ _H. One hypothesis phrase is extracted (hereinafter referred to as an extracted hypothesis phrase).
Next, the phrase certainty calculation unit 261 extracts a phrase that matches the extracted hypothesis phrase from the phrase pattern 331 of the phrase DB 33 (FIG. 6). Next, a direction angle value that matches the estimated sound source angle ∠φ _H is extracted from the direction angle value pattern 332 associated with the extracted phrase. Finally, the phrase input direction suitability 333 (probability that the relationship between the phrase and the direction angle value is correct) associated with the extracted direction angle value is extracted.

Next, the phrase certainty calculating unit 261 calculates a phrase certainty indicating the relationship between the sound source direction of the phrase and the speech recognition result of the phrase. It shows that the probability that the speech recognition result of the extracted hypothesis phrase (the hypothesis phrase recognized by the speech recognition unit 25) is correct is higher as the value of the phrase certainty factor is larger.
<Phrase certainty calculation processing of the phrase certainty calculation unit 261>
The phrase certainty factor calculation process performed by the phrase certainty factor calculation unit 261 will be described.
By expanding the expression (5) for calculating the speech recognition likelihood indicating the correctness of the hypothesis phrase (probability that the hypothesis phrase is correct) by using the phrase input direction suitability 333, the phrase confidence P (W | X, d) can be expressed by the following equation (7). Here, X is an audio signal, W is a phrase (word string), and d is a direction.

Ｐ（Ｗ｜Ｘ）：音声認識尤度、Ｐ（Ｘ｜Ｗ）：音響尤度、Ｐ（Ｗ）：言語確率
Ｐ（Ｗ｜Ｘ，ｄ）：フレーズ確信度、Ｐ（ｄ｜Ｗ）：フレーズ入力方向適合度

P (W | X): speech recognition likelihood, P (X | W): acoustic likelihood, P (W): language probability P (W | X, d): phrase confidence, P (d | W): Phrase input direction suitability

  As described above, by performing the phrase certainty factor calculation process, the phrase certainty factor calculating unit 261 obtains the phrase certainty factor (P (W | X, d)) from the phrase DB 33 for the phrase input direction suitability 333 ( P (d | W)) is multiplied by the speech recognition likelihood (P (W | X)).

  Here, the acoustic likelihood P (X | W) is a likelihood calculated from the input speech signal X and a database called an acoustic model, and assuming that the input speech signal X is the phrase W, the input speech signal X represents how much the phrase W seems to be. As this acoustic model, a hidden Markov model is generally used. As an algorithm for calculating the acoustic likelihood of the input speech signal X with respect to the phrase W, there is a Viterbi algorithm or the like.

Then, the phrase certainty calculation unit 261 associates the three values of the estimated sound source angle ∠φ _H , the hypothesis phrase, and the calculated phrase certainty factor (P (W | X, d)) with the phrase determination unit 262. Output to.
The phrase certainty calculation unit 261 performs this process on all estimated sound source angles ∠φ _H acquired from the speech recognition unit 25, and the phrase certainty (P) of all hypothetical phrases for each estimated sound source angle ∠φ _H. (W | X, d)) is calculated and output to the phrase determination unit 262.

(Phrase determination part)
The phrase determination unit 262 performs the following processing on all estimated sound source angles ∠φ _H acquired from the phrase certainty calculation unit 261.
First, the phrase determination unit 262 acquires the phrase certainty factor of all hypothesis phrases associated with one estimated sound source angle ∠φ _H. Then, the phrase determining unit 262 extracts the maximum phrase certainty factor from the acquired phrase certainty factor P (W | X, d), and determines the hypothetical phrase having the maximum phrase certainty factor as a speech-recognized phrase. To do. The hypothesis phrase W _max having the maximum phrase certainty factor is an utterance phrase uttered from a sound source (speaker Y, see FIG. 1) located in the direction of the estimated sound source angle ∠φ _H from the front of the robot R. Suppose there is.
Then, the utterance phrase and the estimated sound source angle ∠φ _H are output to the main control unit 40.
Here, the hypothesis phrase W _max having the maximum phrase certainty factor can be expressed by the following equation (8).

Ｘ：音声認識したフレーズ、Ｗ：仮説フレーズ

X: Phrase recognized speech, W: Hypothesis phrase

The phrase determination unit 262 performs the above processing for all the estimated sound source angles ∠φ _H acquired from the phrase certainty factor calculation unit 261, and outputs the utterance phrase of each estimated sound source angle ∠φ _H to the main control unit 40. To do.

The robot R according to the present embodiment can perform the following operations.
The main control unit 40 that has acquired the utterance phrase of each estimated sound source angle ∠φ _H refers to the character information stored in the storage unit 30 and acquires the content of the utterance phrase. Here, when the utterance phrase is a sentence that calls the other party, the main control unit 40 moves to the autonomous movement control unit 50 until the front (face) of the head R1 reaches the estimated sound source angle ∠φ _H. An instruction to rotate the head rotation unit R11 can be output. Thus, when the speaker calls “Oi”, the robot R can turn the face in the direction in which the speaker is present.
Further, the main control unit 40 that acquired the utterance phrase of each estimated sound source angle ∠φ _H acquires the peak sound pressure value of the angle of the estimated sound source angle ∠φ _H acquired by the angle extraction unit 232 from the estimated range spectrum. . The main control unit 40 then turns the head rotation unit R11 to the autonomous movement control unit 50 until the front surface (face) of the head R1 reaches the estimated sound source angle ∠φ _H where the peak sound pressure value is maximum. An instruction to move may be output. As a result, the robot R can turn its face in the direction in which the speaker having the maximum sound pressure is present. Therefore, the face can be directed in the direction in which there is a speaker who called out louder than a speaker speaking nearby.

[Robot sound source direction estimation processing]
Next, the sound source direction estimation process of the robot will be described with reference to the flowcharts of FIGS. 9 to 11 (refer to FIGS. 1 to 8 as appropriate).
First, the voice uttered by the user is input to the voice input unit MC (MC1, MC2, MC3,...) Of the robot R (step S101), and the acoustic signal data is input to the voice data processing unit 22 (step S101). S102).
Next, the voice data processing unit 22 determines the arrangement angle ∠H _mc (∠H _mc1 , ∠H _mc2 , ∠H _mc3 ,...) Of each voice input unit MC from the voice input angle DB 31 of the storage unit 30. Obtain a transfer function vector v (θ) from the transfer function DB 34 (step S103).
The audio data processing unit 22 uses the MUSIC method, the acoustic signal data inputted from the audio input unit MC, and the disposed angle ∠H _mc, since the transfer function vector v as (theta), the MUSIC spectrum Generate (step S104).

The filter unit 231 acquires the MUSIC spectrum (omnidirectional sound pressure component data) from the audio data processing unit 22, acquires the excluded angle range r from the excluded angle range DB 32, and receives the head rotation unit from the autonomous movement control unit 50. The rotation angle θ of R11 is acquired (step S105).
Then, the filter unit 231 extracts the estimated range spectrum (effective direction sound pressure component data) from the MUSIC spectrum by removing the data within the range corresponding to the excluded angle range r in consideration of the rotation angle θ. (Step S106).

The angle extraction unit 232 extracts all peak sound pressure values (Peak1, Peak2,...) And the angles of these peak sound pressure values (estimated sound source angle ∠φ _H ) from the estimated range spectrum (step S107). . Then, the angle extraction unit 232 outputs all the extracted estimated sound source angles ∠φ _H (∠φ _H1 , ∠φ _H2 ,...) To the estimated sound source separation unit 24.

The estimated sound source separation unit 24 acquires all the estimated sound source angles ∠φ _H (∠φ _H1 , ∠φ _H2 ,...) From the angle extraction unit 232, and the voice input units MC (MC 1, MC 2, MC 3,. The acoustic signal data input from () is acquired (step S108). Then, the acoustic signal data input from the direction of each estimated sound source angle ∠φ _H is separated and extracted from the acquired acoustic signal data (step S109). Finally, the estimated sound source separation unit 24, together with the estimated sound source angles ∠φ _H (∠φ _H1 , 2φ _H2 ,...), The estimated sound source direction speech data separated and extracted based on the respective angle directions, Output to.

The speech recognition unit 25 acquires all the estimated sound source angles ∠φ _H (∠φ _H1 , ∠φ _H2 ,...) And the estimated sound source direction speech data of the respective angular directions from the estimated sound source separation unit 24 (steps). S110).
(Voice recognition processing)
The speech recognition unit 25 performs speech recognition on the estimated sound source direction speech data input from the estimated sound source separation unit 24, generates a plurality of hypothesis phrases (step S111), and uses a predetermined calculation means to determine the hypothesis phrase. A speech recognition likelihood indicating correctness is calculated (step S112). Then, the speech recognition unit 25 performs the speech recognition process on the estimated sound source direction speech data in all estimated sound source angle ∠φ _H directions.

Through the above processing, the speech recognition unit 25 acquires, for each estimated sound source direction speech data, a plurality of hypothesis phrases generated from the estimated sound source direction speech data and the speech recognition likelihood of each hypothesis phrase. Then, the voice recognition unit 25, the estimated sound source angle ∠Fai in association with _H, all the hypotheses phrase generated from the estimated sound source direction audio data of the estimated sound source angle ∠Fai _H direction, speech recognition likelihood of each hypothesis phrases Is output to the phrase certainty calculation unit 261.

Phrase confidence factor computing unit 261, the speech recognition unit of all the 25 estimated sound source angle _{∠φ H (∠φ H1, ∠φ H2} , ···) and all associated with each estimated sound source angle ∠Fai _H The hypothesis phrase and the speech recognition likelihood of each hypothesis phrase are acquired (step S113, FIG. 10).
Then, the phrase certainty calculation unit 261 extracts one estimated sound source angle ∠φ _H from the acquired estimated sound source angles ∠φ _H (∠φ _H1 , ∠φ _H2 ,...) (Step S114), One is extracted from a plurality of hypothesis phrases associated with the estimated sound source angle ∠φ _H (step S115).
Next, the phrase certainty calculation unit 261 extracts a phrase that matches the hypothesis phrase from the phrase pattern 331 of the phrase DB 33 based on the extracted estimated sound source angle ∠φ _H and the hypothesis phrase. Next, the phrase certainty calculation unit 261 extracts a direction angle value that matches the estimated sound source angle ∠φ _H from the direction angle value pattern 332 associated with the extracted phrase. And the phrase certainty calculation part 261 acquires the phrase input direction adaptability 333 matched with the extracted direction angle value (step S116).

Next, the phrase certainty calculation unit 261 multiplies the phrase input direction suitability 333 acquired from the phrase DB 33 by the speech recognition likelihood of the hypothesis phrase acquired from the speech recognition unit 25 to obtain the phrase confidence of the hypothesis phrase. Calculate (step S117). Then, the phrase certainty calculation unit 261 outputs the estimated sound source angle ∠φ _H , the hypothesis phrase, and the calculated phrase certainty value to the phrase determination unit 262 in association with each other.
And the phrase certainty calculation part 261 determines whether there exists an unextracted hypothesis phrase in step S115 (step S118). If there is an unextracted hypothesis phrase (step S118, Yes), the process returns to step S115.

On the other hand, if there is no unextracted hypothesis phrase (No in step S118), the phrase certainty calculation unit 261 determines whether or not there is an unextracted estimated sound source angle in step S114 (step S119). If there is an unextracted estimated sound source angle (step S119, Yes), the process returns to step S114.
On the other hand, if there is no unextracted estimated sound source angle (step S119, No), the process proceeds to the phrase determination unit 262.

The phrase determination unit 262 calculates the phrase confidences of all estimated sound source angles ∠φ _H (∠φ _H1 , ∠φ _H2 ,...) And all hypothetical phrases associated with each estimated sound source angle ∠φ _H. Obtained (step S120, FIG. 11).
First, the phrase determining unit 262 extracts one estimated sound source angle ∠φ _H from the acquired estimated sound source angle ∠φ _H (∠φ _H1 , ∠φ _H2 ,...) (Step S121).
Then, the phrase determination unit 262 extracts the maximum phrase certainty factor from all the phrase certainty factors associated with the extracted estimated sound source angle ∠φ _H (step S122), and the extracted phrase certainty factor hypothesis The phrase is determined as a phrase from the direction of the estimated sound source angle ∠φ _H recognized by the robot R (step S123). Thereby, one phrase is determined for the direction of one estimated sound source angle ∠φ _H.

Then, the phrase determination unit 262 determines whether or not there is an unextracted estimated sound source angle in step S114 (step S124). If there is an unextracted estimated sound source angle (Yes in step S124), the process returns to step S121.
On the other hand, if there is no unextracted estimated sound source angle (No in step S124), the phrase determining unit 262 outputs the phrases of the estimated sound source angles ∠φ _H determined by the processing so far to the main control unit 40 ( Step S125). Then, the robot R ends the sound source direction estimation process.

Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments, and can be implemented with appropriate modifications.
For example, the exclusion angle range DB 32 stores the exclusion angle range r, but may store the angle range calculated by 360-r. Thereby, effective direction sound pressure component data can be extracted from the MUSIC spectrum.
In addition, the angle extraction unit 232 may use an angle ∠φ _{B based} on the B axis instead of the angle ∠φ _{H based on} the H axis as the estimated sound source angle.

  Further, in the exclusion angle range DB 32, the exclusion angle range r is set to a value of less than 120 ° if there are three speech input units, and a value of less than 180 ° if it is 4, so that at least two speech input units MC. Is located outside the excluded angle range r, the direction in which the speaker Y (see FIG. 1) is present (the sound source direction to be estimated) can be estimated.

The phrase DB 33 may store a two-dimensional histogram composed of the direction angle value pattern 332 and the phrase input direction matching degree 333 for each phrase pattern 331. Thereby, the phrase certainty calculation unit 261 acquires the phrase input direction matching degree 333 associated with the direction angle value corresponding to the estimated sound source angle ∠φ _H acquired from the speech recognition unit 25 by the two-dimensional histogram. It will be.

  The robot R may be a robot that communicates with the speaker Y, but may also be a security robot that directs the camera C and the high-performance microphone (voice input unit MC) in the direction in which a sound is heard.

DESCRIPTION OF SYMBOLS 20 Speech processing part 22 Speech data processing part 23 Sound source direction estimation part 24 Estimated sound source separation part 25 Speech recognition part 26 Sound source localization part 30 Storage part 31 Voice input angle DB
32 Exclusion angle range DB
33 Phrase DB
40 Main Control Unit 50 Autonomous Movement Control Unit (Behavior Control Unit) (Rotation Angle Measurement Unit)
231 Filter unit 232 Angle extraction unit 261 Phrase confidence calculation unit 262 Phrase determination unit MC (MC1, MC2, MC3,...) Voice input unit O Support shaft R Robot R1 Head R11 Head rotation unit

Claims (6)

The torso,
A head supported by a support shaft so as to be rotatable on the upper surface of the body part;
Three or more audio inputs arranged in the head and spaced apart at a predetermined angle around the support shaft in the direction of rotation in which the head rotates, and converting the input sound into sound data and outputting it And
Using the signal decomposition algorithm, omnidirectional sound pressure component data indicating sound pressure values from 360 degrees omnidirectional with respect to the head direction is generated for the sound data input from each of the sound input units. An audio data processing unit;
A rotation angle measurement unit for measuring the rotation angle of the head with respect to the direction of the body,
A storage unit storing an exclusion angle range in which a range of directions not estimated as a sound source based on the direction of the body is set;
A sound source direction estimating unit for estimating a sound source direction,
The sound source direction estimation unit
A robot including a filter unit that generates effective direction sound pressure component data by removing data within the excluded angle range from the omnidirectional sound pressure component data using the measured rotation angle. The storage unit stores voice input angles indicating directions of the voice input units around the support shaft,
The sound source direction estimation unit has at least one angle of sound pressure values larger than both of the sound pressure value at the immediately preceding angle and the sound pressure value at the immediately following angle from the effective direction sound pressure component data generated by the filter unit. It has an angle extraction unit to extract above,
An estimated sound source separation unit that separates and extracts the sound data input from each direction of the estimated sound source angle extracted by the angle extraction unit based on the sound input angle from the sound data output by the sound input unit;
A speech recognition unit that performs speech recognition on the estimated sound source direction speech data extracted by the estimated sound source separation unit, generates a plurality of hypothesis phrases, and calculates a speech recognition likelihood indicating the correctness of each hypothesis phrase;
With the speech input direction suitability indicating the correctness of the relationship between the hypothesis phrase and the direction in which the estimated sound source direction speech data related to the hypothesis phrase is input, weighting the speech recognition likelihood related to the hypothesis phrase, The robot according to claim 1 , further comprising: a phrase certainty calculation unit that calculates a phrase certainty factor of each of the hypothesis phrases and estimates a sound source direction based on the calculated phrase certainty factor. The phrase certainty calculation unit
Extracting the maximum phrase certainty factor from the phrase certainty factor of each of the hypothetical phrases, and estimating the estimated sound source angle related to the estimated sound source direction sound data that generated the hypothetical phrase related to the phrase certainty factor in the sound source direction. The robot according to claim 2, wherein A phrase storage unit and a phrase determination unit;
The phrase storage unit includes a plurality of phrase patterns obtained by collecting phrases that can be input in advance, a direction angle value pattern that indicates a direction with the support shaft as a center and a direction of the head, and the phrase pattern is the direction. Three of the voice input direction matching degree indicating correctness input from the direction of the angle value pattern is stored in association with each other,
The phrase certainty calculation unit obtains a plurality of hypothesis phrases generated by the speech recognition unit and the speech recognition likelihood thereof, and uses the hypothesis phrase and the estimated sound source angle extracted by the angle extraction unit, The speech input direction suitability associated with the phrase pattern and the direction angle value pattern stored in the phrase storage unit is extracted, the speech recognition likelihood is weighted by the speech input direction suitability, and the phrase Calculate confidence,
The phrase determination unit recognizes the speech from the sound source direction estimated by the phrase certainty calculation unit related to the hypothesis phrase as a hypothesis phrase related to the maximum phrase certainty factor extracted by the phrase certainty calculation unit. The robot according to claim 3, wherein the robot is a phrase.   The robot according to claim 4, wherein the phrase storage unit stores a two-dimensional histogram composed of the direction angle value pattern and the voice input direction matching degree for each phrase pattern.   The robot according to claim 2, further comprising an action control unit that rotates the head until the estimated sound source angle extracted by the angle extraction unit is reached.

Images