JPH10243494A - Method and device for recognizing direction of face - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the recognition processing efficiently by arranging a couple of microphones in a way that projections of a couple of microphones on a vertical plane opposed to an utterance person are deviated from each other and discriminating in which direction on a line segment tying the projections of the microphones on the vertical plane a face of the utterance person is directed based on a difference of power outputs of the microphones. SOLUTION: Voice signals from a couple of microphones 202, 203 are given to characteristic amount analyzers 210, 211 respectively. The analyzers 210, 211 detect an acoustic change in a voice depending on a difference from a direction from a lip 209 to the microphones 202, 203 caused by the effect of a shape of a vocal organ and a head. A difference extract device 212 calculates a difference between the two characteristic amounts detected by the analyzers 210, 211. A discrimination analyzer 213 discriminates a direction of a face 201 with respect to a vertical plane based on an output of the extract device 212. A direction of the face with respect to a horizontal plane is discriminated based on a difference from characteristic amounts resulting from voice signals picked up by other couple of microphones 204, 205. The direction of the face is discriminated by a difference from voice power extracted by the analyzers 210, 211.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a method and an apparatus for efficiently discriminating a user (user) 's apparatus or an arbitrary object in an interactive speech understanding apparatus or the like.

[0002]

2. Description of the Related Art In communication between human beings, voices for calling, signaling, and gazing at an object are uttered to a partner or an object to be approached. From this, in the speech understanding system, it is possible to discriminate whether a certain speaker is directed to the system or not, and control rejection of unnecessary speech and movement of the focus in speech understanding. From this point, recognition of the gaze direction of the speaker using a face image has been proposed as a method of identifying whether or not the utterance is directed to the system.

A conventional face direction recognition method will be described with reference to an example of pattern matching using an image. In order to recognize a face direction from an image, there is a method in which an image of a face facing in multiple directions is registered in advance as a reference pattern, and the face direction is recognized by collating with an input image [1]. FIG. 4 shows an example of pattern matching of image information. The entire or partial image of the face in multiple directions is registered in advance as a reference pattern group 101 whose face direction is known. Here, the evaluation pattern 1 whose face direction is unknown
10, the individual reference patterns 102-10
6, the most similar one is determined, and the face direction of the matching pattern is recognized as the face direction of the input pattern.

In order to perform image pattern matching, first, an image is divided into small sections, and a feature amount is calculated for each section from the color and brightness distribution of the image in the section. Next, the feature amounts of all the sections constituting the image are obtained. This procedure is performed for both the reference pattern and the evaluation pattern, and the similarity is calculated from the distribution distance of each feature value. As can be seen from FIG. 4, in this method, it is necessary to register a large number of reference patterns in advance, calculate a feature amount for each pattern, and determine similarity. In addition, many calculations are required to divide an image into small sections and obtain the respective feature amounts.

[0005]

The conventional face direction recognition method has a problem that the calculation cost is high because the pattern matching of the image information is performed. It is an object of the present invention to provide a method and a device capable of more efficiently calculating pattern matching for face direction recognition.

[0006]

According to the present invention, a pair of microphones are arranged so as to be deviated from each other when projected onto a vertical plane facing the speaker, and the pair of microphones are arranged so as to be shifted from each other. The difference between the output powers is detected, and based on the difference, it is determined to which position on the straight line connecting the positions of the projection microphones on the vertical plane the face of the speaker is directed. A power difference is detected for a specific frequency band of the microphone output. By providing a pair of microphones in the left-right direction and the up-down direction, it is detected whether the face is facing in either the left-right direction or the up-down direction.

[0007]

FIG. 1 shows an embodiment of a face direction recognition apparatus according to the present invention. As shown in FIG.
Microphones 202 and 203 arranged vertically in order to determine the direction of the face 201 of the speaker 200 in the vertical direction, and horizontally arranged in order to determine the direction of the face 201 in the horizontal plane. Microphones 204 and 205 are provided. In this example, the microphones 202, 203, 204, 20
5 is a substantially vertical surface 20 facing the front of the speaker.
6, the midpoint of the microphones 202 and 203 coincides with the midpoint of the microphones 204 and 205, and passes through the midpoint 207 and is perpendicular to the vertical plane 206.
08 passes through the lips 209 of the face 201 in a state where the face 201 faces the vertical plane 206. That is, the microphones 202 and 20 are placed on a vertical plane passing through the median head of the speaker 200 (vertical plane passing through the straight line 208 in FIG. 1A).
The microphones 204 and 205 are arranged on the left and right sides of the median head and the vertical plane. In addition, each microphone 202
Reference numeral 205 denotes a uni-directional pattern, and the direction of the
Microphones 202, 203 and 204 forming a pair
And 205 have the same characteristics.

One pair of microphones 202 and 203
From the feature analyzer 21 as shown in FIG. 1B.
0,211. Feature value analyzers 210 and 211
Detects an acoustic change in speech due to a difference in direction from the lips 209 to the microphones 202 and 203 under the influence of the shape of the vocal organs and the head. The difference extractor 212 calculates the difference between the two feature amounts detected by the feature amount analyzers 210 and 211. The discriminant analyzer 213 discriminates the direction (vertical direction) of the face 201 in the vertical plane from the output of the difference extractor 212.

Although not shown in the figure, the feature amounts of the voices captured by the other pair of microphones 204 and 205 are also detected, and the direction (horizontal direction) of the face 201 in the horizontal plane is determined from the difference between these feature amounts. FIG. 1C shows a specific implementation example of the device of the present invention. Feature analyzer 21
Audio powers are extracted as 0 and 211, respectively, and the face direction is determined based on a difference between these audio powers. In other words, the sound pressure around the head is attenuated by several to several tens of dB at the back as compared to the front of the face in both the horizontal plane and the distribution of the median and vertical planes of the head. Therefore, the face direction can be determined from the difference in the audio power.

[0010] In this case, if the frequency characteristics of the microphones for comparing the audio powers are different, a difference occurs in the power due to a change in the spectral envelope. To eliminate this, the frequency domain in which the power is compared is limited, and power fluctuation due to a change in the spectral envelope is suppressed. Feature quantity analyzer 210,
The frequency spectrum analyzers 301 and 302 are used as 211. In this method, the strength of the audio signal is extracted for each frequency at regular intervals. A pair of frequency analyzers 30
The transfer function is calculated by a transfer function calculator 303 instead of calculating the difference between the inputs of the outputs 1 and 302 by the difference extractor 212. The transfer function is represented by the frequency spectrum analyzer 301,
Each analysis frequency spectrum of 302 is Px (f), Py
(F) is obtained by the following equation.

Txy (f) = Pxy (f) / Pxx (f) where Pxx (f) represents the frequency spectrum as the square of Px (f), and Pxy (f) represents the frequency spectrum Px (f), Py
(F). The output of the difference extractor, that is, the output of the transfer function calculator 303 is converted to the face direction by the discriminant analyzer 304 and output. The transfer function is a transfer function in which the output of one microphone, for example, 202 is input to the transmission path and the output of the other microphone 203 is the output of the transmission path.

Next, an experimental example will be described. FIG. 2 shows a case where the present invention is applied to the detection of the face direction of a speaker 200 sitting at a desk 401 in a soundproof room. Speaker 2
On the head midline / vertical plane 402 of 00, the speaker 20
An omni-directional collectlet condenser microphone 403 is attached to the chest of the subject 0, and unidirectional dynamic microphones 204 and 205 are placed on the desk 401 at equal distances from the vertical plane 402 to the left and right, and the directional direction of the speaker 200 is changed.
Turned to the face. The sentence prepared in advance was read aloud with the face of the speaker 200 facing in a predetermined direction. Regarding one male and one female speaker 200, each microphone 20 in four directions of front direction 404, left microphone direction 405, right microphone direction 406, and left and right microphone symmetry plane downward 407.
The outputs of 4,205,403 were taken in as digital data at a sampling frequency of 48 kHz, and each was subjected to frequency spectrum analysis to determine the power spectrum ratio of a paired microphone.

When the utterance is directed toward the front direction 404,
The power spectrum ratios of the left and right microphones 204 and 205 are almost flat up to 20 kHz as shown in FIG. 3A. The left when the speaker 200 turns to the left,
As shown in FIG. 3B, the power spectrum ratio of the right microphones 204 and 205 is 1 in addition to the attenuation above 20 kHz.
Attenuation occurs around 3 kHz and 17 kHz. This attenuation does not occur when the speaker makes the same utterance facing the front. Therefore, it is possible to determine whether the speaker 200 faces the front or the left based on the ratio of the output audio powers of the microphones 204 and 205.
In this case, if only the frequency band of FIGS. 3A and 3B which has a particular change, in this example, only the component of 10 kHz to 20 kHz, is taken out, the difference in the magnitude of the sound power ratio depending on the direction of the face is further increased, and the direction can be correctly determined. Can be. Although not shown in the drawing, it is possible to determine whether the microphone is facing the front or to the right based on the magnitude of the output audio power ratio of the microphones 204 and 205.

FIG. 3C shows the ratio of the power spectrum of the left microphone 204 to the power spectrum of the center microphone 403 when the speaker 200 faces front. In this case, the microphones 204 and 4
Since the frequency characteristics of the frequency band 03 differ, a large continuous drop occurs in a high frequency range of 15 kHz or more. When the speaker 200 is turned downward, the ratio of the power spectrum of the microphone 204 to the power spectrum of the microphone 403 drops as shown in FIG. 3D near 13 kHz as in the comparison of the outputs of the left and right microphones.
For C, a slight difference occurs in the power spectrum ratio in a portion where the frequency characteristics are flat. Therefore, by detecting this difference, it is possible to determine whether the vehicle is facing the front or downward. In this case, to avoid the influence based on the difference in the frequency characteristics between the microphones 204 and 403,
In this example, only the components in the 10-15 kHz band are extracted,
If the magnitude of the ratio of the power spectrum is detected, the determination can be made without error.

As will be understood from the above description, as microphones, at least two microphones arranged on the left and right in the horizontal plane and at least two microphones arranged in the vertical direction on the vertical plane are provided. Often,
In this case, one of the arrangement microphones in the horizontal plane and one of the arrangement microphones in the vertical plane can be shared.

By using two pairs of microphones as described above, it is possible to determine whether the direction of the face is in the left-right direction or in the up-down direction. Therefore, it is also possible to determine diagonal directions such as upper right and lower right. As described above, in addition to the detection of the four directions, by using a pair of microphones, it is possible to determine which position on the straight line that connects the microphone with the position projected on the vertical plane facing the speaker. Can be detected. That is, the present invention can be applied to, for example, only the detection in which of the left and right directions.

The direction of the microphone to be used is to avoid the influence of noise and to increase the reception level of the signal sound.
It is better to point to the speaker's face, but it is not necessary. Microphones used in pairs, that is, microphones having a sound power ratio or difference preferably have the same frequency characteristics, but they need not always be the same.

In order to further limit the band, the required components are extracted after the frequency spectrum analysis has been performed in the above, but the microphone output is passed through a filter having a required band as a pass band, and the audio power of the output is obtained. You may. Also, it was determined whether the face was facing the front, left, right, or downward, that is, whether the face was facing left, right, up, or down. Since the difference with respect to the output ratio when facing the front increases, the magnitude of the output ratio is stored in advance for each angle range, and depending on which range the measured power ratio is, The size of the direction can also be determined.

[0019]

According to the conventional face direction recognition method, it is necessary to prepare a large amount of calculations and reference patterns in order to perform image pattern recognition. However, according to the present invention, by extracting a feature amount from a speech signal and performing discriminant analysis, a reference pattern and a calculation amount required for recognition can be reduced and recognition processing can be performed efficiently.

[Brief description of the drawings]

FIG. 1A shows a target face 20 according to an embodiment of the present invention.
FIG. 1B is a diagram illustrating an example of an arrangement relationship between 1 and a microphone; FIG.
C is a block diagram showing a specific implementation example.

FIG. 2 is a perspective view showing the relationship between the position of a speaker and a microphone and the orientation of a face in an experimental example of the present invention.

FIG. 3 is a diagram showing various examples of a power spectrum ratio in experimental results of the present invention.

FIG. 4 is a conceptual diagram showing a conventional face direction recognition method. References [1] James L. Flanagan. Speech Analysis Synthesis a
nd Perception. Splinger-Verlag, 1972.

Claims (7)

[Claims] 1. A method for recognizing a face direction of a speaker, wherein a pair of microphones for capturing the voice of the speaker is used, and the positions of the microphones projected on a vertical surface facing the speaker are shifted from each other. The difference between the sound powers of the outputs of the pair of microphones is determined, and the speaker is determined in any position direction on a straight line connecting the pair of projected microphones on the vertical surface by the determined sound power difference. A face direction recognizing method characterized in that it is determined whether or not a face is facing. 2. A straight line connecting the positions of the pair of projected microphones on the vertical plane is substantially a horizontal line, and the other pair of microphones capturing the voice of the speaker is used as the other pair of microphones. The straight line connecting the positions projected on the vertical plane facing the speaker is arranged so as to be substantially a vertical line, and the difference between the sound powers of the outputs of the other pair of microphones is obtained. 2. The face direction recognition method according to claim 1, wherein it is determined in which position direction on the vertical line the face of the speaker is facing. 3. The face direction recognition method according to claim 2, wherein one of said pair of microphones and one of said other pair of microphones are common to each other. 4. The face direction recognition method according to claim 1, wherein the determination of the face direction based on the difference in audio power is performed while paying attention to attenuation at a specific frequency. 5. The difference between the audio powers is obtained by calculating the product of the frequency spectrum Px (f) of one output of the pair of microphones and the frequency spectrum Py (f) of the other output as Px
The face direction recognition method according to any one of claims 1 to 4, wherein the value is a value obtained by dividing the square by (f). 6. A device for recognizing a face direction of a speaker, comprising: a pair of microphones for capturing the voice of the speaker; and an acoustic change of voice due to a difference in the face direction from an output of the pair of microphones. A pair of feature amount analyzing means for detecting as a feature amount; a difference extracting means for obtaining a difference between the feature amounts detected by the pair of feature amount analyzing means; and a face direction based on the difference extracted by the difference extracting means. A face direction recognizing device comprising: a discriminant analyzing means for discriminating. 7. The face direction recognition apparatus according to claim 6, further comprising means for restricting detection of the pair of feature amounts to a predetermined frequency band.