JP2003061184A - Information processing apparatus and method, information generating device and method, recording medium and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus that can utilize voice uttered by a user so as to detect a direction of the face of the user. SOLUTION: Microphone amplifiers 2L, 2R amplify sound signal of voice collected by microphones 1L, 1R placed at a prescribed position and uttered by a prescribed sound source, generate processing sound signals L, R and supply them to a detector 3. The detector 3 respectively calculates sound pressures L, R of the processing sound signals L, R and also calculates a sound pressure ratio (= sound pressure R/sound pressure L). The detector 3 detects a direction of the sound source corresponding to the calculated sound pressure ratio on the basis of data denoting a cross-reference between the sound pressure ratio and the direction of the sound source possessed by itself.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an information processing apparatus and method, an information generating apparatus and method, a recording medium, and a program.
The present invention relates to an information processing apparatus and method, an information generating apparatus and method, a recording medium, and a program capable of detecting a position and an orientation of a sound source.

[0002]

2. Description of the Related Art For example, images and sounds in a plurality of remote conference rooms are communicated with each other via a network, and in each conference room, images and sounds in other conference rooms are reproduced, so that the images and sounds are reproduced. There are remote systems that allow meetings to take place as if surrounding a single table.

In this remote conference system, display of video and output of audio are controlled in accordance with the direction of the face of the user participating in the conference. That is, the direction of the user's face is detected.

[0004]

However, a method for detecting the orientation of the user's face using a special sensor or using image data of the user's face has been disclosed. A method for detecting the orientation of the user's face using the voice emitted by the user has not been disclosed yet.

[0005] The present invention has been made in view of such circumstances, and it is an object of the present invention to detect the direction of the face of a sound source using the sound emitted by the sound source.

[0006]

According to the present invention, there is provided an information processing apparatus comprising: a sound source corresponding to a sound generated by a predetermined sound source, which is collected by a plurality of sound collecting devices installed at predetermined positions; And a state detecting means for detecting the state of the sound source based on the characteristic amount calculated by the first calculating means.

The state can be the direction of a sound source at a predetermined position.

[0008] The state detecting means holds correspondence data indicating a correspondence relationship between the characteristic amount and the state, which is obtained in advance, and can detect the state based on the correspondence data and the characteristic amount.

The first calculating means comprises: a first sound signal corresponding to the sound emitted by the sound source, collected by the first sound collecting device; and a sound source collected by the second sound collecting device. The sound pressure ratio between the sound pressure of the first sound signal and the sound pressure of the second sound signal is calculated as a feature amount from the second sound signal corresponding to the sound emitted by the device. Holding the corresponding relationship data indicating the corresponding relationship between the sound pressure ratio and the state,
The state can be detected based on the correspondence data and the sound pressure ratio.

The first calculating means includes a first sound signal corresponding to the sound emitted by the sound source, collected by the first sound collecting device, and a sound source collected by the second sound collecting device. From the second sound signal corresponding to the sound generated by the first sound signal, the sound pressure of the first high-frequency sound signal consisting of the high-frequency part of the first sound signal, and the second high-frequency sound consisting of the high-frequency part of the second sound signal The sound pressure ratio between the sound pressure of the signal and the sound pressure ratio is calculated as a characteristic amount, and the state detection unit holds the previously obtained correspondence data indicating the correspondence between the sound pressure ratio and the state. Based on this, the state can be detected.

The first calculating means includes a first sound signal corresponding to the sound emitted by the sound source, collected by the first sound collecting device, and a sound source collected by the second sound collecting device. The phase difference between the phase of the first audio signal and the phase of the second audio signal is calculated as a feature amount from the second audio signal corresponding to the voice emitted by the device, and the state detection means is obtained in advance. Holds correspondence data indicating the correspondence between the phase difference and the state,
The state can be detected based on the correspondence data and the phase difference.

The first calculating means includes a first sound collecting device, a second sound collecting device,
A sound collector, and a sound collected by a third sound collector,
From the first audio signal, the second audio signal, and the third audio signal corresponding to the audio emitted from the sound source located within the predetermined range, the phases of the first audio signal and the second audio signal are obtained. , And a second phase difference between the phase of the first audio signal and the phase of the third audio signal are calculated as a feature quantity, and the state detecting means calculates the first phase difference and Based on the second phase difference, the position of the sound source on a plane can be detected.

[0013] Sound sources located at a plurality of experimental positions provided within a predetermined range, which are collected by each of the first sound collecting device, the second sound collecting device, and the third sound collecting device, are emitted. From the fourth audio signal, the fifth audio signal, and the sixth audio signal corresponding to the extracted audio, the phase of the fourth audio signal and the fifth
A third phase difference between the phase of the audio signal and the fourth phase difference between the phase of the fourth audio signal and the sixth audio signal are calculated for each experimental position. A second coefficient based on the relationship between the first phase difference and the experimental position, and a second coefficient based on the relationship between the fourth phase difference and the experimental position. Storage means for storing the first coefficient and the second coefficient is further provided; and the state detection means includes a relation between the first phase difference and the first coefficient, and a relation between the second phase difference and the second coefficient. Based on the relationship, the position of the sound source on the plane can be detected.

The first calculating means is configured to calculate a first sound pressure ratio between the sound pressure of the second sound signal and the sound pressure of the third sound signal from the second sound signal and the third sound signal. The amount is further calculated, and the state detection means can detect the direction of the sound source based on the first sound pressure ratio and the detected position of the sound source.

A fourth sound corresponding to a sound emitted by a sound source which is located at a plurality of reference positions and is oriented in a predetermined direction, which is collected by each of the second sound collecting device and the third sound collecting device; A second sound pressure ratio between the sound pressure of the fourth sound signal and the sound pressure of the fifth sound signal is calculated from the signal and the fifth sound signal, and further, a relationship between the second sound pressure ratio and a predetermined direction. A second calculating means for calculating a coefficient based on each of the reference positions,
Storage means for storing the reference position corresponding to the reference position, wherein the state detection means detects a reference position corresponding to the position of the detected sound source, and selects a coefficient corresponding to the detected reference position from the storage means. Then, the direction of the sound source can be detected based on the relationship between the selected coefficient and the first sound pressure ratio.

The first calculating means comprises a first sound collecting device, a second sound collecting device,
A first sound signal and a second sound corresponding to sounds emitted by a sound source located within a predetermined range, which are collected by the sound collecting device, the third sound collecting device, and the fourth sound collecting device. Signal, third
From the audio signal and the fourth audio signal, a first phase difference between the phase of the first audio signal and the phase of the second audio signal, the phase of the second audio signal, and the phase of the third audio signal , And a third phase difference between the phase of the third audio signal and the phase of the fourth audio signal are calculated as a feature quantity, and the state detection unit calculates the first phase difference, A second phase difference, and a third
Based on the phase difference of the sound source, the position of the sound source in space can be detected.

A plurality of experiments provided within a predetermined range and collected by each of the first sound collecting device, the second sound collecting device, the third sound collecting device, and the fourth sound collecting device. From the fifth audio signal, the sixth audio signal, the seventh audio signal, and the eighth audio signal corresponding to the audio emitted by the sound source located at the position, the phase of the fifth audio signal and the sixth audio signal A fourth phase difference from the signal phase, a fifth phase difference between the sixth audio signal phase and the seventh audio signal phase, and a seventh audio signal phase and an eighth audio signal phase Is calculated for each experimental position, a first coefficient based on the relationship between the fourth phase difference and the experimental position, and a sixth coefficient based on the relationship between the fifth phase difference and the experimental position. A second calculating means for calculating a second coefficient and a third coefficient based on a relationship between the sixth phase difference and the experimental position; and a second calculating means. First coefficient calculated by the second
And a storage means for storing a third coefficient and a third coefficient. The state detection means includes a relation between the first phase difference and the first coefficient, and a relation between the second phase difference and the second coefficient. The position of the sound source in space can be detected based on the relationship and the relationship between the third phase difference and the third coefficient.

The first calculating means comprises a first audio signal, a second audio signal,
The first sound pressure ratio between the sound pressure of the first sound signal and the sound pressure of the second sound signal, and the sound of the second sound signal from the sound signal, the third sound signal, and the fourth sound signal. The second sound pressure ratio between the pressure and the sound pressure of the third sound signal, and the third sound pressure ratio between the sound pressure of the third sound signal and the sound pressure of the fourth sound signal are further defined as characteristic amounts. The state detection means calculates the first sound pressure ratio,
The direction of the sound source can be detected based on the second sound pressure ratio, the third sound pressure ratio, and the detected position of the sound source.

The sound pickup device is located at a plurality of reference positions, each of which is collected by each of the first sound collection device, the second sound collection device, the third sound collection device, and the fourth sound collection device, in a predetermined direction. From the fifth audio signal, the sixth audio signal, the seventh audio signal, and the eighth audio signal corresponding to the audio emitted by the sound source directed to
The fourth of the sound pressure of the fifth sound signal and the sound pressure of the sixth sound signal
Sound pressure ratio, a fifth sound pressure ratio between the sound pressure of the sixth sound signal and the sound pressure of the seventh sound signal, and a fifth sound pressure ratio between the sound pressure of the seventh sound signal and the sound pressure of the eighth sound signal. And second storage means for storing the fourth sound pressure ratio, the fifth sound pressure ratio, the sixth sound pressure ratio, and the direction corresponding to the reference position. The state detecting means is further provided with a reference position corresponding to the position of the detected sound source, and a fourth sound pressure ratio corresponding to the first sound pressure ratio, the second sound pressure ratio, and the third sound pressure ratio. , The direction corresponding to the fifth sound pressure ratio and the sixth sound pressure ratio can be detected as the direction of the sound source.

The information processing method according to the present invention is characterized in that a sound signal corresponding to the state of a sound source is obtained from a sound signal corresponding to a sound emitted from a predetermined sound source, which is collected by a plurality of sound collecting devices installed at predetermined positions. It is characterized by including a calculating step of calculating the amount and a state detecting step of detecting the state of the sound source based on the characteristic amount calculated in the processing of the calculating step.

[0021] The program of the first recording medium of the present invention comprises:
From sound signals corresponding to sounds emitted by a predetermined sound source, collected by a plurality of sound collection devices installed at predetermined positions,
The method includes a calculating step of calculating a feature amount corresponding to a state of the sound source, and a state detecting step of detecting a state of the sound source based on the feature amount calculated in the processing of the calculating step.

A first program according to the present invention corresponds to a state of a sound source from an audio signal corresponding to a sound emitted by a predetermined sound source, which is collected by a plurality of sound collection devices installed at predetermined positions. It is characterized by including a calculation step of calculating a feature amount, and a state detection step of detecting a state of the sound source based on the feature amount calculated in the processing of the calculation step.

[0023] In the information processing apparatus and method and the first program according to the present invention, an audio signal corresponding to a sound emitted from a predetermined sound source, collected by a plurality of sound collection devices installed at predetermined positions. Then, the feature amount corresponding to the state of the sound source is calculated, and the state of the sound source is detected based on the calculated feature amount.

The information generating apparatus according to the present invention comprises a plurality of sound collecting units provided within a predetermined range and collected by each of the first sound collecting unit, the second sound collecting unit, and the third sound collecting unit. From the fourth audio signal, the fifth audio signal, and the sixth audio signal corresponding to the audio emitted by the sound source located at the experiment position,
The third of the phase of the fourth audio signal and the phase of the fifth audio signal
A first calculating means for calculating, for each experimental position, a phase difference between the third phase difference and the experimental position, and a fourth phase difference between the fourth audio signal and the sixth audio signal. A second coefficient calculating means for calculating a first coefficient based on the relation and calculating a second coefficient based on a relation between the fourth phase difference and the experimental position.

The detection device includes a seventh sound signal and a seventh sound signal corresponding to sounds emitted by sound sources located at a plurality of reference positions, which are collected by the second sound collection device and the third sound collection device, respectively. The sound pressure ratio between the sound pressure of the seventh sound signal and the sound pressure of the eighth sound signal is calculated from the sound signal of No. 8, and the direction of the sound source is further determined based on the relationship between the sound pressure ratio and the third coefficient. The ninth sound signal and the tenth sound corresponding to the sounds emitted from the sound sources located at the plurality of reference positions, which are detected and collected by the second sound collecting device and the third sound collecting device, respectively. Third calculating means for calculating the sound pressure ratio between the sound pressure of the ninth sound signal and the sound pressure of the tenth sound signal from the signal, and the sound pressure of the ninth sound signal and the sound pressure of the tenth sound signal And a fourth calculating means for calculating a third coefficient based on a relationship between the sound pressure ratio of the first position and the experimental position. It can be provided.

[0026] The information generating method of the present invention comprises the steps of: collecting a plurality of sounds provided by a first sound collecting device, a second sound collecting device, and a third sound collecting device and provided in a predetermined range; From the fourth audio signal, the fifth audio signal, and the sixth audio signal corresponding to the audio emitted by the sound source located at the experiment position,
The third of the phase of the fourth audio signal and the phase of the fifth audio signal
A first calculating step of calculating, for each experimental position, a phase difference between the third phase difference and the experimental position, and a fourth phase difference between the fourth audio signal and the sixth audio signal. A second calculating step of calculating a first coefficient based on the relationship and calculating a second coefficient based on a relationship between the fourth phase difference and the experimental position.

The program of the second recording medium of the present invention comprises:
The sound collected by each of the first sound collecting device, the second sound collecting device, and the third sound collecting device, and emitted from the sound sources located at a plurality of experimental positions provided within a predetermined range. A third phase difference between the phase of the fourth audio signal and the phase of the fifth audio signal, and the fourth audio from the corresponding fourth, fifth, and sixth audio signals A first calculating step of calculating a fourth phase difference between the signal phase and the sixth audio signal for each experimental position, and a first coefficient based on a relationship between the third phase difference and the experimental position. And a second calculating step of calculating a second coefficient based on the relationship between the fourth phase difference and the experimental position.

The second program according to the present invention includes a plurality of programs provided within a predetermined range and collected by each of the first sound collecting device, the second sound collecting device, and the third sound collecting device. From the fourth, fifth, and sixth audio signals corresponding to the audio emitted by the sound source located at the experimental position, the phase of the fourth audio signal and the phase of the fifth audio signal are calculated. And the fourth phase difference between the phase of the fourth audio signal and the sixth audio signal is calculated for each experimental position.
And the first coefficient is calculated based on the relationship between the third phase difference and the experimental position, and the second coefficient is calculated based on the relationship between the fourth phase difference and the experimental position. The program is characterized by causing a computer to execute a process including a second calculating step.

According to the information generating apparatus and method of the present invention, and the second program, the predetermined sound collected by each of the first sound collecting device, the second sound collecting device, and the third sound collecting device. From the fourth, fifth, and sixth audio signals corresponding to the sounds emitted by the sound sources located at the plurality of experimental positions provided in the range of the fourth audio signal, A third phase difference from the phase of the fifth audio signal,
And the fourth of the phase of the fourth audio signal and the sixth audio signal
Is calculated for each experimental position, a first coefficient is calculated based on the relationship between the third phase difference and the experimental position, and based on the relationship between the fourth phase difference and the experimental position, A second coefficient is calculated.

[0030]

BEST MODE FOR CARRYING OUT THE INVENTION

FIG. 1 shows a configuration example of a first embodiment of a sound source state detection system to which the present invention is applied. In this system, the direction of the sound source is detected from the sound emitted by the sound source at a predetermined position. Here, the sound source is the mouth of the user at a predetermined position, and the direction of the mouth of the user (the direction of the face of the user) is determined based on the voice uttered by the user.
Is detected.

As shown in FIG. 2A, when the center point of the user's head is located at position A, the microphone (hereinafter referred to as microphone) 1L and microphone 1R Is located at a position where the distance L between the microphone 1L and the mouth (sound source) of the user and the distance R between the microphone R and the sound source are equal to each other.

Returning to FIG. 1, the microphone 1L and the microphone 1R
Collects the sound emitted from the mouth of the user, which is the sound source, and collects the microphone amplifier 2L and the microphone amplifier 2L.
Supply to R.

The microphone amplifier 2L amplifies the audio signal from the microphone 1L, generates a processing audio signal L, and supplies it to the detection device 3. The microphone amplifier 2R amplifies the audio signal from the microphone 1R, generates a processing audio signal R, and supplies the processing audio signal R to the detection device 3.

From the processing audio signal L supplied from the microphone amplifier 2L and the processing audio signal R supplied from the microphone amplifier 2R, the detection device 3 determines the sound pressure L of the processing audio signal L and the processing audio signal R The sound pressure ratio RL (= sound pressure R / sound pressure L) to the sound pressure R is calculated.

The detection device 3 determines the direction of the sound source corresponding to the calculated sound pressure ratio RL, that is, in this example, the direction of the mouth of the user (the direction of the face of the user) to the sound pressure ratio RL shown in FIG. Detection is performed by referring to data indicating a correspondence relationship with the direction of the mouth (sound source), and the detection is output to an external device (not shown).

By the way, since the sound pressure of the voice increases as the distance from the sound source decreases, as shown in FIG. 2C, the face (mouth) of the user faces rightward with respect to the front direction. If the distance R is shorter than the distance L, the sound pressure detected by the microphone 1R and detected by the user (sound pressure R of the processing sound signal R) is:
The sound pressure is higher than the sound pressure of the sound emitted by the user (sound pressure L of the processing sound signal L) collected by the microphone 1L. That is, in this case, the sound pressure ratio RL becomes a predetermined value larger than 1.

Further, as shown in FIG. 2B, when the mouth is directed to the left with respect to the front direction and the distance R is longer than the distance L, the sound pressure R becomes smaller than the sound pressure L. , The sound pressure ratio RL is a predetermined value smaller than 1.

Also, as shown in FIG. 2A, when the mouth is in the front direction and the distance R is equal to the distance L,
Since the sound pressure R is equal to the sound pressure L, the sound pressure ratio RL is 1
It becomes.

As described above, the sound pressure ratio RL (= sound pressure R / sound pressure L) corresponds to the direction of the mouth (sound source) (more precisely, the distance between the mouth and the microphone 1). The data shown in FIG. 3 is obtained by changing the heading direction and summing up the sound pressure ratio RL of the sound emitted at that time and the value indicating the heading direction.

FIG. 4 shows an example of the configuration of the detection device 3.

The input section 11 inputs the processing audio signal L and the processing audio signal R appropriately supplied from each of the microphone amplifier 2L and the microphone amplifier 2R. Input unit 1
Reference numeral 1 designates a part of the input processing audio signals L and R (for example, a part input within a predetermined time) as a feature value calculation unit 1.
Feed to 2.

The feature quantity calculating section 12 converts the sound pressures L, R from the processing sound signals L, R supplied from the input section 11 into respective sound pressures L, R.
R is calculated, and a sound pressure ratio RL (= sound pressure R / sound pressure L) is calculated and supplied to the detection unit 13. The detection unit 13 holds data indicating the correspondence between the sound pressure ratio RL and the mouth (sound source) shown in FIG. 3, and based on the data, the sound pressure ratio RL supplied from the feature amount calculation unit 12. Is detected, and supplied to the output unit 14.

The output unit 14 supplies the direction of the mouth (sound source) supplied from the detection unit 13 to an external device.

FIG. 5 shows an example of the configuration of the feature amount calculation unit 12. Processing audio signal L supplied from input unit 11
Is supplied to the sound pressure calculation unit 21L, and the processing audio signal R
Are supplied to the sound pressure calculation unit 21R.

The sound pressure calculating section 21L calculates, for example, the mean square value, the square root of the mean square, or the average value of the absolute values of the values indicated by the input processing audio signal L, Pressure L, and the result of the calculation is used as the sound pressure ratio calculation unit 22.
To supply. The sound pressure calculation unit 21R calculates the sound pressure R of the processing audio signal R in the same manner as the sound pressure calculation unit 21L, and supplies the sound pressure R to the sound pressure ratio calculation unit 22.

The sound pressure ratio calculator 22 calculates the sound pressure ratio RL (= sound pressure R / sound pressure L) based on the sound pressure L from the sound pressure calculator 21L and the sound pressure R from the sound pressure calculator 21R. It calculates and supplies the calculation result to the detection unit 13.

Next, the operation of the sound source state detection system according to the first embodiment will be described with reference to the flowchart of FIG.

In step S1, the microphones 1L, 1R
Each collects the sound (from the lips facing the predetermined direction) emitted by the user facing the predetermined direction,
The resulting audio signal is transmitted to microphone amplifiers 2L, 2R
To supply.

In step S2, the microphone amplifier 2
Each of L and 2R amplifies the audio signal supplied from the microphones 1L and 1R, generates processing audio signals L and R, and supplies the processed audio signals L and R to the detection device 3.

In step S3, the detection device 3 detects the direction of the mouth (sound source) based on the processing audio signals L and R supplied from the microphone amplifiers 2L and 2R, respectively. Details of the processing here are shown in the flowchart of FIG.

In step S11, the input unit 11 of the detecting device 3 outputs a part of the input processing audio signals L and R from the microphone amplifiers 2L and 2R (portion input within a predetermined time). It is supplied to the quantity calculation unit 12.

Next, in step S12, the characteristic amount calculation unit 12 (sound pressure calculation units 21L and 21R)
From the processing audio signals L and R supplied from the computer, respectively.

In step S13, the characteristic amount calculation unit 1
2 (sound pressure ratio calculation unit 22) calculates a sound pressure ratio RL (= sound pressure R / sound pressure L) based on the sound pressures L and R calculated in step S <b> 12, and supplies the calculated sound pressure ratio RL to the detection unit 13.

Next, in step S14, the detecting unit 1
3 detects the direction of the sound source corresponding to the sound pressure ratio RL supplied from the feature amount calculation unit 12 with reference to the data (FIG. 3) indicating the correspondence between the sound pressure ratio RL and the direction of the mouth (sound source). ,
It is supplied to the output unit 14.

In step S15, the output unit 14
The direction of the mouth (sound source) supplied from the detection unit 13 is output to an external device.

Thereafter, the processing of the detection device 3 ends, and the operation of the sound source state detection system (FIG. 6) also ends.

FIG. 8 shows a configuration example of a sound source state detection system according to a second embodiment of the present invention. In this system, as in the case of the first embodiment (FIG. 1), the direction of the sound source is detected from the sound emitted by the sound source at a predetermined position.

This system includes a detecting device 3 shown in FIG.
, A detection device 31 is provided. Other configurations are the same as those in FIG. 1, and the description thereof will be omitted as appropriate.

The microphone 1L and the microphone 1R are installed at the same positions as in the first embodiment (FIG. 9).

The detecting device 31 extracts a high-frequency portion (hereinafter referred to as a high-frequency processing audio signal) of the processing audio signal L from the processing audio signal L supplied from the microphone amplifier 2L and the processing audio signal R supplied from the microphone amplifier 2R. A sound pressure ratio RLh (= sound pressure Rh / sound pressure Hh) between a sound pressure Lh of the signal Lh) and a sound pressure Rh of a high-frequency portion of the processing sound signal R (hereinafter, referred to as a high-frequency processing sound signal Rh). Is calculated.

The detecting device 31 detects the direction of the mouth (sound source) corresponding to the calculated sound pressure ratio RLh based on the data shown in FIG. 10 indicating the correspondence between the sound pressure ratio RLh and the direction of the mouth. Is output to an external device (not shown).

By the way, since sound, especially its high frequency part has directivity, the sound pressure of the high frequency part becomes larger as the angle with respect to the sound source becomes smaller. From this, as shown in FIG. 9C, when the mouth faces rightward with respect to the front direction and the angle R is smaller than the angle L, sound is collected by the microphone 1R detected at this time. The sound pressure of the high-frequency part of the sound emitted by the user (the sound pressure Rh of the sound signal Rh for high-frequency processing) is collected by the microphone 1L, It becomes larger than the sound pressure Lh) of the signal Lh. That is, in this case, the sound pressure ratio RLh is a predetermined value larger than 1.

As shown in FIG. 9 (B), when the mouth is directed to the left with respect to the front direction and the angle R is larger than the angle L, the sound pressure Rh becomes smaller than the sound pressure Lh. Therefore, the sound pressure ratio RLh becomes a predetermined value smaller than 1.

Further, as shown in FIG. 9A, when the mouth is directed to the front and the angle R is equal to the angle L,
Since the sound pressure Rh is equal to the sound pressure Lh, the sound pressure ratio RLh
Becomes 1.

As described above, the sound pressure ratio RLh corresponds to the direction of the mouth (sound source) (more precisely, the angle between the direction in which the mouth is facing and the direction in which the microphone 1 is located from the position of the mouth). The direction shown in FIG. 10 is obtained by changing the direction in which the lips face and changing the sound pressure ratio RLh of the sound emitted at that time and the value indicating the direction in which the lips face.

FIG. 11 shows a configuration example of the detection device 31. The detection device 31 includes a feature value calculation unit 41 and a detection unit 42 instead of the feature value calculation unit 12 and the detection unit 13 of the detection device 3 in FIG. The other part is
This is the same as in FIG.

The feature value calculating section 41 outputs the high-frequency processing audio signal Lh and the high-frequency processing audio signal Rh obtained by passing only the high-frequency portions of the processing audio signals L and R supplied from the input section 11. From these, the sound pressure Lh and the sound pressure Rh are calculated respectively, and the calculated sound pressure Lh
And a sound pressure ratio RLh (= sound pressure Rh / sound pressure Lh) of the sound pressure Rh and the sound pressure Rh.

The detector 42 detects the sound pressure ratio R shown in FIG.
It holds data indicating the correspondence between Lh and the direction of the mouth (sound source), and based on the data, the feature amount calculation unit 4
The direction of the mouth corresponding to the sound pressure ratio RLh supplied from 1 is detected and supplied to the output unit 14.

FIG. 12 shows an example of the configuration of the feature amount calculator 41. In the feature amount calculation unit 41, a high-pass filter (HPF) 51L and a high-pass filter 51R are provided before the sound pressure calculation unit 21L and the sound pressure calculation unit 21R of the feature amount calculation unit 12 in FIG. Other configurations are the same as those in FIG.

The high-pass filter 51L passes only the high-frequency portion of the processing audio signal L supplied from the input unit 11 and supplies it to the sound pressure calculation unit 21L. That is, the high-frequency processing audio signal Lh is supplied to the sound pressure calculation unit 21L.

The high-pass filter 51R allows only the high-frequency portion of the processing audio signal R supplied from the input unit 11 to pass and supplies it to the sound pressure calculation unit 21R. That is, the high-frequency processing audio signal Rh is supplied to the sound pressure calculation unit 21R.

FIG. 13 shows the cut-off characteristic of the high-pass filter 51. The cutoff frequency f can be, for example, a fixed value or a frequency proportional to the pitch frequency of the user's voice.

Here, the pitch frequency is the reciprocal of the period T of the basic waveform repeated in the waveform of the user's voice as shown in FIG. The pitch frequency is different for each person.

Returning to FIG. 12, the sound pressure calculator 21L calculates the sound pressure Lh of the high-frequency processing sound signal Lh supplied from the high-pass filter 51L, and supplies the sound pressure Lh to the sound pressure ratio calculator 22. The sound pressure calculation unit 21R calculates the sound pressure Rh of the high-frequency processing audio signal Rh supplied from the high-pass filter 51R, and supplies the sound pressure Rh to the sound pressure ratio calculation unit 22.

The sound pressure ratio calculator 22 calculates a sound pressure ratio RLh (= sound pressure Rh / sound pressure Lh) between the sound pressure Lh from the sound pressure calculator 21L and the sound pressure Rh from the sound pressure calculator 21R. , Detection unit 4
Feed to 2.

Next, the operation of the sound source state detection system according to the second embodiment will be described with reference to the flowchart of FIG.

In steps S21 and S22, FIG.
Since the same processes as those in steps S1 and S2 are performed, description thereof will be omitted.

In step S23, the detecting device 31
Detects the direction of the mouth (sound source) based on the processing audio signals L and R supplied from the microphone amplifiers 2L and 2R, respectively. Details of the processing here are shown in the flowchart of FIG.

In step S31, the input unit 11
A part of the processing audio signals L and R from each of the microphone amplifiers 2L and 2R (for example, a part input within a predetermined time) is supplied to the feature amount calculation unit 41.

Next, in step S 32, the characteristic amount calculating section 41 (high-pass filters 51 L, 51 R) filters the high-frequency portions of the processing audio signals L, R supplied from the input section 11, and Lh and Rh are generated.

In step S33, the characteristic amount calculating section 4
1 (sound pressure calculation units 21L and 21R) calculates the sound pressure L from the high-frequency processing sound signals Lh and Rh generated in step S32.
h and Rh are calculated respectively.

Next, in step S34, the characteristic amount calculation unit 41 (sound pressure ratio calculation unit 22) uses the sound pressure ratio RLh (= sound pressure Rh / sound pressure Lh) based on the sound pressures Lh and Rh calculated in step S33. ) Is calculated and supplied to the detection unit 42.

In step S35, the detection unit 42
With reference to data (FIG. 10) indicating the correspondence between the sound pressure ratio RLh and the direction of the mouth (sound source), the direction of the sound source corresponding to the sound pressure ratio RLh supplied from the feature amount calculation unit 41 is detected.
It is supplied to the output unit 14.

Next, in step S36, the output unit 1
4 outputs the mouth (sound source) supplied from the detection unit 42 to an external device.

Thereafter, the processing of the detection device 31 ends, and the operation of the sound source state detection system (FIG. 15) also ends.

FIG. 17 shows a configuration example of a third embodiment of a sound source state detection system to which the present invention is applied. In this system, the first embodiment (FIG. 1) and the second embodiment
As in the case of the embodiment (FIG. 8), the direction of the sound source is detected from the sound emitted by the sound source at the predetermined position.

This system includes the detecting device 3 shown in FIG.
, A detection device 61 is provided. The same applies to other configurations. (4) The microphone 1L and the microphone 1R are installed at the same positions as those in the first embodiment (FIG. 2).

The detection device 61 determines the phase of the processing audio signal L with respect to the phase of the processing audio signal R from the processing audio signal L supplied from the microphone amplifier 2L and the processing audio signal R supplied from the microphone amplifier 2R. Deviation (phase difference)
Is calculated. The detection device 61 detects the direction of the user's lip (sound source) corresponding to the calculated phase difference based on the data indicating the correspondence between the phase difference and the direction of the lip (sound source) shown in FIG. Output to an external device not shown.

By the way, the longer the distance between the microphone 1 and the sound source, the longer it takes to reach the microphone 1, and the phase of the sound that arrives later is smaller than the phase of the sound that arrives first. There is a delay feature. From this, for example, as shown in FIG. 2 (B), when the mouth is directed to the left with respect to the front direction and the distance R is longer than the distance L,
At this time, since the sound emitted from the user reaches the microphone 1L first and then reaches the microphone 1R, the phase of the sound (processing sound signal R) collected by the microphone 1R detected at this time (see FIG. 19 (B)) is the phase of the sound (processing sound signal L) collected by the microphone 1L (FIG. 19).
(A)). That is, in this case, the difference between the phase of the processing audio signal R and the phase of the processing audio signal R is
This will be a negative value.

At this time, since the distance L is shorter than the distance R, the level of the processing audio signal L shown in FIG. 19A is higher than the level of the processing audio signal R shown in FIG. 19B. .

As shown in FIG. 2 (C), when the mouth is directed to the right with respect to the front direction and the distance R is shorter than the distance L, the voice uttered by the user at this time is the microphone 1R , And then reaches the microphone 1L, so that the phase of the processing audio signal R (FIG. 20B) advances as compared with the phase of the processing audio signal L (FIG. 20A). That is, in this case, the difference between the phase of the processing audio signal R and the phase of the processing audio signal L is a positive value. At this time, since the distance R is shorter than the distance L, the level of the processing audio signal R shown in FIG. 20B is higher than the level of the processing audio signal L shown in FIG.

As shown in FIG. 2A, when the user's face is facing the front and the distance R is equal to the distance L, the voice uttered by the user at this time is
R and the microphone 1L at the same time, so that the processing audio signal R
(FIG. 21B) and the processing audio signal L (FIG. 21B).
The phases of (A)) match. That is, in this case, the phase difference between the phase of the processing audio signal L and the phase of the processing audio signal L is zero.

At this time, since the distance R is equal to the distance L, the level of the processing audio signal R shown in FIG. 21B and the level of the processing audio signal L shown in FIG.

As described above, since the phase difference corresponds to the direction of the mouth (sound source), the direction of the mouth is changed, and the phase difference of the sound emitted at that time and the value indicating the direction of the mouth are totaled. Thus, the data shown in FIG. 18 is obtained.

FIG. 22 shows a configuration example of the detection device 61. The detection device 61 includes a feature value calculation unit 71 and a detection unit 72 instead of the feature value calculation unit 12 and the detection unit 13 of the detection device 3 in FIG. Other configurations are
This is the same as in FIG.

The feature quantity calculating section 71 calculates a phase difference between the phase of the processing audio signal R and the phase of the processing audio signal R from the processing audio signal L and the processing audio signal R supplied from the input section 11. I do.

More specifically, the feature quantity calculating section 71 samples and quantizes the processing audio signals L and R at a predetermined sampling period T. Thus, the processing audio signals L, N samples L ₁ through L _N is, from the processing speech signals R, N samples R ₁ through R _N are obtained, respectively. The subscript number given to the sample L or the sample R indicates the sampling order. That is, samples L ₁ represents the first sampled samples (first-th sample).

Next, the feature quantity calculation unit 71 determines that j is 1 to n.
Is calculated, and the value m of j at which the value of the calculation result is maximum is detected. Then, the feature amount calculating unit 71 sets a value (time) obtained by multiplying the detected value m by the sampling period T as a phase difference between the phase of the processing audio signal R and the phase of the processing audio signal L.

(Equation 1)

[0100] In the formula, Li-j indicates the value of the sample L _ij, Ri indicates the value of the sample R _i.

That is, the correlation value between the processing audio signal R and the processing audio signal L is such that the processing audio signal L is equal to the processing audio signal R for one sampling time up to the time of n samples. Are calculated at the same time, and the time lag when the highest correlation value is obtained in the calculation result is
The difference is the difference between the phase of the processing audio signal R and the phase of the processing audio signal L.

The detecting section 72 holds data indicating the correspondence between the phase difference and the direction of the mouth (sound source) shown in FIG. 18, and is supplied from the feature amount calculating section 71 based on the data. The direction of the sound source corresponding to the detected phase difference is detected and supplied to the output unit 14.

Next, the operation of the sound source state detection system according to the third embodiment will be described with reference to the flowchart in FIG.

In steps S41 and S42, FIG.
Since the same processes as those in steps S1 and S2 are performed, description thereof will be omitted.

In step S43, the detecting device 61
Detects the direction of the mouth (sound source) based on the processing audio signals L and R supplied from the microphone amplifiers 2L and 2R, respectively. Details of the processing here are shown in the flowchart of FIG.

In step S51, the input unit 11
A part of the processing audio signals L and R from each of the microphone amplifiers 2L and 2R (for example, a part input within a predetermined time) is supplied to the feature amount calculation unit 71.

Next, in step S 52, the feature quantity calculating section 71 calculates the position of the phase of the processing audio signal L with respect to the phase of the processing audio signal R from the processing audio signals L and R supplied from the input section 11. The phase difference is calculated and supplied to the detection unit 72.

In step S53, the detection unit 72
With reference to the data (FIG. 18) indicating the correspondence between the phase difference and the direction of the mouth (sound source), the direction of the sound source corresponding to the phase difference supplied from the feature amount calculation unit 71 is detected. Supply.

In step S54, the output unit 14
The direction of the mouth (sound source) supplied from the detection unit 72 is output to an external device.

Thereafter, the processing of the detection device 61 ends, and the operation of the sound source state detection system (FIG. 23) also ends.

FIG. 25 shows a configuration example of a sound source state detection system according to a fourth embodiment of the present invention. In the first to third embodiments, the direction of the sound source at a predetermined position is detected on the assumption that the sound source does not move. However, in this system, the sound source moves within a predetermined range. The position of the sound source and its direction are detected.

In this system, the detecting device 3 shown in FIG.
Instead of the detection device 101 is provided,
Microphone 1L, microphone 1C, and microphone 1R, three microphones 1, microphone amplifier 2L for amplifying audio collected by microphone 1L, microphone amplifier 2C for amplifying audio collected by microphone 1C, and microphone 1R The three microphone amplifiers 2 of the microphone amplifier 2R for amplifying the sound collected by the microphone are provided.

As shown in FIG. 26, for example, when the center of the user's head is located at position A and the mouth is facing the front as shown in FIG. The distance L and the distance R between the microphone 1R and the mouth are set to be equal to each other.
The microphone 1C is installed at a position away from the position A by a predetermined distance in the front direction.

Returning to FIG. 25, the detection apparatus 101 detects the phase of the processing audio signal L supplied from the microphone amplifier 2L with respect to the phase of the signal supplied from the microphone amplifier 2C (hereinafter referred to as the processing audio signal C). Within a predetermined range based on the phase difference CL of the phase of the processing audio signal R supplied from the microphone amplifier 2R with respect to the phase difference CL of the phase and the phase of the processing audio signal C, and a predetermined coefficient (described later). The position of the moving mouth (sound source) is detected.

In the case of this example, as shown in FIG. 27, when the center of the user's head is at position A, as shown in FIG. ) (Hereinafter referred to as a reference position) as an origin (x, y).

The detection apparatus 101 also generates a predetermined coefficient (described later) corresponding to the detected position of the mouth (sound source), and the sound pressure L of the processing audio signal L and the sound pressure R of the processing audio signal R. The direction of the sound source is detected based on the pressure ratio RL (= sound pressure R / sound pressure L).

In the case of this example, the direction of the sound source is indicated by an angle θ with respect to the Y axis as shown in FIG.

FIG. 28 shows a configuration example of the detection device 101.

The detection device 101 is provided with a feature value calculation unit 111 and a detection unit 112 instead of the feature value calculation unit 12 and the detection unit 13 of the detection device 3 in FIG.
Other configurations are the same as those in FIG.

The input section 11 includes microphone amplifiers 2L, 2C,
2R, the processing audio signals L,
C and R are input and supplied to the feature amount calculation unit 111.

The characteristic amount calculating section 111 converts the phase difference CL (processing audio signal C) from the processing audio signal L, the processing audio signal C, and the processing audio signal R supplied from the input section 11.
, And a phase difference CR (a phase difference between the phase of the processing audio signal R and the phase of the processing audio signal C).

The feature quantity calculation unit 111 also calculates a sound pressure ratio R between the sound pressure L of the processing audio signal L and the sound pressure R of the processing audio signal R.
L (= sound pressure R / sound pressure L) is calculated.

The characteristic amount calculation section 111 outputs the calculated phase difference CL and phase difference CR, and the sound pressure ratio RL to the detection section 112.

The detection unit 112 is configured to move within a predetermined range (sound source) based on the phase difference CL and the phase difference CR from the feature amount calculation unit 111 and a predetermined coefficient (described later).
Detect the position of.

The detecting unit 112 also determines a predetermined coefficient (described later) corresponding to the detected position of the mouth (sound source) (more precisely,
The direction of the sound source is detected based on the detected position of the sound source and the sound pressure ratio RL from the feature amount calculation unit 111.

The detecting section 112 supplies the detected position of the mouth and its orientation to the output section 14.

FIG. 29 shows an example of the configuration of the feature amount calculation unit 111.

The phase difference calculator 121 calculates a phase difference CL (a phase difference between the phase of the processing audio signal C and the phase of the processing audio signal L).

More specifically, the phase difference calculator 121 samples and quantizes the processing audio signals L and C at a predetermined sampling period T. Thus, N samples L _{1 to} L _N are obtained from the processing audio signal L, and N samples C _{1 to} C _N are obtained from the processing audio signal C.

Next, the phase difference calculation section 121 calculates the equation (2) when j is 1 to n, and detects the value m of j at which the value of the calculation result is the maximum. Then, the phase difference calculation unit 121 multiplies the detected value m by the sampling period T, and uses the multiplication result (time) as a phase difference CL.
Output to the detection unit 112.

(Equation 2)

That is, the correlation value between the processing audio signal L and the processing audio signal C is such that the processing audio signal L is equal to the processing audio signal C by one sampling time up to the time of n samples. The time (value m × sampling period T) when the highest correlation value is obtained in the calculation result is defined as the phase difference CL.

The phase difference calculation section 122 calculates a phase difference CR (a phase difference between the phase of the processing audio signal C and the phase of the processing audio signal R).

More specifically, the phase difference calculator 122 samples and quantizes the processing audio signals C and R at a predetermined sampling period T. Thus, from the processing speech signals R, N samples R ₁ through R _N are from processing audio signals C, N samples C ₁ to C _N are obtained, respectively.

Next, the phase difference calculation unit 122 calculates the equation (3) when j is 1 to n, and detects the value m of j that maximizes the calculation result. Then, the phase difference calculation unit 122 multiplies the detected value m by the sampling period T, and uses the multiplication result (time) as a phase difference CR.
Output to the detection unit 112.

[Equation 3]

That is, the correlation value between the processing audio signal R and the processing audio signal C is such that the processing audio signal R is different from the processing audio signal C by one sampling time up to the time of n samples. The time (value m × sampling period T) when the highest correlation value is obtained in the calculation result is defined as the phase difference CR.

The sound pressure calculating section 123 calculates the mean square value, the square root of the square mean, or the average value of the absolute values of the values indicated by the input processing audio signal L, and calculates the sound pressure L of the processing audio signal L. And the calculation result is supplied to the sound pressure ratio calculation unit 125. The sound pressure calculator 124 calculates the sound pressure R of the processing audio signal R in the same manner as the sound pressure calculator 123 and supplies the sound pressure R to the sound pressure ratio calculator 125.

The sound pressure ratio calculating section 125 includes a sound pressure calculating section 123
The sound pressure ratio RL (= sound pressure R / sound pressure L) is calculated based on the sound pressure L from the sound pressure calculation unit 124 and the sound pressure R from the sound pressure calculation unit 124, and the calculation result is supplied to the detection unit 112.

FIG. 30 shows a configuration example of the detection unit 112.

The position calculating section 131 is provided with a feature amount calculating section 111.
(Phase difference CL and phase difference CR from (Phase difference calculation units 121 and 122), and primary of x and y consisting of coefficient a _L , coefficient b _L , and coefficient c _L stored in the position coefficient storage unit 132 Solving the equation (Equation (4)) and the linear equation (Equation (5)) of x and y composed of coefficient a _R , coefficient b _R , and coefficient c _R to calculate coordinates (x, y) (FIG. 27) I do. a _L × x + b _L × y + c _L = phase difference CL (4) a _R × x + b _R × y + c _R = phase difference CR (5)

The position calculation section 131 converts the calculated position (coordinates (x, y)) into the direction coefficient determination section 133 and the output section 1.
4

By the way, the distance C between the microphone 1C and the mouth (sound source), the distance L between the microphone 1L and the sound source, and the distance between the microphone 1R and the microphone 1R
And the distance R between the sound source and the sound source can be expressed as shown in Expression (6). Distance L- distance C = delta _L distance R- distance C = Δ _R ... (6)

[0142] delta _L and delta _R, as shown in FIG. 31, X
In a coordinate space consisting of an axis, a Y axis, and a distance axis,
It is represented by a gentle curved surface. That is, the sound source is, when moving within relatively close range from the origin, the delta _L and delta _R,
It is possible to approximate the substantially planar, delta _L and delta
_R can be represented by a linear equation of x and y, as shown in equation (7). _{a 'L × x + b'} L × y + c 'L = Δ L a' R × x + b 'R × y + c' R = Δ R ... (7)

On the other hand, as described above, the longer the distance from the sound source is, the longer it takes to transmit the voice. Therefore, for example, as shown in FIG. Since the distance C is longer than the distance L, the sound from the sound source reaches the microphone 1C later than the microphone 1L.

Also, since the phase of a voice arriving late is later than the phase of a voice arriving earlier, that is, the phase of a voice arriving earlier advances compared to the phase of a voice arriving later, for example, In the case of the example of FIG. 32, the phase of the processing audio signal L is, as shown in FIG.
The phase advances from the phase of the processing audio signal C shown in FIG. The phase shift at this time is the phase difference CL.

On the other hand, when the distance C is longer than the distance R, the sound from the sound source reaches the microphone 1R earlier than the microphone 1C, so that the phase of the processing sound signal R is as shown in FIG. , From the phase of the processing audio signal C. The phase shift at this time is the phase difference CR.

As described above, the phase difference CL and the phase difference CR
Corresponds to the distance L, the distance C, and the distance R as described above. That is, the phase difference CL and the phase difference CR are
If the sound source moves within relatively close range from the origin, the delta _L and delta _R, can be approximated to a plane, delta _L and delta
Since the phase difference CL corresponds to _R , the phase difference CL and the phase difference CR can be expressed by linear equations of x and y as shown in the above-described equations (4) and (5).

[0147] range in this system, the position and orientation of a sound source which moves within a predetermined range is detected, the predetermined range, the delta _L and [Delta] R, can be approximated to a plane, i.e., the formula It means the range where (7) holds.

The coefficient a _L , coefficient b _L , coefficient c _L , coefficient a _R , coefficient b _R , and coefficient c _R stored in the position coefficient storage unit 132 are, as described below, It is obtained by solving a linear equation using the least squares method.

The mouth (sound source) is moved to a plurality of predetermined positions (K positions specified by coordinates (x _k , y _k ) (k = 1, 2,..., K)). The sound emitted from the sound source at the time is collected, and an experiment for calculating the phase difference CL _k and the phase difference CR _k is performed. As shown in Expressions (8) and (9), K values of A linear equation of x and y is determined.

(Equation 4)

(Equation 5)

Equation (8) can be solved by expanding as shown in equation (10) and further as equation (11). Thereby, the coefficient a _L , the coefficient b _L , and the coefficient c _L are obtained. Equation (9) can also be solved by expanding as shown in equation (12) and further equation (13). Thus, a coefficient a _R , a coefficient b _R , and a coefficient c _R are obtained.

(Equation 6)

(Equation 7)

(Equation 8)

(Equation 9)

In the formulas (11) and (13), "
The matrix with the superscript "+" is a pseudo inverse matrix.

Referring back to FIG. 30, the orientation coefficient determining unit 133
A reference position (described later) corresponding to the coordinates (x, y) supplied from the position calculation unit 131 is detected.

The direction coefficient determination unit 133 also stores a coefficient α _p , a coefficient β _p , a coefficient γ _p , and a coefficient δ _p corresponding to each of the P reference positions p (p = 1, 2,... P). A coefficient α, a coefficient β, a coefficient γ, and a coefficient δ corresponding to the detected reference position are selected from the stored coefficients, and are supplied to the direction calculation unit 134.

The direction calculating section 134 is provided for the direction coefficient determining section 13.
Based on the coefficient α, the coefficient β _{, the} coefficient γ, and the coefficient δ from 3 and the sound pressure ratio RL from the feature amount calculation unit 111, a cubic equation of θ shown in Expression (14) is solved to calculate θ. It is supplied to the output unit 14.

(Equation 10)

By the way, the sound pressure ratio RL differs depending on the position and the direction of the sound source. For example, as shown in FIG. 34, the sound pressure ratio RL of the sound emitted when the sound source is at the reference position p specified by the coordinates (x _p , y _p ) (p = 1, 2,... 9) is , Respectively, and also depending on the orientation.

[0156] Figure 35 (A) is located at the reference position p (= 4), representative of the sound pressure ratio RL ₄ voice generated from the sound source when facing the direction of the angle theta _L to the angle theta _R I have. Fig. 35 (B) located at the reference position p (= 5), represents the sound pressure ratio RL ₅ voice generated from the sound source when facing the direction of the angle theta _L to angle theta _R. FIG.
(C) is located at the reference position p (= 6), it represents the sound pressure ratio RL ₆ voice generated from the sound source when facing the direction of the angle theta _L to angle theta _R.

From the trajectories shown in FIGS. 35A to 35C,
The sound pressure ratio RL is approximated by a cubic expression of the angle θ as shown in Expression (14).

The direction coefficient determination unit 133 stores
The coefficient α _p , coefficient β _p , coefficient γ _p , and coefficient δ _p corresponding to each of the P reference positions p are obtained as described below.

Since the sound pressure ratio RL differs depending on the position and the direction of the sound source, the sound source is located at the reference position p and is oriented in a predetermined direction (the direction of the angle θ _q (q = 1, 2,... Q)). By _calculating the sound pressure ratio RL _pq of the sound emitted from the sound source, Q three-dimensional equations of θ are obtained for each of the reference positions 1 to P.

[0160] For example, a sound source located at the reference position 1, the sound produced by the time that each oriented at an angle theta ₁ to theta _Q, by obtaining the sound pressure ratio RL ₁₁ to RL _1Q, formula (15) The Q three-dimensional equations of θ as shown in FIG.

[Equation 11]

Similarly, for each of the reference positions 2 to P, Q three-dimensional equations of θ are obtained, and these are solved by the nonlinear least squares method such as the Newton method or the simplex method. A coefficient α, a coefficient β, a coefficient γ, and a coefficient δ are obtained for each of the reference positions 1 to P.

Next, the operation of the detection apparatus 101 will be described with reference to the flowchart in FIG.

In step S61, the detecting device 101
Input unit 11 receives the input microphone amplifiers 2L, 2C, 2
Each of the processing audio signals L, C, and R from each of the R (portion input within a predetermined time) is converted into a feature amount calculation unit 11.
Supply 1

Next, in step S62, the feature amount calculation unit 111 (phase difference calculation units 121 and 122) calculates the phase difference CL from the processing audio signals C and L supplied from the input unit 11, and From the processing audio signals C and R,
The phase difference CR is calculated. The calculated phase difference CL and phase difference CR are supplied to the detection unit 112.

In step S63, the characteristic amount calculation unit 1
11 (sound pressure calculation units 123 and 124, sound pressure ratio calculation unit 12
5) calculates the sound pressures L and R of the processing audio signals L and R, and calculates the sound pressure ratio RL (= sound pressure R / sound pressure L). The calculated sound pressure ratio RL is supplied to the detection unit 112.

Next, in step S64, the detecting unit 1
12 (the position calculating unit 131), the phase difference CL supplied from the feature amount calculating unit 111, and the coefficients a _L , b _L , and c stored by itself (the position coefficient storing unit 132).
X and y are calculated by solving a simultaneous equation of equation (4) consisting of _L and equation (5) consisting of phase difference CR and coefficients a _R , b _R , and c _R. The calculated coordinates (x,
y) is supplied to the output unit 14.

In step S65, the detecting unit 112
The (direction coefficient determination unit 133) calculates the calculated coordinates (x, y)
The reference position closest to the position specified by is detected.

Next, in step S66, the detecting unit 1
12 (orientation coefficient determination unit 133) stores the stored coefficient α _p , coefficient β _p ,
A coefficient α, a coefficient β, a coefficient γ, and a coefficient δ corresponding to the determined reference position are selected from the coefficient γ _p and the coefficient δ _p .

At step S67, the detecting unit 112
The (direction calculation unit 134) calculates the angle θ by solving the expression (14) including the sound pressure ratio RL from the feature amount calculation unit 111 and the coefficients α, β, γ, and δ selected in step S66. I do. The calculated angle θ is output unit 1
4 is supplied.

Next, in step S68, the output unit 1
4 represents the coordinates (x, y) supplied from the detection unit 112,
The position of the mouth (sound source) and the angle θ are output to an external device as the direction of the sound source.

Thereafter, the processing ends.

In the above description, the detection unit 112 detects the reference position corresponding to the calculated coordinates (x, y), and stores and stores the P coefficients α _p and β _p , respectively. , Coefficient γ _p , and coefficient δ _p , the coefficient α, coefficient β, coefficient γ, and coefficient δ corresponding to the detected reference position are selected.
As shown in Expressions (16) to (19), a function e of x and y for calculating the coefficient α, a function f of x and y for calculating the coefficient β, and a function f of x and y for calculating the coefficient γ The function g and the function h of x and y for which the coefficient δ is calculated are stored, and those functions are calculated at the detected coordinates (x, y) to obtain the coefficient α, the coefficient β, the coefficient γ, and the coefficient Each of δ can also be calculated. α = e (x, y) (16) β = f (x, y) (17) γ = g (x, y) (18) δ = h (x, y) (19)

The operation of the detection device 101 in this case is described in FIG.
This will be described with reference to the flowchart of FIG.

In steps S71 to S74 and step S77, step S61 in FIG.
Since the same processing as in steps S64 and S68 is performed, the description thereof is omitted.

In step S75, the detecting device 101
, The function e (equation (16)), the function f (equation (17)), the function g (equation (18)), and the function h (equation (19)) are calculated, and the coefficient α, A coefficient β, a coefficient γ, and a coefficient δ are calculated.

Next, in step S76, the detecting unit 1
12 calculates the angle θ by calculating the equation (14) using the sound pressure ratio RL from the feature amount calculation unit 111 and the coefficients α, β, γ, and δ calculated in step S75.

Further, in the above description, the feature amount calculating unit 11
1 has been described as an example in which the sound pressures L and R are calculated directly from the processing audio signals L and R, and then the sound pressure ratio RL is calculated. The sound pressures Lh and Rh of the processing audio signals Lh and Rh) are calculated,
The sound pressure ratio RLh can be calculated.

In this example, the characteristic amount calculation unit 111 is used.
38 is shown in FIG. In the feature amount calculation unit, a high-pass filter 141 and a high-pass filter 142 are provided at a stage preceding the sound pressure calculation unit 123 and the sound pressure calculation unit 124 of the feature amount calculation unit 111 in FIG.

The high-pass filter 141 passes only the high-frequency portion of the processing audio signal L supplied from the input unit 11 and supplies the processed audio signal L to the sound pressure calculation unit 123. The high-pass filter 142 passes only the high-frequency portion of the processing audio signal R supplied from the input unit 11 and supplies the processed audio signal R to the sound pressure calculation unit 124.

The sound pressure calculation section 123 calculates the sound pressure Lh of the high-frequency processing audio signal Lh and supplies it to the sound pressure ratio calculation section 125. The sound pressure calculator 124 calculates the sound pressure Rh of the high-frequency processing audio signal Rh, and supplies the calculated sound pressure Rh to the sound pressure ratio calculator 125.

The sound pressure ratio calculator 125 calculates the sound pressure ratio RLh and supplies it to the detector 112.

FIG. 39 shows a configuration example of a sound source state detection system according to a fifth embodiment of the present invention. In this system, as in the fourth embodiment, both the position and the direction of the sound source moving within the predetermined range are detected. In this example, the position and the direction in the three-dimensional space are detected. Is detected.

This system includes the detecting device 3 shown in FIG.
, A detection device 201 is provided, and in addition, four microphones 1-1 to 1-4 and four microphone amplifiers 2-1 to 2-4 are provided.

Microphones 1-1 to 1-4 are shown in FIG.
X-axis, Y-axis and Z-axis, respectively, as shown in
They are arranged with a certain extent in a three-dimensional space consisting of axes. Note that the origin, X axis, and Y axis in FIG. 40 correspond to the origin, X axis, and Y axis in FIG. That is, the Z axis shown in FIG. 40 is an axis extending in the vertical direction from the origin as viewed from the user in FIG.

Returning to FIG. 39, the microphones 1-1 to 1
-4 collects the sound from the sound source (the sound uttered by the user) and supplies it to the microphone amplifiers 2-1 to 2-4 as a sound signal.

The microphone amplifiers 2-1 to 2-4 amplify the audio signals from the microphones 1-1 to 1-4 and supply the amplified audio signals to the detection device 201 as processing audio signals S1 to S4.

The detection device 201 calculates the coordinates (x, y, x, y, y) in the three-dimensional space in FIG. 40 corresponding to the position of the sound source (mouth).
z), and an angle θ around the Z axis, an angle φ around the Y axis, and an angle ψ around the X axis corresponding to the orientation of the mouth are detected.

FIG. 41 shows a configuration example of the detection device 201. The detection device 201 includes a feature value calculation unit 211 instead of the feature value calculation unit 12 of the detection device 3 in FIG. 4, and a detection unit 212 instead of the detection unit 13.

The input section 11 is provided with microphone amplifiers 2-1 to 2
-4, the processing audio signal S1 appropriately supplied from
To S4, and supplies them to the feature amount calculation unit 211.

The feature amount calculating section 211 outputs the processing audio signal S
1 and the phase of the processing audio signal S2 (hereinafter referred to as a phase difference A1), and the difference between the phase of the processing audio signal S2 and the phase of the processing audio signal S3 (hereinafter referred to as a phase difference A2). ) And the difference between the phase of the processing audio signal S3 and the phase of the processing audio signal S4 (hereinafter referred to as phase difference A3)
Calculate each.

The feature amount calculation unit 211 calculates the sound pressures of the processing audio signals S1 to S4, and also calculates the ratio of the sound pressure of the processing audio signal S1 to the sound pressure of the processing audio signal S2 (hereinafter referred to as the sound pressure). Pressure ratio B1), processing audio signal S2
, And the ratio of the sound pressure of the processing audio signal S3 (hereinafter referred to as the sound pressure ratio B2), and the ratio of the sound pressure of the processing audio signal S3 to the sound pressure of the processing audio signal S4 (hereinafter, referred to as the sound pressure ratio B3). Name)
Are calculated respectively.

The feature amount calculating section 211 calculates the calculated phase difference A
1 to phase difference A3, and sound pressure ratio B1 to sound pressure ratio B3
Is output to the detection unit 212.

Based on the phase differences A1 to A3 from the feature amount calculation unit 211 and a predetermined coefficient (described later), the detection unit 212 determines the coordinates (x,
y, z) is calculated.

The detecting unit 212 also calculates a predetermined position, a predetermined direction, and a corresponding relationship between the sound pressure ratios B1 to B3 of the sound emitted from the sound source of the position and the direction, which are obtained in advance. The direction corresponding to the determined sound pressure ratios B1 to B3 and the calculated position (coordinates) of the sound source is detected as the direction of the sound source at this time.

FIG. 42 shows an example of the configuration of the feature amount calculation unit 211.

The phase difference calculator 221 calculates the phase difference A1 and supplies it to the detector 212. The phase difference calculation unit 222
The phase difference A2 is calculated and supplied to the detection unit 212. The phase difference calculation unit 223 calculates the phase difference A3 and supplies the same to the detection unit 212.

The sound pressure calculating section 224 outputs the processing sound signal S1.
Is calculated and supplied to the sound pressure ratio calculation unit 228. The sound pressure calculation unit 225 calculates the sound pressure of the processing audio signal S2,
It is supplied to the sound pressure ratio calculation unit 228 and the sound pressure ratio calculation unit 229.

The sound pressure calculating section 226 outputs the processing sound signal S3
Is calculated and supplied to the sound pressure ratio calculation unit 229 and the sound pressure ratio calculation unit 230. The sound pressure calculator 227 calculates the sound pressure of the processing audio signal S4 and supplies the calculated sound pressure to the sound pressure ratio calculator 230.

The sound pressure ratio calculation section 228 calculates the sound pressure ratio B1 and supplies it to the detection section 212. The sound pressure ratio calculation unit 229
The sound pressure ratio B2 is calculated and supplied to the detection unit 212. The sound pressure ratio calculation unit 230 calculates the sound pressure ratio B3 and supplies it to the detection unit 212.

FIG. 43 shows an example of the configuration of the detection section 212.

The position calculating section 241 includes the feature amount calculating section 21
1 (phase difference calculation units 221 to 223) and the phase coefficient storage unit 2
42 coefficients a ₁ stored in the _coefficient b _1, coefficients c _1, and coefficient d _1, coefficients a _2, the coefficient b _2, coefficients c _2, and the coefficient d _2, and the coefficient a _3, the coefficient b _3, coefficient c ₃ and coefficient d ₃
Is supplied.

The position calculating section 241 calculates the coefficient a _{1, the} coefficient b ₁ ,
X, y, z linear equations (Equation (20)) consisting of the coefficients c ₁ and d ₁ and the phase difference A1, the coefficients a _2, b ₂ , c ₂ and d ₂ , and x consisting retardation A2, y, 1 equations of z (formula (21)), and the coefficient a _3, the coefficient b _3, coefficients c _3, and the coefficient d _3, and consists of a phase difference A3 x, y, and z Linear equation (Equation (22))
X, y, and z are calculated. a ₁ × x + b ₁ × y + c ₁ × z + d ₁ = phase difference A1 (20) a ₂ × x + b ₂ × y + c ₂ × z + d ₂ = phase difference A2 (21) a ₃ × x + b ₃ × y + c ₃ × z + d ₃ = Phase difference A3 (22)

The position calculator 241 calculates the calculated coordinates (x,
y, z) (the position of the sound source) is supplied to the direction detection unit 243 and the output unit 14.

The coefficient a stored in the position coefficient storage unit 242
_{1 to} d ₁ , coefficients a _{2 to} d ₂ , and coefficients a _{3 to} d ₃ are:
Basically, the coefficients a _{L to} c _L in the fourth embodiment are used.
And the coefficients a _{R to} c _R are calculated in the same manner as in the calculation.

That is, the mouth (sound source) is moved to a plurality of predetermined positions (coordinates (x _k , y _k , z _k )), and the sound emitted from the sound source at each position is collected. Phase difference A
By conducting an experiment for calculating 1 _K , the phase difference A2 _k , and the phase difference A3 _k , simultaneous equations as shown in Expressions (23) to (25) are obtained.

(Equation 12)

(Equation 13)

[Equation 14]

By solving the equations (23) to (25) by the least square method, the coefficients a _{1 to} d ₁ , the coefficients a _{2 to} d ₂ ,
And the coefficients a _{3 to} d ₃ are obtained.

The direction detecting section 243 determines a predetermined position (coordinates (x, y,
z)), predetermined directions (angles θ, φ, ψ), and the sound pressure ratio B of the sound emitted from the sound source at the position and the direction.
A correspondence table representing the correspondence between 1, B2, and B3 is stored. That is, the direction detection unit 243 includes the position calculation unit 241
Based on the coordinates (x, y, z) from the data and the sound pressure ratios B1, B2, and B3 from the feature amount calculation unit 211, the angles θ, φ, and 対 応 corresponding thereto are detected as the direction of the sound source, and the output unit 14 is output.

The operation of the detecting device 201 in this example is as follows.
This will be described with reference to the flowchart in FIG.

At step S81, the detecting device 201
Input unit 11 is provided with the input microphone amplifiers 2-1 through 2-
4 (parts input within a predetermined time) from the feature amount calculating unit 21.
Supply 1

Next, in step S82, the characteristic amount calculation unit 211 (phase difference calculation units 221 to 223) converts the processing audio signals S1 to S4 supplied from the microphone amplifiers 2-1 to 2-4 into processing signals. The phase difference A1 between the phase of the audio signal S1 and the phase of the processing audio signal S2, and the phase difference A between the phase of the processing audio signal S2 and the phase of the processing audio signal S3.
2, and a phase difference A3 between the phase of the processing audio signal S3 and the phase of the processing audio signal S4. The calculated phase differences A1 to A3 are supplied to the detection unit 212.

In step S83, the characteristic amount calculation unit 2
11 (sound pressure calculation units 224 to 227, sound pressure ratio calculation unit 22
8 to 230) are processing audio signals S1, S2, S3,
While calculating the sound pressure of S4, the processing sound signal S1,
From the sound pressure of S2, the sound pressure ratio B1 is converted to the processing sound signal S2.
The sound pressure ratio B2 is calculated from the sound pressure of S3, and the sound pressure ratio B3 is calculated from the sound pressures of the processing audio signals S3 and S4. The calculated sound pressure ratio B1 to sound pressure ratio B3 is
12 is supplied.

Next, in step S84, the detecting unit 2
12 (the position calculation unit 241) includes the phase differences A1 to A3 supplied from the feature amount calculation unit 211, and the coefficients a _{1 to} d ₁ stored in itself (the position coefficient storage unit 242).
X, y, and z are calculated by solving simultaneous equations composed of the equations (20) to (22) of the coefficients a _{2 to} d ₂ and the coefficients a _{3 to} d ₃ . The calculated coordinates (x, y, z) are supplied to the output unit 14.

At step S85, the detecting section 212
The (direction detection unit 243) detects the coordinates closest to the calculated coordinates (x, y, z) and the sound pressure ratios closest to the calculated sound pressure ratios B1, B2, and B3 from the correspondence table in FIG. , And the angles θ, φ, and ψ corresponding to them are detected. The detected angle θ, angle φ, and angle ψ are supplied to the output unit 14.

In step S86, the output unit 14
The coordinates (x, y, z) supplied from the detection unit 212 are output to an external device as the position of the mouth (sound source), and the angles θ, φ, and ψ as the directions of the mouth.

Thereafter, the processing ends.

In the above description, the feature value calculating unit 21
1 has been described as an example in which the sound pressure is directly calculated from the processing audio signals S1 to S4 and the sound pressure ratios B1 to B3 are calculated, but the signals of the high frequency portions of the processing audio signals S1 to S4 (high-frequency processing It is also possible to calculate the sound pressures of the audio signals for use S1h to S4h) and calculate their sound pressure ratios B1h to B3h. The sound pressure ratio of the signal in the high-frequency portion changes depending on the distance from the sound source, but since it has directivity, the direction of the sound source can be detected more accurately by using it. Especially effective.

In this example, the feature amount calculation unit 211 is used.
46 is shown in FIG. This feature amount calculation unit 211 includes high-pass filters 251 to 254 before the sound pressure calculation units 224 to 227 of the feature amount calculation unit 211 in FIG.
Is provided.

The high-pass filter 251 passes only the high-frequency portion of the processing audio signal S 1 supplied from the input unit 11 and supplies it to the sound pressure calculation unit 224. The high-pass filter 252 allows only the high-frequency portion of the processing audio signal S2 supplied from the input unit 11 to pass, and
5

The high-pass filter 253 allows only the high-frequency portion of the processing audio signal S3 supplied from the input unit 11 to pass and supplies it to the sound pressure calculation unit 226. The high-pass filter 254 allows only the high-frequency portion of the processing audio signal S4 supplied from the input unit 11 to pass, and
7

[0220] The sound pressure calculation section 224 calculates the sound pressure of the audio signal S1h for high-frequency processing and supplies it to the sound pressure ratio calculation section 228. The sound pressure calculation unit 225 outputs the high-frequency processing audio signal S2h.
Is calculated and supplied to the sound pressure ratio calculation unit 228 and the sound pressure ratio calculation unit 229.

The sound pressure calculation section 226 calculates the sound pressure of the high frequency processing audio signal S3h, and supplies it to the sound pressure ratio calculation section 229 and the sound pressure ratio calculation section 230. The sound pressure calculation section 227 calculates the sound pressure of the high-frequency processing audio signal S4h and supplies it to the sound pressure ratio calculation section 230.

The sound pressure ratio calculating section 228 calculates the sound pressure ratio B1 between the sound pressure of the high-frequency processing audio signal S1h and the sound pressure of the high-frequency processing audio signal S2h, and supplies it to the detection section 212. The sound pressure ratio calculation unit 229 calculates a sound pressure ratio B2 between the sound pressure of the high-frequency processing audio signal S2h and the sound pressure of the high-frequency processing audio signal S3h, and supplies this to the detection unit 212. Sound pressure ratio calculator 230
Calculates the sound pressure ratio B3 between the sound pressure of the high-frequency processing sound signal S3h and the sound pressure of the high-frequency processing sound signal S4h, and supplies the sound pressure ratio B3 to the detection unit 212.

Although the timing of detecting the state (position or direction) of the sound source (mouth) has not been described above, the detection devices 3, 31, 61, 10
An operation unit is provided in each of the devices 201 and 201 so that when the operation unit is operated, the state of the sound source is detected, or is detected at a predetermined cycle and continuously for a predetermined period. Can be.

Further, in the above, the first to fifth embodiments have been described separately, but they may be combined to detect the state of the user. For example, in the fourth and fifth embodiments, the case where both the position and the direction of the sound source are calculated has been described as an example. However, only the position of the sound source may be detected.

In the fourth and fifth embodiments, the case where the sound pressure is directly calculated from the processing audio signal and the sound pressure ratio is calculated based on the calculated sound pressure is different from the case where the sound pressure ratio is calculated based on the sound pressure. Although the case where the sound pressure is calculated and the sound pressure ratio is calculated based on the sound pressure has been described separately, they can be used in combination.

The series of processes described above can be realized by hardware, but can also be realized by software. When a series of processing is realized by software, a program constituting the software is installed in a computer, and the program is executed by the computer, so that the detection devices 3, 31, 61, 101, 201 described above are used. Is functionally realized.

FIG. 47 shows the detection devices 3 and 3 as described above.
FIG. 1 is a block diagram illustrating a configuration of an embodiment of a computer 501 that functions as 1, 61, 101, and 102.
A bus 515 is provided for a CPU (Central Processing Unit) 511.
An input / output interface 516 is connected via the input / output interface 516. When a command is input from the user via the input / output interface 516 to the input unit 518 including a keyboard, a mouse, and the like, the CPU 511 reads, for example, a ROM (Read
Only memory) 512, a hard disk 514, or a program stored in a recording medium such as a magnetic disk 531, an optical disk 532, a magneto-optical disk 533, or a semiconductor memory 534 mounted on the drive 520 is stored in a random access memory (RAM) 513. And run it. Thereby, the various processes described above are performed. Further, the CPU 511 transmits the processing result to, for example,
LCD (Liquid Cr) via the input / output interface 516
Output to a display unit 517 such as a ystalDisplay) as necessary. The program is stored on the hard disk 51
4 or ROM 512 in advance, and the computer 50
1 and the magnetic disk 53
1. It can be provided as a package medium such as an optical disk 532, a magneto-optical disk 533, or a semiconductor memory 534, or can be provided to a hard disk 514 via a communication unit 519 from a satellite, a network, or the like.

In the present specification, the step of describing a program provided by a recording medium may be performed not only in chronological order according to the described order but also in chronological order. This also includes processing executed in parallel or individually.

[0229] In this specification, the system is
It represents the entire device composed of a plurality of devices.

[0230]

According to the information processing apparatus and method and the first program of the present invention, it is possible to cope with a sound emitted by a predetermined sound source, which is collected by a plurality of sound collection devices installed at predetermined positions. From the audio signal to be calculated, a feature amount corresponding to the state of the sound source is calculated, and based on the calculated feature amount, the state of the sound source is detected.
The state of the sound source can be detected.

According to the information generating apparatus and method of the present invention, and the second program, the sound collected by each of the first sound collecting apparatus, the second sound collecting apparatus, and the third sound collecting apparatus, A fourth sound signal corresponding to a sound emitted from a sound source located at a plurality of experiment positions provided within a predetermined range, a fifth sound signal
From the audio signal and the sixth audio signal, the third phase difference between the phase of the fourth audio signal and the phase of the fifth audio signal, and the phase of the fourth audio signal and the sixth audio signal. Since the fourth phase difference is calculated for each experimental position,
A first coefficient is calculated based on the relationship between the phase difference and the experimental position, and based on the relationship between the fourth phase difference and the experimental position,
A second coefficient can be calculated.

[Brief description of the drawings]

FIG. 1 shows a first example of a sound source state detection system to which the present invention is applied.
It is a block diagram which shows the example of a structure of 1st Embodiment.

FIG. 2 is a diagram showing an installation position of a microphone 1;

FIG. 3 is a diagram showing a correspondence relationship between a sound pressure ratio RL and a direction of a sound source.

FIG. 4 is a block diagram showing a configuration example of a detection device 3 of FIG.

FIG. 5 is a block diagram illustrating a configuration example of a feature amount calculation unit 12 in FIG. 4;

FIG. 6 is a flowchart illustrating an operation of the sound source state detection system.

FIG. 7 is a flowchart illustrating details of a process in step S3 of FIG. 6;

FIG. 8 shows a second example of the sound source state detection system to which the present invention is applied.
It is a block diagram which shows the example of a structure of 1st Embodiment.

FIG. 9 is another diagram showing an installation position of the microphone 1;

FIG. 10 is a diagram showing a correspondence relationship between a sound pressure ratio RLh and a direction of a sound source.

11 is a block diagram illustrating a configuration example of a detection device 31 in FIG.

12 is a block diagram illustrating a configuration example of a feature amount calculation unit 41 in FIG. 11;

FIG. 13 is a diagram illustrating cutoff characteristics.

FIG. 14 is a diagram illustrating a pitch frequency.

FIG. 15 is a flowchart illustrating another operation of the sound source state detection system.

FIG. 16 is a flowchart illustrating details of a process in step S23 in FIG. 15;

FIG. 17 is a block diagram illustrating a configuration example of a third embodiment of a sound source state detection system to which the present invention has been applied.

FIG. 18 is a diagram illustrating a correspondence relationship between a phase difference and a face direction.

FIG. 19 is a diagram illustrating a phase difference.

FIG. 20 is another diagram illustrating a phase difference.

FIG. 21 is another diagram illustrating a phase difference.

22 is a block diagram illustrating a configuration example of a detection device 61 in FIG.

FIG. 23 is a flowchart illustrating another operation of the sound source state detection system.

FIG. 24 is a flowchart illustrating details of a process in step S43 of FIG. 23;

FIG. 25 is a block diagram illustrating a configuration example of a fourth embodiment of a sound source state detection system to which the present invention has been applied.

FIG. 26 is a diagram illustrating an arrangement position of the microphone 1.

FIG. 27 is a diagram illustrating a position and an orientation.

FIG. 28 is a block diagram illustrating a configuration example of a detection device 101 in FIG. 25.

29 is a block diagram illustrating a configuration example of a feature amount calculation unit 111 in FIG. 28.

30 is a block diagram illustrating a configuration example of a detection unit 112 in FIG. 28.

FIG. 31 is a diagram illustrating ΔL and ΔR.

FIG. 32 is a diagram illustrating a distance C and a distance L.

FIG. 33 is a diagram illustrating a phase difference CL and a phase difference CR.

FIG. 34 is a diagram illustrating a reference position p.

FIG. 35 is a diagram illustrating a sound pressure ratio RL.

FIG. 36 is a flowchart illustrating an operation of the detection device 101.

FIG. 37 is a flowchart illustrating another operation of the detection device 101.

38 is a block diagram illustrating another configuration example of the feature amount calculation unit 111. FIG.

FIG. 39 is a block diagram illustrating a configuration example of a fifth embodiment of a sound source state detection system to which the present invention has been applied.

FIG. 40 is another diagram illustrating a position and an orientation.

41 is a block diagram illustrating a configuration example of a detection device 201 in FIG. 39.

42 is a block diagram illustrating a configuration example of a feature amount calculation unit 211 in FIG. 41.

FIG. 43 is a block diagram illustrating a configuration example of a detection unit 212 in FIG. 41.

FIG. 44 is a diagram showing an example of a correspondence table showing a correspondence relationship among coordinates, sound pressure ratios, and angles.

FIG. 45 is a flowchart illustrating an operation of the detection device 202.

FIG. 46 is a block diagram illustrating another configuration example of the feature amount calculation unit 211.

FIG. 47 is a block diagram illustrating a configuration example of a personal computer 501.

[Description of Signs] 1 microphone, 2 microphone amplifier, 3 detection device, 11 input unit, 12 feature amount calculation unit, 13
Detection unit, 14 output unit, 21 sound pressure calculation unit, 2
2 sound pressure ratio calculation section, 31 detection device, 41 feature quantity calculation section, 42 detection section, 51 high-pass filter,
61 detection device, 71 feature amount calculation unit, 72 detection unit, 101 detection device, 111 feature amount calculation unit,
112 detection section, 121 phase difference calculation section, 122 phase difference calculation section, 123 sound pressure calculation section, 124 sound pressure calculation section, 125 sound pressure ratio calculation section, 131 position calculation section, 132 position coefficient storage section, 133 direction coefficient determination section , 134 orientation calculation unit, 141 high-pass filter, 142 high-pass filter, 201 detection device, 211 feature amount calculation unit, 212 detection unit, 2
21 phase difference calculator, 222 phase difference calculator, 22
3 phase difference calculator, 224 sound pressure calculator, 225
Sound pressure calculating section, 226 sound pressure calculating section, 227 sound pressure calculating section, 228 sound pressure ratio calculating section, 229 sound pressure ratio calculating section, 230 sound pressure ratio calculating section, 241 position calculating section,
242 position coefficient storage unit, 243 direction detection unit,
251 high-pass filter, 252 high-pass filter, 253 high-pass filter, 254 high-pass filter

   ────────────────────────────────────────────────── ─── Continuation of front page    (72) Inventor Koichi Fujishima             6-7-35 Kita Shinagawa, Shinagawa-ku, Tokyo Soni             ー Inc. (72) Inventor Tomoyuki Otsuki             6-7-35 Kita Shinagawa, Shinagawa-ku, Tokyo Soni             ー Inc. F term (reference) 5D018 BB23                 5D020 BB04

Claims (22)

[Claims] 1. A method for calculating a feature amount corresponding to a state of a sound source from an audio signal corresponding to a sound emitted from a predetermined sound source, which is collected by a plurality of sound collecting devices installed at a predetermined position. An information processing apparatus, comprising: a first calculating unit; and a state detecting unit that detects a state of the sound source based on the feature amount calculated by the first calculating unit. 2. The information processing apparatus according to claim 1, wherein the state is a direction of the sound source at a predetermined position. 3. The apparatus according to claim 2, wherein the state detecting means is obtained in advance.
2. The apparatus according to claim 1, wherein correspondence data indicating a correspondence relationship between the feature amount and the state is held, and the state is detected based on the correspondence data and the feature amount.
An information processing apparatus according to claim 1. 4. The first sound signal corresponding to the sound emitted by the sound source, collected by the first sound collecting device, and the second sound collecting device. A sound pressure ratio between a sound pressure of the first sound signal and a sound pressure of the second sound signal, from a second sound signal corresponding to a sound emitted by the sound source collected by Calculated as an amount, the state detection means holds the previously obtained correspondence data indicating the correspondence between the sound pressure ratio and the state, based on the correspondence data and the sound pressure ratio,
The information processing apparatus according to claim 3, wherein the state is detected. 5. The first calculating means includes: a first sound signal corresponding to a sound emitted by the sound source, collected by the first sound collecting device; and a second sound collecting device. A sound pressure of a first high-frequency sound signal composed of a high-frequency portion of the first sound signal from the second sound signal corresponding to the sound emitted by the sound source, and a second sound A sound pressure ratio between a sound pressure of a second high-frequency sound signal composed of a high-frequency portion of the signal and the sound pressure is calculated as the characteristic amount. Holding the corresponding relationship data shown, based on the corresponding relationship data and the sound pressure ratio,
The information processing apparatus according to claim 3, wherein the state is detected. 6. The first sound signal corresponding to the sound emitted by the sound source, collected by the first sound collecting device, and the second sound collecting device. A phase difference between the phase of the first audio signal and the phase of the second audio signal from the second audio signal corresponding to the audio emitted by the sound source, collected as Calculating, the state detection means holds the correspondence data indicating the correspondence between the phase difference and the state obtained in advance, based on the correspondence data and the phase difference,
The information processing apparatus according to claim 3, wherein the state is detected. 7. The first calculating means is located within a predetermined range collected by the first sound collecting device, the second sound collecting device, and the third sound collecting device. From the first audio signal, the second audio signal, and the third audio signal corresponding to the audio emitted by the sound source, the phase of the first audio signal and the phase of the second audio signal And the second phase difference between the phase of the first audio signal and the phase of the third audio signal are calculated as the characteristic amount. And the second
The information processing apparatus according to claim 1, wherein the position of the sound source on a plane is detected based on the phase difference of the sound source. 8. A plurality of experimental positions provided within a predetermined range and collected by each of the first sound collecting device, the second sound collecting device, and the third sound collecting device. From the fourth audio signal, the fifth audio signal, and the sixth audio signal corresponding to the audio emitted by the located sound source, the phase of the fourth audio signal and the phase of the fifth audio signal Calculating a third phase difference with respect to the phase and a fourth phase difference between the phase of the fourth sound signal and the sixth sound signal for each of the experimental positions; Second calculating means for calculating a first coefficient based on a relationship between the first phase difference and the experimental position, and a second coefficient based on a relationship between the fourth phase difference and the experimental position; and the second calculating means. Storage means for storing the first coefficient and the second coefficient calculated by The state detecting means includes: on the plane of the sound source, based on a relationship between the first phase difference and the first coefficient and a relationship between the second phase difference and the second coefficient. The information processing apparatus according to claim 7, wherein the position of the information processing device is detected. 9. The method according to claim 1, wherein the first calculating unit calculates a sound pressure of the second sound signal and a sound pressure of the third sound signal from the second sound signal and the third sound signal. 1 is further calculated as the characteristic amount, and the state detection means detects the direction of the sound source based on the first sound pressure ratio and the detected position of the sound source. The information processing device according to claim 7. 10. A sound source, which is collected by each of the second sound collecting device and the third sound collecting device, is located at a plurality of reference positions and corresponds to a sound emitted by the sound source directed in a predetermined direction. Calculating a second sound pressure ratio between the sound pressure of the fourth sound signal and the sound pressure of the fifth sound signal from the fourth sound signal and the fifth sound signal. A second calculating unit that calculates a coefficient based on the relationship between the sound pressure ratio and the predetermined direction for each of the reference positions; and a storage unit that stores the coefficient in association with the reference position. The state detection means detects the reference position corresponding to the detected position of the sound source, selects the coefficient corresponding to the detected reference position from the storage means, and selects the selected coefficient and the first coefficient. Based on the relationship with the sound pressure ratio of The information processing apparatus according to claim 9, characterized in that detecting the direction of the source. 11. The first calculating means, wherein sound is collected by a first sound collecting device, a second sound collecting device, a third sound collecting device, and a fourth sound collecting device. From the first audio signal, the second audio signal, the third audio signal, and the fourth audio signal corresponding to the audio emitted by the sound source located within a predetermined range, A first phase difference between the phase of the audio signal and the phase of the second audio signal, a second phase difference between the phase of the second audio signal and the phase of the third audio signal, and A third phase difference between the phase of the third audio signal and the phase of the fourth audio signal is calculated as the feature quantity, and the state detection unit calculates the first phase difference and the second phase difference And detecting a spatial position of the sound source based on the third phase difference.
An information processing apparatus according to claim 1. 12. Within a predetermined range collected by each of the first sound collecting device, the second sound collecting device, the third sound collecting device, and the fourth sound collecting device. A fifth sound signal, a sixth sound signal, and a seventh sound signal corresponding to sounds emitted by the sound sources located at the provided plurality of experimental positions.
From the audio signal and the eighth audio signal, the fourth phase difference between the phase of the fifth audio signal and the phase of the sixth audio signal, the phase of the sixth audio signal and the fourth Calculating a fifth phase difference from the phase of the audio signal No. 7 and a sixth phase difference between the phase of the seventh audio signal and the phase of the eighth audio signal for each of the experimental positions; A first coefficient based on the relationship between the fourth phase difference and the experimental position, a second coefficient based on the relationship between the fifth phase difference and the experimental position, and the sixth phase difference A second calculating means for calculating a third coefficient based on a relationship with an experimental position; and the first coefficient calculated by the second calculating means;
A storage unit that stores the second coefficient and the third coefficient, wherein the state detection unit determines a relationship between the first phase difference and the first coefficient, the second phase difference The position of the sound source in space is detected based on a relationship between the sound source and the second coefficient and a relationship between the third phase difference and the third coefficient. An information processing apparatus according to claim 1. 13. The method according to claim 1, wherein the first calculating unit calculates the first audio signal from the first audio signal, the second audio signal, the third audio signal, and the fourth audio signal. A first sound pressure ratio between a sound pressure and a sound pressure of the second sound signal; a second sound pressure ratio between a sound pressure of the second sound signal and a sound pressure of the third sound signal; A third sound pressure ratio between the sound pressure of the sound signal of No. 3 and the sound pressure of the fourth sound signal is further calculated as the characteristic amount. The information processing apparatus according to claim 11, wherein the direction of the sound source is detected based on the sound pressure ratio of 2, the third sound pressure ratio, and the detected position of the sound source. 14. A plurality of reference positions collected by each of the first sound collecting device, the second sound collecting device, the third sound collecting device, and the fourth sound collecting device. A fifth sound signal, a sixth sound signal, a seventh sound signal, and an eighth sound signal corresponding to sounds emitted by the sound source positioned and directed in a predetermined direction; 5
The fourth of the sound pressure of the sound signal of
, A fifth sound pressure ratio between the sound pressure of the sixth sound signal and the sound pressure of the seventh sound signal, and the sound pressure of the seventh sound signal and the sound of the eighth sound signal. Second calculating means for calculating a sixth sound pressure ratio with respect to the pressure, respectively, and the fourth sound pressure ratio, the fifth sound pressure ratio, and the sixth sound pressure ratio, A storage unit for storing the direction, wherein the state detection unit stores the reference position corresponding to the detected position of the sound source from the storage unit;
The direction corresponding to the sound pressure ratio, the fourth sound pressure ratio corresponding to the second sound pressure ratio and the third sound pressure ratio, and the direction corresponding to the fifth sound pressure ratio and the sixth sound pressure ratio is defined as the direction of the sound source. The information processing apparatus according to claim 13, wherein the information is detected. 15. A calculation for calculating a characteristic amount corresponding to a state of a sound source from an audio signal corresponding to a sound emitted by a predetermined sound source, which is collected by a plurality of sound collection devices installed at a predetermined position. And a state detecting step of detecting a state of the sound source based on the feature amount calculated in the processing of the calculating step. 16. A calculation for calculating a feature amount corresponding to a state of a sound source from an audio signal corresponding to a sound emitted by a predetermined sound source, which is collected by a plurality of sound collection devices installed at a predetermined position. And a state detecting step of detecting a state of the sound source based on the feature amount calculated in the processing of the calculating step. A recording medium in which a computer-readable program is recorded. 17. A calculation for calculating a feature amount corresponding to a state of a sound source from an audio signal corresponding to a sound emitted by a predetermined sound source, which is collected by a plurality of sound collection devices installed at a predetermined position. A program causing a computer to execute a process including: a step; and a state detection step of detecting a state of the sound source based on the feature amount calculated in the processing of the calculation step. 18. A first sound collecting device, a second sound collecting device, and a first sound collecting device, the first sound collecting device corresponding to the sound emitted from a sound source located within a predetermined range, which is collected by a third sound collecting device. Audio signal,
From the second audio signal and the third audio signal, the first
Of the phase of the second audio signal and the phase of the second audio signal
And the phase of the first audio signal and the third
Calculating a second phase difference with the phase of the audio signal of
The first coefficient and the first coefficient for a detection device that detects the position of the sound source based on the relationship between the phase difference and the first coefficient, and the relationship between the second phase difference and the second coefficient. An information generation device that generates a second coefficient, provided in a predetermined range collected by each of the first sound collection device, the second sound collection device, and the third sound collection device. From the fourth sound signal, the fifth sound signal, and the sixth sound signal corresponding to sounds emitted by the sound source located at the plurality of test positions, the phase of the fourth sound signal and Calculating a third phase difference between the phase of the fifth audio signal and a fourth phase difference between the phase of the fourth audio signal and the sixth audio signal for each experimental position; Calculating means, based on a relationship between the third phase difference and the experimental position,
An information generating apparatus comprising: a second calculating unit configured to calculate the first coefficient and calculate the second coefficient based on a relationship between the fourth phase difference and the experimental position. . 19. The detection device according to claim 1, wherein said detection device is configured to generate a sound corresponding to a sound emitted by said sound source located at a plurality of reference positions and collected by each of said second sound collection device and said third sound collection device. Calculating a sound pressure ratio between the sound pressure of the seventh sound signal and the sound pressure of the eighth sound signal from the sound signal of No. 7 and the eighth sound signal; Based on the relationship, the direction of the sound source is further detected, and the sound emitted by the sound source located at a plurality of reference positions is collected by each of the second sound collecting device and the third sound collecting device. Third calculating means for calculating a sound pressure ratio between the sound pressure of the ninth sound signal and the sound pressure of the tenth sound signal from the ninth sound signal and the tenth sound signal corresponding to The sound pressure ratio between the sound pressure of the ninth sound signal and the sound pressure of the tenth sound signal The experiments on the basis of the relationship between the position information generating apparatus according to claim 18, further comprising a fourth calculating means for calculating the third coefficient. 20. A first sound collecting device, a second sound collecting device, and a first sound collecting device, the first sound collecting device corresponding to a sound emitted from a sound source located within a predetermined range, which is collected by a third sound collecting device. Audio signal,
From the second audio signal and the third audio signal, the first
Of the phase of the second audio signal and the phase of the second audio signal
And the phase of the first audio signal and the third
Calculating a second phase difference with the phase of the audio signal of
The first coefficient and the first coefficient for a detection device that detects the position of the sound source based on the relationship between the phase difference and the first coefficient, and the relationship between the second phase difference and the second coefficient. In the information generation method of the information generation device that generates a second coefficient, a predetermined sound collected by each of the first sound collection device, the second sound collection device, and the third sound collection device The fourth sound signal, the fifth sound signal, and the sixth sound signal corresponding to sounds emitted by the sound sources located at a plurality of experimental positions provided in a range, from the fourth sound signal, A third phase difference between the signal phase and the fifth audio signal, and a fourth phase difference between the fourth audio signal and the sixth audio signal are determined for each of the experimental positions. A first calculating step of calculating; and a relation between the third phase difference and the experimental position. On the basis of,
A second calculating step of calculating the first coefficient and calculating the second coefficient based on a relationship between the fourth phase difference and the experimental position. . 21. A first sound collecting device, a second sound collecting device, and a first sound collecting device, the first sound collecting device corresponding to a sound emitted from a sound source located within a predetermined range, which is collected by a third sound collecting device. Audio signal,
From the second audio signal and the third audio signal, the first
Of the phase of the second audio signal and the phase of the second audio signal
And the phase of the first audio signal and the third
Calculating a second phase difference with the phase of the audio signal of
The first coefficient and the first coefficient for a detection device that detects the position of the sound source based on the relationship between the phase difference and the first coefficient, and the relationship between the second phase difference and the second coefficient. A program of an information generating device for generating a second coefficient, wherein a predetermined sound collected by each of the first sound collecting device, the second sound collecting device, and the third sound collecting device. The fourth sound signal, the fifth sound signal, and the sixth sound signal corresponding to sounds emitted by the sound sources located at a plurality of experimental positions provided in a range, from the fourth sound signal, A third phase difference between the signal phase and the fifth audio signal, and a fourth phase difference between the fourth audio signal and the sixth audio signal are determined for each of the experimental positions. A first calculating step of calculating; and a relationship between the third phase difference and the experimental position. Based on,
A second calculating step of calculating the first coefficient and calculating the second coefficient based on a relationship between the fourth phase difference and the experimental position. A recording medium on which a possible program is recorded. 22. A first sound collecting device, a second sound collecting device, and a first sound collecting device, the first sound collecting device corresponding to the sound emitted from a sound source located within a predetermined range, collected by the third sound collecting device. Audio signal,
From the second audio signal and the third audio signal, the first
Of the phase of the second audio signal and the phase of the second audio signal
And the phase of the first audio signal and the third
Calculating a second phase difference with the phase of the audio signal of
The first coefficient and the first coefficient for a detection device that detects the position of the sound source based on the relationship between the phase difference and the first coefficient, and the relationship between the second phase difference and the second coefficient. A program of an information generating device for generating a second coefficient, wherein a predetermined sound collected by each of the first sound collecting device, the second sound collecting device, and the third sound collecting device. The fourth sound signal, the fifth sound signal, and the sixth sound signal corresponding to sounds emitted by the sound sources located at a plurality of experimental positions provided in a range, from the fourth sound signal, A third phase difference between the signal phase and the fifth audio signal, and a fourth phase difference between the fourth audio signal and the sixth audio signal are determined for each of the experimental positions. A first calculating step of calculating; and a relationship between the third phase difference and the experimental position. Based on,
Calculating a first coefficient, and causing a computer to execute processing including a second calculation step of calculating the second coefficient based on a relationship between the fourth phase difference and the experimental position. Features program.