JP5564873B2 - Sound collection processing device, sound collection processing method, and program - Google Patents

Description

  The embodiments discussed herein relate to audio signal processing techniques.

  As a technique for clearly collecting a speaker's utterance, several techniques for controlling a sound collection directivity by a microphone that collects a speaker's utterance (hereinafter sometimes referred to as “microphone”) are known. .

  In such a technology, a camera having an angle of view equal to the beam width of the directivity of a single sound-collecting directional microphone is prepared, and the microphone and the camera are integrated so that the directivity and the angle of view match. Some cameras are used with a camera-integrated microphone for a camera conference. In this camera-integrated microphone, the face image of the speaker is detected from the image captured by the camera. Then, control is performed so that the camera is directed so that the detected face image is positioned at the center of the captured image, and the center of the sound collection directivity of the microphone (hereinafter sometimes simply referred to as “directivity”) Is directed to the speaker. In this camera-integrated microphone, a technique is known in which the directionality of the microphone is controlled in accordance with the number of face images detected from the captured image. This first technique directs the microphone's directivity to the speaker closest to the microphone when the number is odd, and the speaker closest to the microphone and the second closest speaker when the number is even. The direction of the microphone is directed between the two and the voice of the speaker at the conference is accurately captured.

  Another technique is to control the sharpness of the directivity of the microphone based on the size of the person image included in the captured image on the captured image. In this second technique, when the image of a person is large on the captured image, it is determined that the intention of capturing the image emphasizes the person, and the microphone directivity is set so as to clearly collect the person's utterance. Control to sharpen. On the other hand, if the image of the person is small on the captured image, it is determined that the capturing intention is the entire surrounding environment including the person, and the sound collection of the surrounding environment sound is collected together with the sound collection of the person's utterance In order to take into consideration, the microphone directivity is controlled to be dull.

  In addition, as a technique related to the embodiment discussed in this specification, among the audio signals obtained by collecting sounds from sound sources existing in a plurality of directions, the sound emitted by the sound sources in a predetermined direction is emphasized. A third technique for suppressing ambient noise is known. In this technology, sound from sound sources existing in a plurality of directions is collected by a plurality of microphones (microphone arrays), and a sound signal on the time axis output from each microphone is subjected to, for example, Fourier transform, thereby generating a frequency axis. Each of the above audio signals is converted. For each audio signal on the frequency axis, a phase difference at the same frequency is calculated for each frequency, and the probability that a sound source exists in a predetermined direction is specified for each frequency based on the phase difference. Then, based on this probability, a suppression function that suppresses an audio signal component based on a sound source other than the sound source in the predetermined direction is obtained, and the obtained suppression function is multiplied by the audio signal on the frequency axis. After that, when the multiplication result is subjected to inverse Fourier transform, for example, and restored to a signal on the time axis, an audio signal based on a sound source can be obtained in a predetermined direction.

JP 2009-49734 A JP 2009-65587 A JP 2007-318528 A

  When there are multiple speakers within the microphone collection range, the speaker is not closest to the microphone even if the directionality of the microphone is controlled as in the first technique described above. In some cases, the voice of the speaker cannot be accurately captured. In addition, with the second technique described above, it is difficult to clearly collect a plurality of persons simultaneously speaking.

  The present invention has been made in view of the above-described problems, and a problem to be solved is to clearly collect simultaneous sound generation by a plurality of sound generators.

  One of the sound collection devices described later in this specification includes a sound collection processing unit, an acquisition unit, and a sound collection directivity range setting unit. Among these, the sound collection processing means generates an output sound signal in which sound collection directivity is controlled based on a plurality of sound collection signals collected by a microphone array including a plurality of microphones whose relative positions are fixed. . Further, the acquisition means acquires input of information on the number and arrangement of sounding bodies existing within the sound collection range of the microphone array. Then, the sound collection directivity range setting means sets the direction of the sound collection directivity directed from the microphone array toward the sound generator for each sound generator based on the arrangement information about the sound generator acquired by the acquisition means. In addition, the sound collection directivity range setting means sets the sharpness of the sound collection directivity directed toward the sound generator for each sound generator based on the number information about the sound generator acquired by the acquisition means. In the sound collecting device having these, the sound collecting processing means described above generates and outputs an output sound signal in which the sound collecting directivity is controlled in the direction and sharpness set by the sound collecting directivity range setting means.

  Also, one of the sound collection methods described later in this specification is an output in which sound collection directivity is controlled based on a plurality of sound collection signals collected by a microphone array including a plurality of microphones whose relative positions are fixed. A sound signal is generated.

  In this method, first, input of information on the number and arrangement of sounding bodies existing within the sound collection range of the microphone array is acquired. Next, the direction of sound collection directivity directed from the microphone array to the sounding body is set for each sounding body based on the acquired arrangement information about the sounding body. At the same time, the sharpness of the sound collection directivity directed to each of the sounding bodies is set for each of the sounding bodies based on the number information about the acquired sounding bodies. Then, an output sound signal in which the sound collection directivity is controlled to the set direction and sharpness is generated and output.

  In addition, one of the programs described later in this specification is an output sound whose sound collection directivity is controlled based on a plurality of sound collection signals collected by a microphone array including a plurality of microphones whose relative positions are fixed. A program for causing a computer to generate a signal. By executing the program, the computer causes the computer to perform an acquisition process, a sound collection directivity range setting process, and a sound collection process. Here, the sound collection process is a process of acquiring input of information on the number and arrangement of sounding bodies existing within the sound collection range of the microphone array. The sound collection directivity range setting process is a process of setting the direction of sound collection directivity directed from the microphone array to the sounding body for each sounding body based on the arrangement information about the sounding body acquired in the acquisition process. It is. In addition, the sound collection directivity range setting process includes a process of setting the sharpness of the sound collection directivity directed toward the sound generator for each of the sound generators based on the information on the number of sound generators acquired by the acquisition process. . The sound collection process is a process of generating and outputting an output sound signal in which the sound collection directivity is controlled in the direction and sharpness set by the sound collection directivity range setting process.

  The sound collection device described later in this specification can clearly collect simultaneous sound generation by a plurality of sound generators.

It is a 1st example of a structure of a sound collection system. It is an example of the frequency characteristic of the phase difference range between two microphones of the collected sound signal. It is an example of data which a face position detection system acquires from a picked-up image. It is explanatory drawing (the 1) of the setting of the sharpness of sound collection directivity. It is explanatory drawing (the 2) of the setting of the sharpness of sound collection directivity. It is explanatory drawing (the 3) of the setting of the sharpness of sound collection directivity. It is explanatory drawing (the 1) of extraction of the sounding body excluded from the sound source of an output audio | voice. It is explanatory drawing (the 2) of extraction of the sounding body excluded from the sound source of an output audio | voice. It is a 2nd example of a structure of a sound collection system. This is a configuration of a computer that operates as a sound collection device. It is the flowchart which illustrated the processing content of the control processing performed by a computer.

  First, FIG. 1 will be described. FIG. 1 illustrates a first example of the configuration of a sound collection system. This sound collection system includes a sound collection device 1, a microphone array 2, a camera 3, and a face position detection system 4.

The sound collection device 1 performs signal processing on a sound collection signal from the microphone array 2 and outputs an output sound whose sound collection directivity is controlled.
The microphone array 2 is configured by arranging a plurality of microphones, for example, in a line in the horizontal direction. The relative positions between the microphones constituting the microphone array 2 are fixed.

The sound collection device 1 includes a sound generator information acquisition unit 10, a sound collection directivity range setting unit 20, a sound collection processing unit 30, and an excluded sound generator extraction unit 40.
The sound generator information acquisition unit 10 acquires information on the number and arrangement of sound generators existing in the sound collection range of the microphone array 2. Note that the sound generator may be an animal such as a dog or a cat as long as it emits sound. Furthermore, the sound generator is a sound emitting device equipped with a speaker, or operates as noise that does not have the original purpose of sound generation. It may be a machine that emits sound. However, in the sound collection system of FIG. 1, a sounding person is assumed as a sounding body existing within the sound collection range of the microphone array 2, and the sounding body information acquisition unit 10 performs the sound collection range of the microphone array 2. Input of information on the number and arrangement of humans existing in the face is obtained from the face position detection system 4.

  The sound collection directivity range setting unit 20 uses the direction of the sound collection directivity directed from the microphone array 2 to the sounding body as arrangement information about the sounding body (human in this embodiment) acquired by the sounding body information acquisition unit 10. Based on this, each sound generator (human) is set. The sound collection directivity range setting unit 20 further sets the sharpness of the sound collection directivity directed from the microphone array 2 toward the sounding body (human) by the number of the sounding body (human) acquired by the sounding body information acquisition unit 10. Based on the information, settings are made for each sound generator (human). A method of setting the direction and sharpness of the sound collection directivity performed by the sound collection directivity range setting unit 20 will be described later.

  The sound collection processing unit 30 has a sound collection directivity in the direction and sharpness set by the sound collection directivity range setting unit 20 based on a plurality of sound collection signals collected by each microphone constituting the microphone array 2. A signal of an output sound (output sound) in which is controlled is generated. In the present embodiment, the sound collection processing unit 30 performs the sound collection directivity control as follows using a known method disclosed in Patent Document 3 described above.

The sound collection processing unit 30 includes a directional sound reception processing unit 31 and an output audio signal generation unit 32.
The directional sound reception processing unit 31 first performs analog-digital conversion on each of the plurality of sound collection signals collected by the microphone array 2 to obtain time-domain sound collection signal data. Next, by applying time-frequency conversion such as fast Fourier transform to the collected sound signal data, frequency spectrum data of each collected sound signal is obtained.

  Next, the directional sound reception processing unit 31 calculates the phase difference of the spectrum from the frequency spectrum data of each other collected sound signal when the frequency spectrum data of one of the collected sound signals is used as a reference. A process for calculating each spectral frequency is performed.

  Next, the directivity sound reception processing unit 31 obtains a weight value to be given to the frequency spectrum of the collected sound signal in order to obtain the sound collection directivity having the direction and sharpness set by the sound collection directivity range setting unit 20. Then, the weighting is performed by multiplying the frequency spectrum of the reference sound collection signal described above by the weighting value for each spectrum frequency. The weighting value for each spectrum frequency is obtained as follows, for example.

  First, the directivity sound receiving processing unit 31 uses the sound collection directivity direction setting and the sharpness information set by the sound collection directivity range setting unit 20, and the sound that has arrived from the sound collection directivity range falls within the microphone array 2. The frequency characteristics of the phase difference range between the microphones of the collected sound signal that can be generated when the sound is picked up by the above are obtained.

Here, FIG. 2 will be described. FIG. 2 is an example of frequency characteristics of the phase difference range between two microphones of the collected sound signal. Sounds coming from within the sound collection directivity range where the sharpness is a beam width of ± θ _def are obtained by the two microphones. The frequency characteristic of the phase difference range between the microphones of the collected sound signal that can be generated when the sound is collected is shown. In FIG. 2, the horizontal axis represents the frequency of the sound emitted from the sound source, and the vertical axis represents the phase difference between the two microphones when the sound is collected. The relationship between the frequency and the phase difference indicated by the frequency characteristics can be calculated geometrically using the direction angle of the sound source when the center point of the arrangement positions of the two microphones is the center as a parameter.

  In order to obtain this frequency characteristic, for example, a table showing the relationship between the sharpness of the sound collection directivity and the frequency characteristic of the phase difference range between the microphones of the sound collection signal is obtained for each direction of the sound collection directivity. It is also possible to create a database in advance and store it in the directivity received sound processing unit 31. In this case, the directivity sound reception processing unit 31 refers to this database and reads out the information associated with the information set by the sound collection directivity range setting unit 20 from the table, thereby The frequency characteristic of the phase difference range is obtained.

  Next, a weighting value based on the phase difference of each spectrum to be given to the frequency spectrum of the collected sound signal is set for each frequency of the spectrum. This weighting value given to each spectrum is obtained as follows.

First, referring to the frequency characteristic of the phase difference range of the collected sound signal obtained earlier, the phase difference range at the frequency of the spectrum to which the weighting is applied is obtained from the frequency characteristic.
Next, a weighting value is set based on the relationship between the phase difference previously determined for each spectral frequency and the phase difference range. For example, for a spectrum whose phase difference is within the phase difference range and close to a predetermined value from the center of the range, this weighting value is set to “1.0” and the phase difference is within the phase difference range. For the outer spectrum, this weighting factor is set to “0.0”. In addition, the phase difference within the phase difference range but separated from the center of the range by more than the predetermined value depends on the distance from the center in the range of “1.0” to “0.0”. For example, linear interpolation is performed, and the weighting value is set so as to be continuous with the above-described setting value at the boundary of the range.

The weighting value given to each spectrum is obtained as described above. This weighting value corresponds to what is referred to as “suppression function” in Patent Document 3.
Note that a table in which the relationship between the phase difference and the phase difference range and the above-described weighting setting value is shown in advance as a database for each spectrum frequency may be stored in the directivity reception processing unit 31. . In this case, the directional sound reception processing unit 31 reads out and sets the phase difference in each spectrum and the weighting value associated with the phase difference range with reference to this database.

  In the present embodiment, the weighted value obtained as described above for the number of phase differences previously obtained is given to the frequency spectrum of the collected sound signal as a reference by obtaining an average for each spectrum frequency. A weight value for each spectral frequency to be obtained is obtained.

The output audio signal generation unit 32 performs inverse transformation (for example, fast Fourier inverse) on the frequency spectrum of the collected sound signal given the above weighting by the directivity reception processing unit 31 with respect to the conversion in the directivity reception processing unit 31. Conversion) to convert to time domain audio signal data and output. This sound signal data is generated based on a plurality of sound pickup signals picked up by the microphone array 2, and the output sound of which the sound pickup directivity is controlled to the direction and sharpness set by the sound pickup directivity range setting unit 20. Signal.
The sound collection processing unit 30 is configured as described above.

The excluded sound generator extracting unit 40 excludes the sound generator of the output sound generated by the sound collection processing unit 30 based on the arrangement information of the sound generator (human in this embodiment) acquired by the sound generator information acquiring unit 10. Extract the pronunciation body (human). When the excluded sound generator extraction unit 40 performs this extraction, the sound collection directivity range setting unit 20 determines the direction and sharpness of the sound collection directivity directed from the microphone array 2 toward the sound generator (human). The setting is made for each of the humans other than those extracted by the excluded sound generator extraction unit 40. A method of extracting excluded sound generators by the excluded sound generator extraction unit 40 will be described later.
The sound collection device 1 includes the above components.

  The camera 3 repeatedly captures images within a sound collection range of the microphone array 2 at a fixed magnification at predetermined time intervals. In the present embodiment, it is assumed that the camera 3 is disposed at substantially the same position as the microphone array 2.

  The face position detection system 4 (hereinafter simply referred to as “detection system 4”) is present within the sound collection range of the microphone array 2 by performing image processing on an image captured by the camera 3. Information on the number and arrangement of sound generators (humans in the present embodiment) is obtained. This information is input to the sound collection device 1 and acquired by the sound generator information acquisition unit 10.

Here, image processing by the detection system 4 will be described.
First, the detection system 4 performs a process of detecting an image of a human face from an image captured by the camera 3. A known technique is used for this face detection method. In the present embodiment, the image of the partial area cut out from the image and each of the face pattern images read out from the face pattern database prepared in advance are collated to calculate the degree of correlation between them. This degree of correlation is calculated comprehensively based on, for example, the face contour, the relative positions of the eyes, nose, and mouth, the color of the face, and the like. If there is an object whose correlation degree is higher than a predetermined value, the partial area is set as a detection result of a human face image. By performing this process over the entire captured image while changing the position and size of the partial area, the number of facial images included in the captured image and the position and size of each facial image in the captured image are determined. To detect. Of these, the detection result of the number of face images included in the photographed image is the number of sounding bodies present in the sound collection range of the microphone array 2 obtained from an image obtained by photographing the sound collection range of the microphone array 2. Information is output from the detection system 4.

  Next, the detection system 4 determines the direction angle from the microphone array 2 to the face represented in the partial area based on the position on the captured image of the partial area (that is, the face image) detected from the captured image. The process which calculates | requires is performed. In order to obtain the direction angle, for example, a table created by actually measuring the relationship between the position of the partial region and the direction angle can be stored in the detection system 4 in advance. In this case, the detection system 4 refers to this table and reads the direction angle associated with the position of the partial area from the table, thereby obtaining the direction angle from the microphone array 2 to the face.

  Next, the detection system 4 performs processing for obtaining the distance from the microphone array 2 to the face represented in the partial area based on the size of the partial area (that is, the face image) detected from the captured image. Do. In order to obtain this distance, for example, a table created by actually measuring the relationship between the size of the partial area and the distance can be stored in the detection system 4 in advance. In this case, the detection system 4 refers to this table and reads the distance associated with the size of the partial area from the table to obtain the distance from the microphone array 2 to the face.

Next, the detection system 4 performs processing for obtaining the orientation of the face based on the positional relationship between the eyes, nose, and mouth represented in the face image detected from the captured image. In this process, for example, based on the positions of both eyes, nose and mouth contained in the face image, from the straight line passing through the nose position and mouth position to the right eye position and the left eye position. First, find the distance. Based on the ratio of the two distances, an angle indicating the face orientation is obtained. In order to obtain this angle, for example, a table created by actually measuring the relationship between the above-mentioned distance ratio and the above-mentioned angle can be stored in the detection system 4 in advance. In this case, the detection system 4 refers to this table and reads the angle associated with the above-mentioned distance ratio obtained based on the captured image from the table, thereby obtaining an angle indicating the orientation of the face. Try to ask.

FIG. 3 shows data acquired from the captured image by the detection system 4 as described above.
FIG. 3 shows a case where there are two persons (person A and person B) within the photographing range of the camera 3 (that is, within the sound collection range of the microphone array 2). Here, the detection system 4, the above processing, the direction angle theta _A human A shown in FIG. 3, the distance d _A, and the angle .theta.2 _A face direction angle of the human B theta _B, the distance d _B and the face angle θ2 _B are obtained.

In addition, instead of the detection system 4 obtaining the direction angle θ _A and the distance d _A of the person _A and the direction angle θ _B and the distance d _B of the person B, the two-dimensional coordinates indicating the arrangement positions of the person A and the person B are shown. The values (X _A , Y _A ) and (X _B , Y _B ) may be obtained from the captured image.

  The detection system 4 obtains the direction angle θ, the distance d, and the face for each person whose face image is included in the photographed image obtained from the image obtained by photographing the sound collection range of the microphone array 2 as described above. Are output as information on the arrangement of sounding bodies existing in the sound collection range.

As described above, the detection system 4 obtains information on the number and arrangement of sounding bodies existing within the sound collection range of the microphone array 2 from the image every time the camera 3 captures an image within the sound collection range. Obtained and output to the sound collecting device 1. The information output from the detection system 4 is acquired by the sound generator information acquisition unit 10 of the sound collection device 1.
The sound collection system of FIG. 1 has the above components.

  Next, a method of setting the direction and sharpness of sound collection directivity performed by the sound collection directivity range setting unit 20 of the sound collection device 1 will be described.

The sound collection directivity range setting unit 20 stores in advance an angle table in which the relationship between the number of persons existing in the sound collection range of the microphone array 2 and the angle indicating the sharpness of the sound collection directivity is set. Yes. In the present embodiment, when the angle table is set, the maximum value (dullest value) θ _MAX (for example, 90 °) of the sound collection directivity is related when there is only one person within the sound collection range. It is assumed that a prescribed value θ _def (for example, 30 °) is associated with two people.

  The sound collection directivity range setting unit 20 firstly includes the number information and the direction angle θ for each person as information on the number and arrangement of sounding bodies (humans) existing in the sound collection range of the microphone array 2. , Distance d, and face angle θ2 are acquired from the sound generator information acquisition unit 10. Here, the operation of the excluded sound generator extraction unit 40 is not considered.

Next, the sound collection directivity range setting unit 20 performs setting processing of the sharpness of the sound collection directivity directed toward each person based on the information on the number of persons. This setting process will be described with reference to FIG.
When only one person exists within the sound collection range of the microphone array 2, the sound collection directivity range setting unit 20 performs the setting process to set the sharpness of the sound collection directivity to the direction of the person. Set the angle to ± θ _MAX centered at. In the example of (1) of FIG. 4, for the person _A whose direction angle θ _A is 0 °, the sound collection directivity range setting unit 20 sets the sharpness of the sound collection directivity to an angle of ± θ _MAX. .

On the other hand, when there are two or more persons in the sound collection range, the sound collection directivity range setting unit 20 performs the setting process to focus the sharpness of the sound collection directivity on the direction toward the person. Set the angle to ± θ _def . In the example of FIG. 4 (2), the sound collection directivity range setting unit 20 for the person A located at the direction angle θ _A (<0) and the person B located at the direction angle θ _B (> 0) The sharpness of the sound collection directivity is set to an angle of ± θ _def around the direction angle.

  In this way, the sound collection directivity range setting unit 20 sets the sharpness of the sound collection directivity directed to each sound generator based on the information on the number of sound generators existing within the sound collection range of the microphone array 2. Do. Since the sound collection processing unit 30 generates an output sound signal in which the sound collection directivity is controlled to the sharpness set by the sound collection directivity range setting unit 20, the sound collection directivity range setting unit 20 performs such setting. By doing so, it is possible for the sound collection device 1 to clearly collect sound simultaneously generated by a plurality of sound generators.

  Note that, by this setting process, the sound collection directivity range setting unit 20 indicates the sharpness of the sound collection directivity directed toward each sound generator (human), and further, the sound generator information acquisition unit 10 acquires the sound from the detection system 4. You may make it set based also on the information of arrangement | positioning about a body (human).

For example, as shown in (1) of FIG. 5, the arrangement interval between the person A and the person B existing within the sound collection range of the microphone array 2 is farther than in the case of (2) of FIG. A part of the sound directivity range may exceed the sound collection possible range of the microphone array 2 in some cases. That is,
θ _A −θ _def <−θ _MAX
θ _B + θ _def > θ _MAX
This is the case. (The distance from the microphone array 2 to the person A and the distance from the microphone array 2 to the person B are the same.) In such a case, the sound collection directivity range setting unit 20 executes the setting process, The azimuth angles α and β indicating the angle range of the sound collection directivity for each of the person A and the person B are set within a range represented by the following mathematical formula.
−θ _MAX <α <θ _A + θ _def
θ _B −θ _def <β <θ _MAX

On the other hand, the arrangement interval between the person A and the person B existing within the sound collection range of the microphone array 2 is closer than the case of (2) in FIG. Some of the sound directivity ranges may overlap. (The distance from the microphone array 2 to the person A is the same as the distance from the microphone array 2 to the person B.) That is,
-Θ _A + θ _B <2θ _def
This is the case. In such a case, the sound collection directivity range setting unit 20 performs the setting process to set the sound collection directivity angle ranges α and β for the person A and the person B, respectively, in the ranges represented by the following mathematical formulas. Within.
θ _A −θ _def <α <(− θ _A + θ _B ) / 2
(−θ _A + θ _B ) / 2 <β <θ _B + θ _def

  As described above, the sound collection directivity range setting unit 20 uses the sharpness of the sound collection directivity directed to each sound generator as well as information on the arrangement interval between sound generators existing within the sound collection range of the microphone array 2. You may make it set based on.

  Furthermore, by this setting process, the sound collection directivity range setting unit 20 determines the sharpness of the sound collection directivity directed to each sounding body based on the distance information between the sounding body and the microphone array 2 as follows. You may make it set.

The sound collection directivity range setting unit 20 stores in advance a reference value d _{def for} the distance between a person existing in the sound collection range of the microphone array 2 and the microphone array 2. Here, when the distance between the person and the microphone array 2 matches the reference distance d _def , the sound collection directivity range setting unit 20 sets the sharpness of the sound collection directivity set in the angle table described above. Is set as it is as the sharpness of the sound collection directivity toward the person.

On the other hand, in the example of FIG. 6, for the sound collection directivity for the person A whose distance to the microphone array 2 is the distance d _A shorter than the reference distance d _def , the sound collection directivity range setting unit 20 The angle is set narrowly in a range of ± θ _def × (d _A / d _def ) centered on the azimuth angle θ _A to _A.

Further, in the example of FIG. 6, for the sound collection directivity of the human B is a longer distance d _B than the distance the reference distance d _def of a microphone array 2, sound collection directivity range setting unit 20, the person Widely set within an angle range of ± θ _def × (d _B / d _def ) centered on the azimuth angle θ _B to _B.

  Note that the sound collection directivity range setting unit 20 sets the sound collection directivity narrower as the distance from the microphone array 2 becomes shorter as described above. Good sound collection is possible when this distance is short. Therefore, it is intended to suppress noise other than the target sound. On the other hand, the longer the distance, the wider the sound collection directivity. The longer the distance, the greater the attenuation due to sound propagation depending on the sound collection frequency band. It is.

Note that the sound collection system according to the present embodiment is not intended to significantly increase the sound collection possible distance, and therefore, control for narrowing the sound collection directivity is not performed in order to extend the sound collection possible distance.
The setting of the direction and sharpness of sound collection directivity by the sound collection directivity range setting unit 20 is performed as described above.

Next, a method of extracting sound generators excluded from the sound source of the output sound generated by the sound collection processing unit 30 performed by the excluded sound generator extraction unit 40 will be described.
First, the excluded sound generator extraction unit 40 uses the direction angle θ, the distance d, and the distance d for each person described with reference to FIG. 3 as information on the arrangement of sound generators (humans) existing within the sound collection range of the microphone array 2. And the data of the angle θ2 of the face are acquired from the sound generator information acquisition unit 10.

  Next, the excluded sound generator extraction unit 40 performs an extraction process for extracting a sound generator (human) to be excluded from the sound source of the output sound generated by the sound collection processing unit 30 based on the information on the arrangement. This extraction process will be described.

  First, FIG. 7 will be described. FIG. 7 represents a state in which the person B is moving among two persons (person A and person B) within the photographing range of the camera 3 (that is, within the sound collection range of the microphone array 2).

In the case of the example in FIG. 7, the value of the information about the person B in the arrangement information acquired by the sound generator information acquisition unit 10 from the detection system 4 every time the camera 3 performs shooting changes. Exclusion sounding body extraction unit 40, the variation of this value, more specifically, determined from the direction angle theta _B and the distance d _B of human B, the amount of change in position of the person B at the time of shooting of each image ( That is, the movement distance is calculated. If the amount of change exceeds a predetermined threshold, it is determined that it is difficult to clearly collect the utterance by the person B, and the person B is used as a sounding body to be excluded from the output sound source. To extract.

  As described above, in the example of FIG. 7, the excluded sounding body extraction unit 40 is based on the amount of change in the sounding body arrangement information acquired by the sounding body information acquisition unit 10 from the sound source of the output sound signal. Extract phonetics to exclude.

  Next, FIG. 8 will be described. FIG. 8 shows that among two persons (person A and person B) within the shooting range of the camera 3 (that is, within the sound collection range of the microphone array 2), the face of the person B faces the camera 3 (that is, the microphone array 2). On the other hand, it represents a state of facing sideways.

In the case of the example of FIG. 8, attention is paid to the face angle θ 2 _B of the person B. If this value is outside a predetermined threshold range that can be said to be facing the camera 3 (that is, the microphone array 2), it is difficult to clearly collect the utterance by the person B. The person B is extracted as a sounding body to be determined and excluded from the sound source of the output sound.

  As described above, in the example of FIG. 8, the excluded sounding body extraction unit 40 indicates the orientation of the human face in the information on the arrangement of the human being that is the sounding body acquired by the sounding body information acquisition unit 10. Based on the information, a sounding body to be excluded from the sound source of the output audio signal is extracted.

  The excluded sound generator extraction unit 40 notifies the sound collection directivity range setting unit 20 of the sound generator information extracted as described above. The sound collection directivity range setting unit 20 sets the direction and sharpness of the sound collection directivity directed from the microphone array 2 to the sounding body for the sounding bodies other than those extracted by the excluded sounding body extraction unit 40. Or it sets based on the information of arrangement | positioning.

It should be noted that the excluded sound generator extraction unit 40 may use other methods as a method for extracting sound generators to be excluded from the sound source of the output sound signal.
For example, when the detection system 4 outputs information on the presence or absence of mouth movements in the human face image detected from the captured image, the sound collection directivity range setting unit 20 determines the direction of the sound collection directivity described above. The sharpness can be set based on the information on the presence or absence of the mouth movement.

  For example, the detection system 4 performs the process of extracting the contour shape of the mouth (lips) from the face image detected from the photographed image with respect to each photographed image photographed by the camera 3 at a time interval. A process for calculating the amount of change in shape is performed. When the amount of change exceeds a predetermined threshold value, it is determined that the mouth has movement, and when the amount of change is less than the threshold value, the mouth is determined to have no movement. The detection system 4 outputs the determination result information on the movement of each person's mouth thus detected from the captured image to the sound collection device 1. This information output from the detection system 4 is acquired by the sound generator information acquisition unit 10 of the sound collection device 1.

  Based on the determination result information acquired by the sound generator information acquisition unit 10, the excluded sound generator extraction unit 40 regards a person whose mouth is determined not to move as a person who does not speak, and outputs an output sound signal. As a sound generator to be excluded from the sound source. The sound collection directivity range setting unit 20 sets the direction and sharpness of the sound collection directivity directed from the microphone array 2 to the sounding body for the sounding bodies other than those extracted by the excluded sounding body extraction unit 40. Or it sets based on the information of arrangement | positioning.

  As described above, the excluded sound generator extraction unit 40 is excluded from the sound source of the output sound signal generated by the sound collection processing unit 30 based on the information on the presence or absence of the movement of the human mouth acquired by the sound generator information acquisition unit 10. The sounding body to be played may be extracted.

In addition, it can also comprise so that the determination process of each person's mouth movement mentioned above may be performed by the excluded sounding body extraction part 40 instead of the detection system 4. FIG.
The sound collection system configured as shown in FIG. 1 enables clear sound collection of simultaneous sound generation by a plurality of sound generators by operating each component as described above.

  Next, FIG. 9 will be described. FIG. 9 illustrates a second example of the configuration of the sound collection system. In FIG. 9, components that perform the same operations as those in the first example illustrated in FIG. 1 are denoted by the same reference numerals, and detailed description thereof is omitted.

  Similar to the first example illustrated in FIG. 1, the second example of the sound collection system includes a sound collection device 1, a microphone array 2, a camera 3, and a face position detection system 4. However, in the second example, the sound collection processing unit 30 in the sound collection device 1 includes a sound generation detection unit 33 between the directional sound reception processing unit 31 and the output sound signal generation unit 32. This is different from the first example.

  The sound generation detection unit 33 detects the presence or absence of sound generation by each of the sounding bodies existing within the sound collection range of the microphone array 2 and notifies the output sound signal generation unit 32 of the detection result. The output audio signal generation unit 32 generates an output audio signal using only the sound generators that are detected within the sound collection range of the microphone array 2 and whose sound generation is detected by the sound generation detection unit 33 as sound sources.

The pronunciation detection unit 33 will be further described. The sound generation detection unit 33 detects the presence or absence of sound generation by each sound generator based on the amplitude level of a predetermined frequency band in the output sound.
As described above, the directivity sound reception processing unit 31 outputs the frequency spectrum of the collected sound signal given the weight for obtaining the sound collection directivity having the direction and sharpness set by the sound collection directivity range setting unit 20. Then, the output audio signal generation unit 32 converts this into audio signal data in the time domain. Therefore, the frequency spectrum output by the directional sound reception processing unit 31 is the frequency spectrum of the output audio signal output from the output audio signal generation unit 32.

  The sound generation detection unit 33 adds the levels of the spectrum included in the predetermined frequency band in the frequency spectrum of the signal of the output sound, and obtains the total value as the amplitude level of the predetermined frequency band in the output sound. Then, it is determined whether or not the amplitude level exceeds a predetermined threshold value. If the amplitude level exceeds the predetermined threshold value, it is determined that there is a pronunciation by the sounding body. If not, the sounding by the sounding body is not determined. Judge that there is no.

  In the present embodiment, the frequency band for obtaining the amplitude level is a frequency band of voices produced by humans (around 300 to 3400 Hz). Instead of this, the frequency band of the first formant (in the vicinity of 300 to 1000 Hz) in the frequency spectrum of the uttered sound by humans may be used.

  In addition, the sound generation detection unit 33 can detect the presence or absence of sound generation by each sound generator based on the amplitude level of a predetermined frequency band in the output sound as follows.

  For example, the sound generation detection unit 33 extracts a spectrum that is equal to or greater than a predetermined value from the frequency spectrum, and adds the extracted spectra to obtain a total value. Then, it is determined whether or not the total value exceeds a predetermined threshold value. If the total value exceeds the predetermined threshold value, it is determined that there is a pronunciation by the sounding body. If not, the sounding by the sounding body is not determined. Judge that there is no. It is also possible to detect the presence or absence of pronunciation by each of the sounding bodies by the sounding detection unit 33 in this way.

  Alternatively, the sound generation detection unit 33 obtains the maximum value of the spectrum in this frequency spectrum. Then, it is determined whether or not the maximum value exceeds a predetermined threshold value. If the maximum value is exceeded, it is determined that there is a pronunciation by the sounding body. If not, the sounding by the sounding body is not determined. Judge that there is no. It is also possible to detect the presence or absence of pronunciation by each of the sounding bodies by the sounding detection unit 33 in this way.

  The sound generation detection unit 33 notifies the output sound signal generation unit 32 of the determination result of the presence or absence of sound generation by the sounding body determined as described above. Based on the determination result notified from the sound generation detection unit 33, the output sound signal generation unit 32 outputs a signal of the output sound when it is determined that there is sound generation by the sounding body, and determines that there is no sounding by the sounding body. Stops outputting the output audio signal when

Note that the sound generation detection unit 33 may be silent instead of stopping the output of the output sound signal. Further, in order to reduce a sense of incongruity due to the sudden occurrence of a silent part, instead of silence, a predetermined level of white noise may be output, or the steady state that this sound collection system generates constantly. Noise may be output.
The sound collection system configured as shown in FIG. 9 operates as described above.

  The operation of the sound collection device 1 in the sound collection system illustrated in each of FIGS. 1 and 9, that is, the output sound whose sound collection directivity is controlled based on a plurality of sound collection signals collected by the microphone array 2. The signal generation operation can also be performed by a computer.

First, FIG. 10 will be described. FIG. 10 illustrates the configuration of a computer 50 that causes the sound collection device 1 to operate.
The computer 50 includes an MPU 51, a ROM 52, a RAM 53, a hard disk device 54, an input device 55, a display device 56, an interface device 57, and a recording medium drive device 58. These components are connected via a bus 59, and various data can be exchanged under the management of the MPU 51.

An MPU (Micro Processing Unit) 51 is an arithmetic processing unit that controls the operation of the entire computer 50.
A ROM (Read Only Memory) 52 is a read-only semiconductor memory in which a predetermined basic control program is recorded in advance. The MPU 51 reads out and executes this basic control program when the computer 50 is activated, thereby enabling operation control of each component of the computer 50.

  A RAM (Random Access Memory) 53 is a semiconductor memory that can be written and read at any time and used as a working storage area as needed when the MPU 51 executes various control programs.

  The hard disk device 54 is a storage device that stores various control programs executed by the MPU 51 and various data. The MPU 51 can perform control processing described later by reading and executing a predetermined control program stored in the hard disk device 54. In the present embodiment, the relationship between the sharpness of the sound collection directivity (angle value) and the frequency characteristics of the phase difference range between the microphones of the sound collection signal is shown for each direction (direction angle) of the sound collection directivity. It is assumed that a database of stored tables is stored in the hard disk device 54 in advance. In addition, it is assumed that a database of a table for each spectral frequency indicating the relationship between the phase difference and the phase difference range and the above-described weighting setting value is also stored in the hard disk device 54 in advance.

  The input device 55 is, for example, a keyboard device or a mouse device. When operated by a user of the computer 50, the input device 55 acquires input of various information from the user associated with the operation content, and acquires the acquired input information. Is sent to the MPU 51.

The display device 56 is, for example, a liquid crystal display, and displays various texts and images according to display data sent from the MPU 51.
The interface device 57 manages the exchange of various data with various devices connected to the computer 50. More specifically, reception of data sent from the detection system 4, analog-digital conversion of the collected sound signal output from each of the microphones constituting the microphone array 2, and the converted collected sound signal data Temporary buffering, transmission of output audio data to subsequent devices, etc.

  The recording medium driving device 58 is a device that reads various control programs and data recorded on the portable recording medium 60. The MPU 51 can read out and execute a predetermined control program recorded on the portable recording medium 60 via the recording medium driving device 58 to perform various control processes described later. Examples of the portable recording medium 60 include a CD-ROM (Compact Disc Read Only Memory) and a DVD-ROM (Digital Versatile Disc Read Only Memory).

  In order to operate such a computer 50 as the sound collection device 1, first, a control program for causing the MPU 51 to perform processing contents of a control process described later is created. The created control program is stored in advance in the hard disk device 54 or the portable recording medium 60. Then, a predetermined instruction is given to the MPU 51 to read and execute this control program. By doing so, the MPU 51 functions as the sound generator information acquisition unit 10, the sound collection directivity range setting unit 20, the sound collection processing unit 30, and the excluded sound generator extraction unit 40. Functions can be provided.

Next, FIG. 11 will be described. FIG. 11 is a flowchart illustrating the processing contents of the control processing performed by the MPU 51 in the computer 50 of FIG.
In FIG. 11, when the execution of this control process is started, first, in S101, an angle defining the relationship between the number of sounding bodies within the sound collection range of the microphone array 2 and the angle indicating the sharpness of the sound collection directivity. Initial setting processing of the table and the maximum value of the sound collection directivity sharpness and the reference distance is performed. In this processing, the angle table, the maximum value θ _MAX of the sound collection directivity, and the reference value d _def of the distance between the sound generator and the microphone array 2 are acquired from the input device 55 and the hard disk device 54. The process to store in is performed. This process is a process for the operation as the sound collection directivity range setting unit 20.

  Next, in S102, it is determined whether or not the interface device 57 has received the data output from the detection system 4 and representing the number and arrangement information of the sounding bodies existing within the sound collection range of the microphone array 2. A determination process is performed. When the MPU 51 determines that the data has been received (when the determination result is Yes), the MPU 51 advances the process to S103. On the other hand, when the data is not received (when the determination result is No), the MPU 51 ends the control process of FIG.

  Next, in S103, sound generator information acquisition processing is performed. In this process, the interface device 57 receives the data from the detection system 57 representing the number and arrangement information of the sounding bodies existing within the sound collection range of the microphone array 2 from the interface device 57, and the RAM 53 This is a process of storing in a predetermined area. This process is a process for the operation as the sound generator information acquisition unit 10. When the detection system 4 outputs data representing the presence / absence of movement of the mouth of a human being who is a sounding body, the MPU 51 performs processing for storing this data in a predetermined area of the RAM 53 as well. .

  Next, in S104, an excluded sound generator extraction process is performed. This process is a process of extracting a sounding body to be excluded from the sound source of the output sound generated by the sound collection processing unit 30 based on the information on the arrangement of the sounding body acquired by the sounding body information acquisition unit 10. This process is a process for the operation as the excluded sound generator extraction unit 40.

  In this process, first, a process of reading information on the number and arrangement of sound generators stored in a predetermined area of the RAM 53 is performed. Then, based on the read information, a calculation process of the change amount of the arrangement position of the sounding body or an acquisition process of the face direction of each person who is the sounding body is performed. Then, a process for determining whether or not the amount of change exceeds a predetermined threshold value or a process for determining whether or not the face orientation is outside the predetermined threshold range is performed. Note that the amount of change in the arrangement position of the sound generators is the information on the arrangement stored in the RAM 53 in the process of S103 executed most recently and the arrangement stored in the RAM 53 in the process of S103 executed earlier. Calculate from information.

When information on the presence or absence of the movement of the human mouth, which is a sounding body, is stored in a predetermined area of the RAM 53, this information is read out and a process for determining the presence or absence of movement of the human mouth is performed.
Here, when the calculation result of the amount of change exceeds a predetermined threshold, when the face orientation is outside the predetermined threshold range, or when there is no movement of the human mouth, such a case A process of extracting the sounding body corresponding to is excluded from the sound source of the output sound signal.

  Next, in S105, a target sounding body determination process is performed. This process is a process for the operation as the sound collection directivity range setting unit 20. In this process, various kinds of sound generator information stored in the predetermined area of the RAM 53 by the latest process of S103 are updated based on the information on the exclusion of the sound generator obtained by the process of S104. In this update process, the number of sounding bodies out of the various types of sounding body information is subtracted from the number of sounding bodies extracted by the process of S104. Note that the number of target sounding bodies as the subtraction result is further substituted into a variable n. In addition, regarding the information on the arrangement position of the sounding body and the presence / absence of mouth movement, the sounding body extracted by the process of S104 is deleted. Various pieces of information on the target sounding body after this update processing are stored in another predetermined area of the RAM 53.

  Next, in S106, an audio data reading process is performed. This process is also a process for the operation as the sound collection directivity range setting unit 20. In this process, collected sound signal data output from each of the microphones constituting the microphone array 2 that are temporarily buffered by the interface device 57 are read and stored in a predetermined area of the RAM 53 in a lump. It is.

  Next, in S107, processing for determining whether or not the current value of the variable n is a positive value is performed. When the MPU 51 determines that the value of the variable n is a positive value (when the determination result is YES), the MPU 51 advances the process to S108. On the other hand, when the MPU 51 determines that the value of the variable n is not a positive value (when the determination result is NO), the MPU 51 returns the processing to S102 and the next sound pickup buffered by the interface device 57. The process related to the signal data is executed again.

  The subsequent processing from S108 to S113 is the nth target in each sounding body (target sounding body) obtained by removing the sounding body existing in the sound collection range of the microphone array 2 from the sounding body extracted by the processing in S104. This is a process that is performed on the sound generator.

  First, in S108, sound collection directivity range setting processing is performed. This process is also a process for the operation as the sound collection directivity range setting unit 20. In this process, the sharpness of the sound collection directivity directed toward the nth target sounding body is set based on the number of target sounding bodies, and the direction of the sound collection directivity is arranged for the nth target sounding body. This processing is set based on the information.

  In this process, first, information on the number of target sounding bodies and data on the direction angle θ and the distance d for the nth target sounding body (human) are acquired from the RAM 53. Next, a process of acquiring an angle value associated with the number of target sounding bodies is performed with reference to the angle table acquired in the process of S101. The angle value and the direction angle θ represent the sharpness and direction of the sound collection directivity.

  At this time, the sharpness of the sound collection directivity directed to the nth target sounding body is further set as described above based on the arrangement information about the sounding body (human). May be.

In this case, first, a process of acquiring the direction angle data for the target sounding body adjacent to the nth target sounding body from the RAM 53 is performed. Then, by using the obtained direction angle data and the maximum value θ _MAX of the sound collection directivity sharpness acquired in the processing of S101, the n-th target is obtained as described with reference to FIG. Processing for setting the sharpness of the sound collection directivity toward the sounding body is performed.

Furthermore, as described with reference to FIG. 6, processing for setting the sharpness of the sound collection directivity toward the nth target sounding body may be performed. In this case, first, a process for acquiring distance data about the target sounding body adjacent to the nth target sounding body from the RAM 53 is performed. Next, using the obtained distance data and the reference distance d _def acquired in the process of S101, the process of setting the sharpness of the sound collection directivity is performed as described with reference to FIG. Is called.

  Note that the angle value and the direction angle representing the sharpness and direction of the sound collection directivity set by the sound collection directivity range setting process of S108 are stored in a predetermined area of the RAM 53.

Next, in S109, directivity sound reception processing is performed. This process is a process for the operation as the sound collection directivity reception process unit 31 in the sound collection process unit 30.
In this process, first, the collected sound signal data for each microphone stored in the RAM 53 by the process of S106 is read out, and time-frequency conversion (for example, Fourier transform) is performed on each of them, and the frequency spectrum of each collected signal is obtained. Processing to obtain data is performed. Next, a process of calculating a spectrum phase difference with respect to the frequency spectrum data of each other collected sound signal for each spectrum frequency when one frequency spectrum data of the collected sound signals is used as a reference. Done.

  Next, a process of reading the angle value and the direction angle stored in the RAM 53 by the process of S108 is performed. Next, referring to the database in the hard disk device 54, a process of reading out the frequency characteristics of the phase difference range between the microphones of the collected sound signal, which is associated with the read angle value, from the read direction angle table. Is done. And the process which acquires the phase difference range in each spectrum frequency in the frequency spectrum data of each sound collection signal from this table is performed.

  Next, referring to the database in the hard disk device 54, a process of obtaining a weighting value associated with a phase difference and a phase difference range in each spectrum from a table for each spectrum frequency is performed. Then, the weighting is performed by multiplying the frequency spectrum of the reference sound collection signal by the weight value for each spectrum frequency.

  Subsequent processing of S110, S111, and S113 is processing for operation as the sound generation detection unit 33 in the sound collection processing unit 30 of FIG. Therefore, when the sound collecting device 1 of FIG. 1 is realized by the computer 50, the processing of S110, S111, and S113 is not necessary, and the processing of S112, which will be described later, is executed after S109. What is necessary is just to perform the process of below-mentioned S114.

  First, in S110, a sound generation detection level acquisition process is performed. In this process, the level of the spectrum included in the predetermined frequency band described above among the frequency spectra weighted by the directional sound receiving process of S109 is added, and the total value is added to the predetermined frequency band in the output sound. The processing for obtaining the amplitude level of is performed. The amplitude level obtained in this way is treated as a sound generation detection level.

  Next, in S111, a process of determining whether or not the sound generation detection level obtained by the process of S110 exceeds a predetermined value that is a threshold value is performed. Here, when the MPU 51 determines that the pronunciation detection level exceeds the predetermined value (when the determination result is Yes), the MPU 51 proceeds to S112, while the pronunciation detection level does not exceed the predetermined value. (When the determination result is No), the process proceeds to S113.

  In the processes of S110 and S111, the detection of the presence or absence of sound generation by each sound generator can be performed as follows instead of being performed based on the sound detection level obtained as described above.

  For example, in S110, the MPU 51 performs a process of extracting a spectrum that is equal to or greater than a predetermined value from the weighted frequency spectrum and adding the extracted spectra to obtain a total value. In subsequent S111, the MPU 51 performs a process of determining whether or not the total value exceeds a predetermined threshold value. If it is determined that the sound has been exceeded, it is determined that there is a sound produced by the sounding body, and the process proceeds to S112. If it is determined that the sound has not been exceeded, it is determined that there is no sound by the sounding body. Then, the process proceeds to S113.

  Or in S110, MPU51 performs the process which calculates | requires the maximum value of the spectrum in the frequency spectrum to which weighting was given. In subsequent S111, the MPU 51 determines whether or not the maximum value exceeds a predetermined threshold value. If it is determined that the sound has been exceeded, it is determined that there is a sound produced by the sounding body, and the process proceeds to S112. If it is determined that the sound has not been exceeded, it is determined that there is no sound by the sounding body. Then, the process proceeds to S113.

Even if the processing of S110 and S111 is performed as described above, it is possible to detect the presence or absence of sound generation by each sound generator.
In S112, an output sound generation process is performed. This process is a process for the operation as the output audio signal generation unit 32. In this process, the frequency spectrum of the collected sound signal weighted by the directional sound reception process of S109 is subjected to an inverse transform (for example, a fast Fourier inverse transform) with respect to the conversion performed in the directional sound reception process to obtain a time. A process of converting to the audio signal data of the area and outputting it is performed. When the MPU 51 completes the process of S112, the process proceeds to S114.

  On the other hand, in S113, non-speech processing is performed. This process is a process for stopping the output of the output audio signal. Note that the MPU 51 may perform a process of outputting silence data instead of the process of stopping the output of the output audio signal. Alternatively, the MPU 51 may perform a process of outputting white noise data of a predetermined level, or a process of outputting data corresponding to stationary noise that is regularly generated by the sound collection system. The MPU 51 may perform this.

  Next, in S114, a process of decrementing the value of the variable n, that is, a process of subtracting 1 from the current value of the variable n and substituting the value of the subtraction result into the variable n is performed, and thereafter, the process proceeds to S107. The process is returned, and the process based on the new value of the variable n is executed again.

By causing the MPU 51 to perform the above control processing, the computer 50 in FIG. 10 can function as the sound collection device 1.
In addition, this invention is not limited to embodiment described so far, In an implementation stage, it is possible to variously change and combine in the range which does not change the summary.

  For example, in the above-described embodiment, the camera 3 is arranged at substantially the same position as the microphone array 2, but the camera 3 and the microphone array 2 can be arranged at positions separated from each other. In the case of such an arrangement, for example, a conversion table for converting the positional relationship between the camera 3 and the microphone array 2 is prepared in the sound generator information acquisition unit 10 of the sound collection device 1. Then, the sound generator information acquisition unit 10 refers to the conversion table for the arrangement information of the position, angle, and distance detected by the position detection system 4 from the image captured by the camera 3, and the arrangement at the position of the microphone array 2. What is necessary is just to make it convert into information.

In addition, the following additional remarks are disclosed regarding the embodiment described above.
(Appendix 1)
Sound collection processing means for generating a signal of output sound in which sound collection directivity is controlled based on a plurality of sound collection signals collected by a microphone array including a plurality of microphones whose relative positions are fixed;
Obtaining means for obtaining input of information on the number and arrangement of sounding bodies existing within the sound collection range of the microphone array;
The direction of the sound collection directivity directed from the microphone array to the sounding body is set for each of the sounding bodies based on the arrangement information about the sounding body acquired by the acquisition unit, and the direction toward the sounding body is set. Sound collection directivity range setting means for setting the sharpness of the sound collection directivity for each of the sounding bodies based on the information on the number of the sounding bodies acquired by the acquisition means;
Have
The sound collection processing means generates and outputs an output sound signal in which the sound collection directivity is controlled in the direction and sharpness set by the sound collection directivity range setting means.
A sound collecting device characterized by that.
(Appendix 2)
Note that the information on the number and arrangement of sounding bodies present in the sound collection range of the microphone array acquired by the acquisition means is obtained from an image obtained by photographing the sound collection range of the microphone array. The sound collecting device according to 1.
(Appendix 3)
The sound collection directivity range setting means is based on the information on the number of sound generators acquired by the acquisition means, and the acquisition means acquires the sharpness of the sound collection directivity directed toward the sound generator. 3. The sound collecting device according to appendix 1 or 2, wherein the sound generator is set for each of the sound generators based on arrangement information about the sound generators.
(Appendix 4)
The information on the arrangement of the sound generators on which the sound collection directivity range setting means sets the sharpness of the sound collection directivity directed to the sound generator is the sound generators existing within the sound collection range of the microphone array. The sound collection device according to attachment 3, wherein the sound collection device is information on an arrangement interval between them.
(Appendix 5)
The information about the arrangement of the sounding body on which the sound collection directivity range setting means sets the sharpness of the sound collection directivity toward the sounding body is information on the distance between the sounding body and the microphone array. The sound collecting device according to Supplementary Note 3, wherein the sound collecting device is provided.
(Appendix 6)
The sound collection directivity range setting means sets the sharpness of the sound collection directivity at a narrower angle when the distance is shorter than when the distance between the sounding body and the microphone array is long. The sound collection device according to appendix 5.
(Appendix 7)
Based on the information on the arrangement of the sounding bodies acquired by the acquiring means, further comprising excluded sounding body extracting means for extracting sounding bodies to be excluded from the sound source of the output sound generated by the sound collection processing means,
The sound collection directivity range setting means sets the direction and sharpness of the sound collection directivity based on information about the sound generators other than those extracted by the excluded sound generator extraction means.
The sound collection device according to any one of Supplementary notes 1 to 6, which is characterized by that.
(Appendix 8)
The excluded sound generator extraction unit extracts a sound generator to be excluded from the sound source of the output sound signal generated by the sound collection processing unit based on the amount of change in the information of the sound generator arrangement acquired by the acquisition unit.
The sound collecting device according to appendix 7, characterized by:
(Appendix 9)
The sounding body is human,
The excluded sound generator extraction unit is configured to generate an output sound signal from the sound source generated by the sound collection processing unit based on the human face orientation information of the human arrangement information acquired by the acquisition unit. Extract phonetics to exclude,
The sound collecting device according to appendix 7, characterized by:
(Appendix 10)
The sounding body is human,
The excluded sound generator extraction unit extracts a sound generator to be excluded from the sound source of the output sound signal generated by the sound collection processing unit based on the information on the presence or absence of movement of the human mouth acquired by the acquisition unit. ,
The sound collecting device according to appendix 7, characterized by:
(Appendix 11)
Further comprising sound detection means for detecting the presence or absence of sound generation by a sounding body present within the sound collection range of the microphone array;
The sound collection processing means outputs a signal of the output sound when the sound generation is detected by the sound generation detection means.
The sound collecting device according to any one of Supplementary notes 1 to 10, characterized in that:
(Appendix 12)
The sound collection device according to appendix 11, wherein the sound generation detection means detects presence or absence of sound generation by the sound generator based on an amplitude level of a predetermined frequency band in the output sound.
(Appendix 13)
The sounding body is human,
The predetermined frequency band is set to the frequency band of the first human formant.
The sound collecting device according to appendix 12, characterized by:
(Appendix 14)
The sound collection processing means assigns a weight to obtain the sound collection directivity having the direction and sharpness set by the sound collection directivity range setting means with respect to the frequency spectrum of the sound collection signal collected by the microphone array. By generating a signal of the output sound by converting the frequency spectrum to which the weight is given to time axis information,
The sound detection means detects the presence or absence of sound generation by the sound generator based on the sum total value of the spectrum of the frequency spectrum to which the weight is given and the spectrum is a predetermined value or more.
The sound collecting device according to Supplementary Note 11, wherein
(Appendix 15)
The sound collection processing means assigns a weight to obtain the sound collection directivity having the direction and sharpness set by the sound collection directivity range setting means with respect to the frequency spectrum of the sound collection signal collected by the microphone array. And generating a signal of the output sound by converting the weighted frequency spectrum into a time-domain sound signal,
The sound detection means detects presence or absence of sound generation by the sound generator based on the maximum value of the spectrum in the frequency spectrum to which the weight is given.
The sound collecting device according to Supplementary Note 11, wherein
(Appendix 16)
A sound collection method for generating an output sound signal in which sound collection directivity is controlled based on a plurality of sound collection signals collected by a microphone array including a plurality of microphones whose relative positions are fixed,
Obtaining input of information on the number and arrangement of sounding bodies present within the sound collection range of the microphone array,
The direction of the sound collection directivity directed from the microphone array to the sounding body is set for each of the sounding bodies based on the acquired arrangement information about the sounding body, and the sound collection directivity directed toward the sounding body Setting the sexual sharpness for each of the sounding bodies based on the acquired number of information about the sounding body;
Generating and outputting an output sound signal in which the sound collection directivity is controlled in a set direction and sharpness;
A sound collecting method characterized by that.
(Appendix 17)
A program for causing a computer to generate an output sound signal in which sound collection directivity is controlled based on a plurality of sound collection signals collected by a microphone array having a plurality of microphones whose relative positions are fixed. By causing the computer to execute,
An acquisition process for acquiring input of information on the number and arrangement of sounding bodies existing within the sound collection range of the microphone array;
The direction of the sound collection directivity directed from the microphone array to the sounding body is set for each of the sounding bodies based on the arrangement information about the sounding body acquired in the acquisition process, and is directed to the sounding body. A sound collection directivity range setting process for setting the sharpness of the sound collection directivity for each of the sound generators based on the information on the number of the sound generators acquired by the acquisition process;
Sound collection processing for generating and outputting an output sound signal in which the sound collection directivity is controlled to the direction and sharpness set by the sound collection directivity range setting processing;
A program for causing the computer to execute.

DESCRIPTION OF SYMBOLS 1 Sound collection device 2 Microphone array 3 Camera 4 Face position detection system 10 Sound body information acquisition part 20 Sound collection directivity range setting part 30 Sound collection process part 31 Directional sound reception process part 32 Output sound signal generation part 33 Sound detection detection 40 Excluded sound generator extraction unit 50 Computer 51 MPU
52 ROM
53 RAM
54 Hard Disk Device 55 Input Device 56 Display Device 57 Interface Device 58 Recording Medium Drive Device 59 Bus 60 Portable Recording Medium

Claims (7)

Sound collection processing means for generating a signal of output sound in which sound collection directivity is controlled based on a plurality of sound collection signals collected by a microphone array including a plurality of microphones whose relative positions are fixed;
Obtaining means for obtaining input of information on the number and arrangement of sounding bodies existing within the sound collection range of the microphone array;
The direction of the sound collection directivity directed from the microphone array to the sounding body is set for each of the sounding bodies based on the arrangement information about the sounding body acquired by the acquisition means, and the sound generation from the microphone array The sharpness of the sound collection directivity toward the body is based on the information on the number of the sounding bodies acquired by the acquisition means, and further, the sound generation existing within the sound collection range of the microphone array acquired by the acquisition means Based on the arrangement information about the sounding body, which is composed of at least one of the information on the arrangement interval between the bodies or the distance information between the sounding body and the microphone array , the storage set for each of the sounding bodies. Sound directivity range setting means;
Have
The sound collection processing means generates and outputs an output sound signal in which the sound collection directivity is controlled in the direction and sharpness set by the sound collection directivity range setting means.
A sound collecting device characterized by that. Based on the information on the arrangement of the sounding bodies acquired by the acquiring means, further comprising excluded sounding body extracting means for extracting sounding bodies to be excluded from the sound source of the output sound generated by the sound collection processing means,
The sound collection directivity range setting means sets the direction and sharpness of the sound collection directivity based on information about the sound generators other than those extracted by the excluded sound generator extraction means.
The sound collection device according to claim 1. The excluded sound generator extraction unit extracts a sound generator to be excluded from the sound source of the output sound signal generated by the sound collection processing unit based on the amount of change in the information of the sound generator arrangement acquired by the acquisition unit.
The sound collecting device according to claim 2 , wherein The sounding body is human,
The excluded sound generator extraction unit is configured to generate an output sound signal generated by the sound collection processing unit from a sound source based on the human face orientation information of the human arrangement information acquired by the acquisition unit. Extract phonetics to exclude,
The sound collecting device according to claim 3 . Further comprising sound detection means for detecting the presence or absence of sound generation by a sounding body present within the sound collection range of the microphone array;
The sound collection processing means outputs a signal of the output sound when the sound generation is detected by the sound generation detection means.
The sound collecting device according to any one of claims 1 to 4 , wherein the sound collecting device is provided. A sound collection method for generating an output sound signal in which sound collection directivity is controlled based on a plurality of sound collection signals collected by a microphone array including a plurality of microphones whose relative positions are fixed,
Obtaining input of information on the number and arrangement of sounding bodies present within the sound collection range of the microphone array,
The direction of the sound collection directivity directed from the microphone array to the sounding body is set for each of the sounding bodies based on the acquired arrangement information about the sounding body, and is directed from the microphone array to the sounding body. the該収sound directivity of sharpness, based rather with the number of information about the acquired emitting sound body, further, information of arrangement intervals of the sound bodies present in the sound pickup range of the acquired said microphone array Or each of the sound generators based on the information on the arrangement of the sound generators, comprising at least one of the distance information between the sound generators and the microphone array ,
Generating and outputting an output sound signal in which the sound collection directivity is controlled in a set direction and sharpness;
A sound collecting method characterized by that. A program for causing a computer to generate an output sound signal in which sound collection directivity is controlled based on a plurality of sound collection signals collected by a microphone array having a plurality of microphones whose relative positions are fixed. By causing the computer to execute,
An acquisition process for acquiring input of information on the number and arrangement of sounding bodies existing within the sound collection range of the microphone array;
The direction of the sound collection directivity directed from the microphone array to the sounding body is set for each of the sounding bodies based on the arrangement information about the sounding body acquired in the acquisition process, and the microphone array the該収sound directivity of sharpness directing the sounding body, based rather with the number of information about the said mounting obtain processed by the obtained emitting sound body, further, the sound collection range of the microphone array obtained by said mounting obtain treatment Based on the information on the arrangement of the sound generators, which is at least one of the information on the arrangement interval between the sound generators existing in the information and the information on the distance between the sound generators and the microphone array . Sound collection directivity range setting processing to be set for each,
Sound collection processing for generating and outputting an output sound signal in which the sound collection directivity is controlled to the direction and sharpness set by the sound collection directivity range setting processing;
A program for causing the computer to execute.