JP7403778B2 - Sound source direction identification device - Google Patents

Description

The present invention relates to a sound source direction identification device and the like.

Patent Document 1 describes a sound source direction estimation device that includes three microphones and estimates a sound source direction. Specifically, a method is disclosed in which the horizontal angle of the sound source direction is calculated using three arrival time differences.

Japanese Patent Application Publication No. 2015-161659

As a method of identifying a sound source using three microphones, the method disclosed in Patent Document 1 may not be able to accurately identify the direction.

The present application has been proposed in view of various problems such as those mentioned above, and uses a method different from the conventional technology to identify the sound source direction by using, for example, three microphones to accurately identify the direction of the sound source. The purpose is to provide equipment, etc.
The purpose of the invention of the present application is not limited to this, and we intend to acquire rights through divisional applications, amendments, etc. for structures that aim to obtain effects from the parts of the structure disclosed in this specification, drawings, etc. . For example, the present specification discloses a problem in which the phrase ``can be done'' is replaced with ``the problem is.'' Each of the problems is described as independent, and we intend to obtain rights for the structure to solve these problems independently through divisional applications, amendments, etc. Even if the problem is implicitly understood from the description of the specification, the present applicant has the intention of claiming a part of the structure described in the specification in an amendment or divisional application.

(1) The sound source direction identification device of the present application determines the position of the sound source in the triangle based on three microphones arranged at the vertices of the triangle and the difference in arrival time of sound from the sound source to each of the three microphones. a specifying unit that specifies a sound source direction from a position projected onto a plane containing the triangle along a direction perpendicular to the plane containing the triangle to a reference position located inside an area surrounded by the triangle on the plane. It is characterized by In this way, the direction of the sound source can be specified using three microphones.

For example, if the midpoint of a line segment connecting two microphones is used as a reference position, and a plane including the two microphones is assumed, the two microphones inevitably impose restrictions on identifying the direction of the sound source.
As shown in Figure 1, a plane is assumed that includes microphones MIC (Ach) (hereinafter referred to as MICa), MIC (Bch) (hereinafter referred to as MICb) arranged with an interval Dab, and a sound source. do. In FIG. 1, arrows indicate the direction of the sound source from the position of the sound source (hereinafter referred to as the sound source position) toward the reference position. Also, the sound source direction is determined by the angle between the sound source direction and the perpendicular to the line connecting the microphones MICa and MICb (denoted as 0° in Figure 1), which passes through the midpoint of the line connecting the microphones MICa and MICb. (hereinafter referred to as the sound source angle). In the following description, a perpendicular line may be referred to as a 0° line.
Assuming that the sound is a plane wave, the distance difference Ddiff, which is the difference between the distance from the sound source position to the microphone MICa and the distance from the sound source position to the microphone MICb, is determined by the hypotenuse being the line segment connecting the microphones MICa and MICb, and one side being This is the length of the other side of the right triangle that is perpendicular to the direction of the sound source. Therefore, the distance difference Ddiff is expressed by the following (Equation 1). Regarding the distance from the sound source position, microphone MICa is longer than microphone MICb by a distance difference Ddiff.
Ddiff=Dab×sinθ...(Formula 1)
When (Formula 1) is transformed, the angle θ is expressed by the following (Formula 2).
θ=arcsin(Ddiff/Dab)...(Formula 2)
Furthermore, using the sound speed Vs, the relationship between the distance difference Ddiff and the arrival time difference Tdiff, which is the difference between the sound arrival time at the microphone MICa and the sound arrival time at the microphone MICb, is expressed by the following (Equation 3). .
Tdiff=Ddiff/Vs...(Formula 3)
When (Formula 3) is transformed, the distance difference Ddiff is expressed by the following (Formula 4).
Ddiff=Vs×Tdiff...(Formula 4)
Substituting (Formula 4) into (Formula 2) results in the following (Formula 5).
θ=arcsin(Vs×Tdiff/Dab)...(Formula 5)
In (Equation 5), if the interval Dab and the sound speed Vs are known, the angle θ can be calculated by determining the arrival time difference Tdiff by measurement or the like.
However, as shown in FIG. 2, there are two sound source directions with the same arrival time difference Tdiff for the microphones MICa and MICb: the direction indicated by the angle θ and the direction indicated by the angle θ'. Here, the angle θ' is an angle obtained by subtracting the angle θ from 180°. Therefore, it is not possible to specify whether the sound source angle is the angle θ or the angle θ′ using only the arrival time difference Tdiff. When one direction is called the front direction and the other direction is called the back direction with respect to the line passing through the microphones MICa and MICb, if there is a sound source in the front direction, the front direction is actually the real image and the back direction is the virtual image. However, it is not possible to distinguish which image is a real image based on the arrival time difference Tdiff alone.

On the other hand, according to the configuration of the present application, which is based on the difference in arrival time between three microphones arranged at the vertices of a triangle that are not on the same straight line, it is possible to specify the direction of the sound source. For example, on a plane containing the microphones MICa, MICb and the sound source, a third microphone, microphone MIC (Cch) (hereinafter referred to as , MICc) are placed. In this case, if the sound reaches the microphone MICc earlier than the microphones MICa, MICb, the sound source direction is sound 2 whose direction is angle θ', and if the sound arrives at the microphone MICc later than the microphones MICa, MICb, then It can be specified that the sound source direction is sound 1 at an angle θ.
In other words, the angle θ is subtracted from the angle θ and 180°, which are calculated by substituting the arrival time difference Tca, which is the difference between the arrival time of the sound to the microphone MICa and the arrival time of the sound to the microphone MICb, into (Equation 5). Of the angle θ', the arrival time difference Tca is the difference between the time the sound arrives at the microphone MICa and the time the sound arrives at the microphone MICc, or the time the sound arrives at the microphone MICb and the time the sound arrives at the microphone MICc. Based on the arrival time difference Tbc, which is the difference between the two, it is possible to identify either one as the sound source angle. In the following description, when distinguishing between which set of microphones MICa to MICc the arrival time differences are, the arrival time differences are written as Tab, Tbc, and Tca, and when they are collectively referred to, the arrival time differences are written as Tdiff.
Here, it is assumed that the arrival time difference Tdiff is a time calculated by subtracting the shorter time from the longer time required for the sound to reach the two microphones. Of course, depending on the length of time required for sound to reach the two microphones, it is possible to specify which of the two microphones is farther from the sound source.

In the following explanation, in order to distinguish between the angles θ and θ′ shown in FIG. The side without the microphone is called the "front", and the side with the remaining microphone is called the "back". For example, in FIG. 2, in a pair of microphones MICa and MICb, when microphone MICc is in the position shown in FIG. 2, the side where sound 1 is located is the "front" side, and the side where sound 2 is located is the "back". In the following description, one set of microphones may be described as a microphone set; for example, one set of microphones MICa and MICb may be referred to as microphone set MICa⇔MICb.

Now, specify which of the six regions the sound source direction falls in, passing through the orthocenter of triangle ABC with the positions of the three microphones as vertices, and having three lines parallel to each side of the triangle as boundary lines. By doing so, it is possible to efficiently distinguish between the front and the back of the sound source direction in each of the three sets of microphones.
As shown in FIG. 3, the boundary lines are three parallel lines PLab, PLbc, and PLca that pass through the orthocenter of triangle ABC whose vertices are the positions of microphones MICa to MICc and are parallel to each side of triangle ABC. The six regions are referred to as regions 1 to 6. Here, the orthocenter of triangle ABC is taken as the reference position. The direction from the sound source position to the reference position is the sound source direction, and an example of the sound source direction is shown by an arrow in FIG. Note that since sound is assumed to be a plane wave, the direction of the sound source can be considered to be moved arbitrarily on the plane including the microphones MICa to MICc. The areas located on the "front" of the microphone set MICa⇔MICb are regions 1 to 3, and the regions located on the "back" are regions 4 to 6. Areas located on the "front" of the microphone group MICb⇔MICc are regions 3 to 5, and regions 1, 2, and 6 are located on the "back". The areas located on the "front" of the microphone group MICc⇔MICa are regions 1, 5, and 6, and the regions located on the "back" are regions 2 to 4. Therefore, for example, if it is specified that the sound source direction is in area 3, then the microphone group MICa⇔MICb is the “table”, the microphone group MICb⇔MICc is the “table”, and the microphone group MICc⇔MICa is the “table”. can be identified efficiently.

By the way, as shown in Fig. 4, in addition to parallel lines PLab, PLbc, and PLca passing through the orthocenter O of triangle ABC and parallel to each side of triangle ABC, there are parallel lines drawn from each of vertices A to C to each opposite side. In 12 regions whose boundaries are a total of six perpendicular lines PLa, PLb, and PLc, the order of arrival time difference Tdiff is automatically determined. For the purpose of explanation, the 12 areas will be referred to as follows. With the orthocenter O as the base point, the area from the orthocenter O to the perpendicular line PLa on the apex A side clockwise to the parallel line PLca is referred to as a region R1a, and from the region R1a clockwise in order, the regions R1b, R2a, R2b, R3a, R3b, They are called R4a, R4b, R5a, R5b, R6a, and R6b. Note that the orthocenter O is the intersection of the perpendicular lines PLa, PLb, and PLc.

Here, for the sake of simplicity, a case where triangle ABC is an equilateral triangle will be explained as an example.
For example, when the sound source position is on the perpendicular line PLa, the arrival time difference Tab between the microphone group MICa⇔MICb is the same as the arrival time difference Tca between the microphone group MICa⇔MICc, and the arrival time difference Tbc between the microphone group MICb⇔MICc is 0. becomes. That is, the arrival time difference Tab, Tca is the maximum, and the arrival time difference Tbc is the minimum. When the sound source positions are on perpendicular lines PLb and PLc, the order of arrival time difference Tdiff is determined in the same way.
Further, for example, when the sound source position is on the parallel line PLab, the arrival time difference Tbc and the arrival time difference Tca are the same, and the arrival time difference Tab is twice the arrival time difference Tbc and the arrival time difference Tca. That is, the arrival time difference Tab becomes the maximum, and the arrival time differences Tbc and Tca become the minimum. This is the difference between the distance from the sound source position to microphone MICa and the distance from the sound source position to microphone MICb.The distance difference DDab is the length of side AB, and the distance from the sound source position to microphone MICb and the distance from the sound source position to microphone MICb are The distance difference DDbc, which is the difference from the distance to MICc, is the distance from vertex B to the midpoint of side AB, and the distance is the difference between the distance from the sound source position to microphone MICc and the distance from the sound source position to microphone MICa. This is because the difference DDca is the distance from the vertex A to the midpoint of the side AB. When the sound source positions are on parallel lines PLbc and PLca, the order of arrival time difference Tdiff is determined in the same way. In the following description, when distinguishing between which set of microphones MICa to MICc, the distance differences are written as DDab, DDbc, and DDca, and when they are referred to collectively, the distance differences are written as Ddiff.
Next, a case where the sound source position is not on a line will be described using FIG. 5, taking as an example a case where the sound source position is in the region R2a and the angle between the sound source direction and the perpendicular line PLc is an angle θ. Note that since the sound source position is in the region R2a, the angle θ is less than 30°.

FIG. 5 is a diagram in which, in order to calculate the distance difference Ddiff for each pair, a right triangle with each side as a hypotenuse is drawn in the triangle ABC whose vertices are the positions of microphones MICa to MICc, as in FIG. 1. . Specifically, the right triangle ABD is a right triangle whose hypotenuse is the side AB and one side is the side BD perpendicular to the sound source direction. Further, the right triangle BCF is a right triangle whose hypotenuse is side BC and one side is side FB orthogonal to the sound source direction. Further, the right triangle CAE is a right triangle in which the side CA is the hypotenuse and one side is the side AE orthogonal to the sound source direction.
Here, the angle of the angle DBA is θ, the angle of the angle FBC is (60°+θ), and the angle of the angle CAE is (60°−θ). The distance difference DDab is the length of the side AD of the right triangle ABD. Further, the distance difference DDbc is the length of the side CF of the right triangle BCF. Further, the distance difference DDca is the length of the side EC of the right triangle CAE.
If the distance between the microphones MICb and MICc is the distance Dbc, and the distance between the microphones MICc and MICa is the distance Dca, the distance difference DDab is Dab×sinθ, the distance difference DDbc is Dbc×sin(60+θ), and the distance difference DDca is Dca×sin (60-θ). Here, since θ<30°, sin θ<sin(60-θ)<sin(60+θ), and Dab=Dbc=Dca, so Dab×sinθ<Dca×sin(60-θ)<Dbc ×sin(60+θ). That is, DDab<DDca<DDbc.
For other areas, the order of the distance differences Ddiff is determined in the same way. Further, the order of increasing distance difference Ddiff is the same as the order of increasing arrival time difference Tdiff, so the order of increasing arrival time difference Tdiff in each region is as shown in FIG. In FIG. 6, three arrival time differences Tdiff are listed in ascending order in each region. Note that, as described above, depending on the length of time required for the sound to arrive, it is possible to specify which of the two microphones is located farther from the sound source. In FIG. 6, microphones located far from the sound source are shown in parentheses. For example, in the region R1a, the maximum arrival time difference Tdiff is the arrival time difference Tca, indicating that the microphone farther from the sound source position is the microphone MICc.
Above, we have explained the order of distance difference Ddiff in each region using the case where triangle ABC is an equilateral triangle as an example. However, when triangle ABC is not an equilateral triangle and the orthocenter is inside the region surrounded by triangle ABC, Even in the case of a triangle in which all angles are 90° or less, the order of the distance differences Ddiff in each region is similarly determined. Triangles in which all angles are 90° or less include, for example, right triangles, acute triangles, and the like. Note that in the case of triangles that do not correspond to triangles in which all angles are 90 degrees or less, or obtuse triangles, the order of the distance differences Ddiff from large to large is not as shown in FIG. 6.

By the way, in (Formula 1), as the angle θ approaches 90 degrees, the amount of change in the distance difference Ddiff becomes smaller with respect to the amount of change in the angle θ. This is clear from the following (Formula 6) obtained by differentiating (Formula 1).
Ddiff'=(Dab×sinθ)'=cosθ...(Formula 6)
Therefore, as shown in FIG. 7, for example, the difference between the distance difference Ddiff when the sound source angle is the angle θ and the distance difference Ddiff when the sound source angle is the angle (θ+Δθ) becomes smaller as the angle θ approaches 90°. . Therefore, as the angle θ approaches 90°, it becomes easier to calculate the angle θ which is greatly affected by the measurement error of the distance difference Ddiff, and the accuracy of the calculated angle θ becomes worse. Note that FIG. 7 is a diagram similar to FIG. 1, and the perpendicular line of the line connecting the microphones MICa and MICb (indicated as 0° in FIG. 7) passes through the midpoint of the line segment connecting the microphones MICa and MICb. ) and the direction of the sound source, which is the angle θ. Here, when the angle θ approaches 90°, it means that the distance difference Ddiff and the arrival time difference Tdiff approach the maximum in the target microphone group MICa⇔MICb.
In view of the above, it is possible to calculate a total of three angles θ based on each of the three arrival time differences Tdiff calculated from each set, with two of the three microphones as one set. The inventors have found that by excluding the maximum arrival time difference Tdiff from the calculation of the angle θ, it is possible to improve the accuracy of the angle θ indicating the direction of the sound source.

Now, if the arrival time differences Tdiff of the remaining two sets excluding the one set with the maximum arrival time difference Tdiff are to be used to calculate the angle θ, specify which of the 12 regions (see Figure 6) the sound source direction is in. It is not necessary to do this, and it is sufficient to specify which of the six regions the sound source direction is located in, which is specified by the maximum arrival time difference Tdiff.
The six regions identified by the maximum arrival time difference Tdiff are shown in FIG. 8: region R1 including regions R1a and R1b, region R2 including regions R2a and R2b, region R3 including regions R3a and R3b, and region R4a and R4b. There are six regions: region R4 including region R5, region R5 including region R5a and R5b, and region R6 including region R6a and R6b. As shown in FIG. 8, the maximum arrival time differences Tdiff in each of regions R1 to R6 are arrival time differences Tca, Tbc, Tab, Tca, Tbc, and Tab, respectively.
For example, if the microphone pair with the maximum arrival time difference Tdiff is the microphone group MICa⇔MICb, and the distant microphone is the microphone MICa, the sound source direction is specified to be in the region R3. Region R3 in FIG. 8 is a region spanning regions 3 and 4 in FIG. 3. Therefore, if the sound source direction is specified as region R3, it is not possible to specify whether the sound source direction is front or back for the microphone group MICa⇔MICb, but for the microphone group MICb⇔MICc, the sound source direction cannot be specified. is on the "front", and it can be specified that the sound source direction is on the "back" in the microphone group MICc⇔MICa. Incidentally, even if the set with the second or third largest arrival time difference Tdiff is known, it is not possible to identify the front and back sides of the remaining two sets other than this set.

The explanation above has been made on the assumption that the sound source is on a plane that includes the microphones MICa, MICb, and MICc. However, the configuration (1) is not limited to the case where the sound source is on a plane including the microphones MICa, MICb, and MICc. FIG. 9 shows a case where the sound source is not on the plane including the microphones MICa, MICb, and MICc. Here, the plane containing the microphones MICa, MICb, and MICc is called the XY plane, the distance from the sound source to the XY plane is called the distance Dz, and the position of the sound source is projected onto the XY plane along the direction perpendicular to the XY plane. The position is referred to as a projection position, the distance from the projection position to the reference position is referred to as distance Dxy, and the distance from the sound source to the reference position is referred to as distance Dd. Further, in a triangle whose vertices are the sound source, the reference position, and the projection position, the angle formed by the line segment connecting the sound source and the reference position and the line segment connecting the reference position and the projection position is referred to as an angle θz. If the distance Dxy is sufficiently long with respect to the distance Dz, the angle θz will be small, so the distance Dxy can be approximated to the distance Dd. Similarly, the distances from the projection position to each of the three microphones MICa, MICb, and MICc can be approximated to the distances from the sound source to each of the three microphones MICa, MICb, and MICc, respectively. Therefore, for example, the difference between the distance from the projection position to the microphone MICa and the distance from the projection position to the microphone MICb can be approximated to the distance difference DDab. Therefore, based on the difference in arrival time of sound from the sound source to each of the three microphones MICa, MICb, and MICc, the difference in distance from the projection position to each of the three microphones MICa, MICb, and MICc can be determined. In other words, even when the sound source is not on the XY plane, the direction of the sound source from the projection position, which is the projection position of the sound source onto the XY plane along the direction perpendicular to the XY plane, toward the reference position is from the sound source to the three microphones MICa, It can be specified based on the difference in arrival time of sound to each of MICb and MICc.
In view of the above, the inventors have found that the following configuration (2) is good in identifying the direction of a sound source using three microphones.

(2) In the sound source direction identification device of the present application, the reference position is the orthocenter of the triangle, and the identification unit is configured to identify the three microphones calculated from each set, with two microphones among the three microphones as one set. The sound source direction is divided by three perpendicular lines passing through the reference position drawn to each of the lines passing through the two microphones, based on one set that is the largest difference in arrival time among the differences in arrival times. Determine which of the six regions surrounding the reference position belongs to, and decide among the six regions based on the difference in arrival time of the remaining two sets excluding one set which is the maximum difference in arrival time. The present invention is characterized in that a sound source angle is calculated, which is an angle between the sound source direction and one of the three perpendicular lines, which corresponds to the area where the sound source is located. In this way, the direction of the sound source can be specified with high accuracy.

In other words, it is determined which of the six regions the sound source direction belongs to based on one set with the maximum arrival time difference Tdiff, and it is calculated from the arrival time differences Tdiff of the remaining two sets excluding the one set with the maximum arrival time difference Tdiff. Of the angles θ and θ′ that are candidates for the sound source angle, the angle that corresponds to the determined region among the six regions is used to calculate the sound source angle. Based on the one set with the maximum arrival time difference Tdiff, it is possible to specify which of the remaining two sets are the front and back of the sound source direction and the angles θ and θ'. Furthermore, by excluding one set with the maximum arrival time difference Tdiff, the accuracy of the sound source angle can be improved.

(3) Preferably, the triangle is an equilateral triangle. In this way, the calculation for deriving the angle θ can be simplified. The load placed on the specific unit during calculation can be reduced.

(4) The identification unit calculates the difference in arrival times based on the phase difference calculated from each set of two electrical signals out of the three electrical signals output by each of the three microphones. It is good to have a configuration that does this.

The electrical signal includes errors due to variations in microphone frequency characteristics, the environment, and the like. For this reason, for example, if the difference in arrival time is calculated based on the time when the level of the electrical signal exceeds the threshold, the accuracy of the direction of the sound source may deteriorate. By calculating the difference in arrival time based on the phase difference, the direction of the sound source can be specified with high accuracy. Note that when the sound includes a plurality of frequency components, as in case (5) where the sound is a human voice, it is preferable to perform frequency analysis on the electrical signal and calculate a phase difference for each frequency component.

(5) The sound source may be a human voice, and the distance between each of the three microphones may be 57 mm or more and 170 mm or less. In this way, the direction of the person who is the source of the sound can be specified with high accuracy.

In the case of configuration (4) in which the arrival time difference Tdiff is calculated from the phase difference, the inter-microphone distance may be determined based on the frequency of the frequency component used to calculate the phase difference. For example, if the distance between microphones is set to one wavelength of frequency, there is a possibility that a wave that enters one microphone will enter the other microphone in a range from a wave that is one cycle ahead to a wave that is one cycle behind. appears, making it impossible to identify the phase difference.
For example, when a wave whose phase difference is -1/2π with respect to the wave entering one microphone enters the other microphone, the wave that actually enters the other microphone is delayed by 1/4 period. , 3/4 period earlier waves may also enter, so it is not possible to specify whether the phase difference is -1/2π or +3/2π based on the electrical signal.
Therefore, if the distance between the microphones is set to half a wavelength of the frequency used to calculate the phase difference, the wave entering one microphone will be 1/2 period from the wave that is 1/2 period ahead of the wave entering the other microphone. Limited to late waves. As mentioned above, when a wave whose phase difference is -1/2π with respect to the wave entering one microphone enters the other microphone, the wave entering the other microphone is a wave delayed by 1/4 period, and the phase difference is -1/2π can now be specified.
To give specific numerical values, for example, if the speed of sound is 340 m/s and the distance between microphones is 57 mm, it is possible to specify the phase difference for waves with a frequency of 3 kHz or less. Furthermore, if the distance between the microphones is 170 mm, it is possible to specify the phase difference for waves with a frequency of 1 kHz or less.

In this way, the shorter the distance between the microphones, the more it becomes possible to specify the phase difference even at a high frequency, so the range of frequencies in which the phase difference can be specified becomes wider. However, if the distance between the microphones is short, the phase difference becomes small, so the phase difference becomes small especially at low frequencies, which may lead to errors in the phase difference.

By the way, it is known that the upper limit of the fundamental frequency of human voice is about 200 Hz, the upper limit of the first formant frequency is about 1 kHz, and the upper limit of the second formant frequency is about 3 kHz. Here, the formant frequency is a frequency that characterizes a vowel and has a peak sound pressure level. For example, even in the case of a short human voice such as "a", the electrical signal output by the microphone includes two frequency components of the fundamental frequency and the first formant frequency below 1 kHz. Furthermore, below 3 kHz, three frequency components are included: the fundamental frequency, the first formant frequency, and the second formant frequency.
The inventors have found that in order to accurately identify the direction of a sound source, the frequency range used to identify the position of the sound source is preferably 1 kHz or less, and particularly preferably 3 kHz or less. As mentioned above, if the frequency range is 1 kHz or less, at least the two frequency components of the fundamental frequency and the first formant frequency are included, and if the range is further expanded to a frequency range of 3 kHz or less, the fundamental frequency and the first formant frequency are included. This is because three frequency components, the formant frequency and the second formant frequency, are included. Furthermore, the sound source can be identified with high accuracy without using a frequency higher than 3 kHz.
As mentioned above, to calculate the phase difference for frequencies below 1 kHz, the distance between the microphones should be 170 mm, and to calculate the phase difference for frequencies below 3 kHz, the distance between the microphones should be 57 mm. . When the distance between the microphones is in the range of 57 mm or more and 170 mm or less, the upper limit of the frequency at which the phase difference can be determined is 1 kHz to 3 kHz. Therefore, if the distance between the microphones is in the range of 57 mm or more and 170 mm or less, at least the fundamental frequency and the first formant frequency can be used to calculate the phase difference. Furthermore, by setting the upper limit of the frequency used to calculate the phase difference to about the second formant frequency, it is possible to improve the accuracy of the phase difference at low frequencies. In this way, the direction of the sound source of human voices can be identified with high accuracy.

(6) The identification unit performs sampling processing for converting the electrical signals output from each of the three microphones into digital values during a predetermined period, and identifies the direction based on the digital values converted by the sampling processing. It is preferable to have a configuration in which specific processing is repeatedly executed.

In this way, even if it is not known when the sound will be emitted, the direction of the sound source can be specified according to the sound generation. For example, the present invention may be applied to devices that operate in response to sound, such as communication robots that operate in response to human voices, and surveillance cameras that record the location of sound. When applied to a device that operates in response to sound, it is preferable to perform control to move toward the direction of the sound source. It is also good to have a function to identify the position where a person speaks. It is also good to provide a voice recognition function. Further, it is preferable to have a function of directing a predetermined part in the direction of speech of the specified person. It is especially good to use communication robots. Further, it is preferable that the time from the start time of a sampling process to the start time of the next sampling process is 200 ms or less. In this way, for example, in the case of a communication robot, even in response to a short call such as "hey", the location of the sound source can be specified according to the sound, and the robot can operate reliably. Further, according to the configuration of the present application, the distance between the microphones can be set to a size suitable for a communication robot, especially as in (5).

The inventions shown in (1) to (6) above can be combined arbitrarily. For example, at least a part of the structure of at least one invention described in (2) and subsequent ones may be added to all or part of the structure of the invention shown in (1). In particular, in the invention shown in (1), (2)
The invention may include at least a part of at least one of the following inventions. Further, arbitrary configurations may be extracted from the inventions shown in (1) to (6) and the extracted configurations may be combined. The applicant of this application intends to acquire rights to inventions containing these structures.

Moreover, the inventions shown in (A) to (I) described below may be combined arbitrarily. For example, a configuration may be adopted in which at least a part of the configuration of at least one of the following inventions (B) is added to all or part of the configuration of the invention shown in (A). In particular, it is preferable to create an invention in which at least a part of the structure of at least one invention after (B) is added to the invention shown in (A). Moreover, the inventions shown in (1) to (6) above and the inventions shown in (A) to (I) described below can be combined arbitrarily. Alternatively, arbitrary configurations may be extracted from the inventions shown in (A) to (I) and the extracted configurations may be combined. Any configuration may be extracted from the inventions shown in (1) to (6), any configurations may be extracted from the inventions shown in (A) to (I), and the extracted configurations may be combined. The applicant of this application intends to acquire rights to inventions containing these structures.

(A) A device comprising a plurality of microphones, wherein a predetermined reference direction and a sound source are determined based on the order of arrival times of sound to two of the plurality of microphones and the difference in arrival times of sound to the two microphones. Using the angle calculation function, which is a function to calculate the angle formed with the direction of The apparatus may include a sound source direction specifying function, which is a function of specifying in which region the sound source is present out of two regions divided by a plane including a plane.
In this way, it is possible to determine on which side of the two areas the sound source is located, which is divided by the plane perpendicular to the plane including the positions of the three microphones and including the positions of the two microphones. can be determined, and the angle between the predetermined reference direction and the direction of the sound source can be determined.
The predetermined reference direction may be, for example, a predetermined direction in a plane that includes the positions of the three microphones, and the direction of the sound source is a direction in the plane that includes the positions of the three microphones (for example, a component of a three-dimensional vector in the plane) It is good to do this.
The other microphone used in the sound source direction specifying function may be one microphone, or may be a plurality of microphones.

(B) "Using another microphone" refers to "using the context of the arrival time of sound between two other microphones that are placed in positions that are not parallel to the two microphones." It is good to do this.
In this way, it is possible to determine on which side of the two areas the sound source is located, which is divided by the plane perpendicular to the plane including the positions of the three microphones and including the positions of the two microphones. can be determined more reliably and accurately. For example, microphones A to E are placed at the vertices of a regular pentagon ABCDE, and microphones A and B are used as the two microphones used in the angle calculation function, and microphones C and D are used as other microphones used in the sound source direction identification function. It is good to do this.

(C) "Using another microphone" means "using one microphone other than the two microphones and one of the two microphones in relation to the arrival time of the sound." It is better to say "using".
In this way, by simply adding at least one microphone, you can select which of the two areas divided by the plane perpendicular to the plane containing the positions of the three microphones and including the positions of the two microphones. It is possible to more reliably and accurately determine whether a sound source exists on the area side. For example, place microphones X to Z at the vertices of an equilateral triangle It is preferable to define one of the microphones as microphone X.

(D) Out of the plurality of microphones, a pair of microphones to be used as the two microphones used in the angle calculation function and a microphone to be used as the other microphone to be used in the sound source direction identification function are selected according to a predetermined rule. It would be good to have a function to make a decision based on.
In this way, even if the position of the sound source changes, it can be divided more reliably and accurately by a plane that is perpendicular to a plane that includes the positions of the three microphones and that includes the positions of the two microphones. It is possible to determine in which region of the two regions the sound source is present, and to determine the angle between the predetermined reference direction and the direction of the sound source.
In particular, the predetermined rule may be a rule based on the sounds detected by each of the plurality of microphones, or may be a rule based on a comparison result of the sounds detected by each of the plurality of microphones. For example, the rule may be based on a difference in arrival time, such as a phase shift of sounds detected by each of the plurality of microphones.

(E) The predetermined rule specifies that any one of the plurality of microphones other than the two microphones of the reference pair, which is the pair of microphones with the largest sound arrival time difference, is used in the angle calculation function. It is preferable to set the rule to use at least one of the two microphones.
In this way, regardless of the position of the sound source, it is possible to prevent the accuracy of calculating the angle between the reference direction and the direction of the sound source by the angle calculation function from becoming significantly lower.
(F) The predetermined rule specifies that, among the plurality of microphones, at least one of the two microphones of the reference pair, which is a pair of microphones having the largest sound arrival time difference, is used in the sound source direction identification function. It would be a good idea to make it a rule to use a microphone.
(G) The sound source direction identification function is performed using two microphones of a reference pair, which is a pair of microphones having the largest sound arrival time difference among pairs of microphones forming sides connecting vertices of a polygon having the plurality of microphones as vertices. The direction in which the sides formed by the two microphones of the reference pair intersect in the direction from before to after the arrival time of the sound of the reference pair with respect to each side of the polygon formed by Which of the two areas is divided by a plane including the positions of the two microphones forming at least one of the sides, based on whether the area is outside the area or from the outside to the inside. It is preferable to specify whether or not the sound source exists in the region.
In this way, regardless of the position of the sound source, it is possible to more reliably specify in which of the two regions the sound source is present. For example, if each microphone is placed at the apex position of triangle ABC, and microphones B and C are a pair of microphones with the largest difference in sound arrival time, side BC goes from A to B, and side AB goes from triangle ABC. It has a geometric property that the direction is from the outside to the inside.

(H) As the plurality of microphones, a first microphone, a second microphone, and a third microphone are provided at the apex positions of the triangle, and the sound source direction identification function is performed using a pair consisting of the first microphone and the second microphone. and the pair consisting of the second microphone and the third microphone, and the pair consisting of the third microphone and the first microphone, which form the three sides of the triangle, the difference in arrival time of the sound is the greatest. A pair of two microphones having a larger value is set as a reference pair, and the triangle is defined in the two areas divided by the plane including the microphone to which the sound reached first among the reference pair and a microphone position different from the reference pair. It is assumed that the sound has arrived from an area outside of the reference pair, or of the two areas divided by the plane that includes a microphone that the sound reached later among the reference pair and a microphone position different from the reference pair. It is preferable to perform at least one of the following, assuming that the sound has arrived from an area inside the triangle.
In this way, it is possible to more reliably identify in which region the sound source is located using the three microphones.

(I) The triangle is preferably an equilateral triangle.
In this way, the accuracy of the three pairs will be equal, and 360° can be covered with less bias due to direction. Therefore, an extremely excellent effect is exhibited in a device that detects from which direction around the device the sound has arrived.

According to the present application, it is possible to provide a sound source direction identification device and the like that can accurately identify the direction of a sound source using, for example, three microphones, using a method different from the conventional technology. The effects of the invention of the present application are not limited to these, but the effects obtained from the parts of the structure disclosed in the specification, drawings, etc. are also disclosed, and the rights to the structure that produces the effects have also been acquired through divisional applications, amendments, etc. have the intention to do so. For example, in this specification, a portion where it is written as "can be done" is a statement that clearly indicates the effect to be achieved, and there are parts that show an effect even if there is no mention of "can be done". Further, even without such a description, there are effects that can be understood from the configuration.

FIG. 2 is a diagram illustrating the relationship between the difference in distance from a sound source to each of two microphones and the direction of the sound source. FIG. 3 is a diagram illustrating that there are two sound source positions in which the difference in arrival time of sound to each of two microphones is the same. FIG. 7 is a diagram showing the relationship between six areas surrounding the orthocenter of triangle ABC and the front and back sides of the sound source direction in each set of three microphones. FIG. 7 is a diagram showing arrival time differences at the boundaries of 12 regions surrounding the orthocenter O of the triangle ABC. 5 is a diagram for deriving the difference in distance from the sound source to each of three microphones when the sound source position is in the region R2a shown in FIG. 4. FIG. FIG. 7 is a diagram showing the order of arrival time differences in each of 12 regions surrounding the orthocenter O of triangle ABC. FIG. 3 is a diagram illustrating that the closer the sound source is to a straight line passing through two microphones, the larger the measurement error of distance difference becomes. FIG. 6 is a diagram showing six areas surrounding the orthocenter O of the triangle ABC, which are identified by one set having the maximum difference in arrival time of sound. FIG. 6 is a diagram illustrating a case where the sound source is not on a plane that includes three microphones. FIG. 1 is a perspective view of a robot according to an embodiment. FIG. 3 is a perspective view of the sound source direction identification device shown together with the fixed lower housing. FIG. 2 is an electrical configuration diagram of a sound source direction identification device. FIG. 2 is a diagram illustrating the polarity of sound source angles in one set of microphones. In each of the six cases specified by the pair with the largest distance difference and the farthest microphone of the two microphones forming the pair, which of the front and rear sound source angles of each pair is to be adopted for calculating the sound source angle. This is the table shown. It is a figure explaining the relationship between the sound source angle in each set and the sound source angle in the whole.

The robot 1 shown in FIG. 10 is a communication robot that operates in response to human voices. The robot 1 includes a fixed part 2 and a movable part 3 movable with respect to the fixed part 2. The directions shown in FIGS. 10 and 11 will be used in the following description. The fixed part 2 includes a lower fixed part casing 21, an upper fixed part casing 22, a sound source direction identifying device 10, and the like. The fixed lower housing 21 has a bowl shape with a bottom surface 23, and houses the sound source direction identification device 10 and the like therein. Note that the plane parallel to the bottom surface 23 is the XY plane. The fixed part upper housing 22 has a cylindrical shape, is located above the fixed lower housing 21, and covers the lower part of the movable part 3. A slight gap is provided between the fixed lower housing 21 and the fixed part upper housing 22, and the microphones MICa to MICc (see FIG. 11) included in the sound source direction identification device 10 are arranged in the gap. . Note that the inside of the fixed part upper housing 22 is not packed with members and does not have a structure for sound insulation. Therefore, the sound actually escapes inside the fixed part upper housing 22, and the microphones MICa to MICc can pick up the sound from behind the slave boards 12a to 12c (FIG. 11), respectively. The movable part 3 includes a movable part housing 31, a display device 32, and the like. The display device 32 is realized by, for example, a touch panel, a liquid crystal display, or the like. The movable part housing 31 has a spherical shape with a portion cut out in a planar shape. The display device 32 is attached to a planar portion of the movable part housing 31. The movable part 3 is rotatable by 360° around a Z-axis perpendicular to the bottom surface 23 of the fixed lower housing 21 using a motor (not shown) as a drive source. When a sound is emitted, the robot 1 rotates the movable part 3 so that the display device 32 faces the source of the sound, such as a person. The sound source direction specifying device 10 is a device for specifying the direction of a sound source for rotating the movable part 3.

As shown in FIG. 11, the sound source direction identification device 10 includes a disk-shaped substrate 11, microphones MICa to MICc, and the like. The board 11 has daughter boards 12a to 12c to which microphones MICa to MICc are attached. Each of the daughter boards 12a to 12c has microphones MICa to MICc attached to one surface thereof, and the other surface thereof is attached to the substrate 11 so as to be perpendicular to the substrate 11. The microphones MICa to MICc are omnidirectional condenser microphones, and are fixed to the substrate 11. The substrate 11 is approximately parallel to the bottom surface 23 of the fixed lower housing 21, and the positions of the microphones MICa to MICc in the Z direction are approximately the same with respect to the bottom surface 23. The microphones MICa to MICc are arranged so as to be located at the vertices of an equilateral triangle ABC (see FIG. 3) drawn on the XY plane, respectively. As a result, for example, the distance between each of the microphones MICa to MICc is common to the three groups, so the derivation method such as formula (5) can be made common to the three groups, and the sound source angle θ The calculation for deriving can be simplified. The length of one side of the equilateral triangle ABC, that is, the distance between each of the microphones MICa to MICc is, for example, about 100 mm. As a result, at least the fundamental frequency and the first formant frequency are included in the calculation of the phase difference in (processing 5) described later, and the accuracy of the phase difference at low frequencies can be improved. (described later) can accurately specify the sound source angle θ for human voices.

Further, the sound source direction specifying device 10 includes amplifiers AMPa to AMPc, sample and hold circuits SHa to SHc, a microcomputer 41, etc. below the board 11, as shown in FIG. The microphone MICa, the amplifier AMPa, and the sample-and-hold circuit SHa are connected in series in this order. Similarly, microphone MICb, amplifier AMPb, and sample-and-hold circuit SHb are connected in series, and microphone MICc, amplifier AMPc, and sample-and-hold circuit SHc are connected in series. That is, the sound source direction identifying device 10 has three channels from each of the microphones MICa to MICc to each of the sample and hold circuits SHa to SHc. Each of the three channels is referred to as channels Ach to Cch. The amplifiers AMPa to AMPc amplify electrical signals output from the electrically connected microphones MICa to MICc and output them to the electrically connected sample and hold circuits SHa to SHc. The sample-and-hold circuits SHa to SHc hold input electrical signals in synchronization with the sampling clock signal output from the microcomputer 41, and output the held electrical signals to the microcomputer 41. The frequency of the sampling clock signal, that is, the sampling frequency is approximately 20 to 40 kHz.

When the robot 1 is powered on and activated, the microcomputer 41 starts (processing 1) to be described later. Furthermore, (processing 1) to (processing 8) are repeatedly executed for specifying the direction of the sound source during a predetermined period. Thereby, even if it is not known when the sound will be emitted, the direction of the sound source can be specified according to the sound generation. Note that the cycle for executing (process 1) is 200 ms or less. As a result, the direction of the sound source can be specified according to the generation of the sound, even if it is a short human voice such as "hey". Further, when the power of the robot 1 is turned off, the microcomputer 41 terminates any one of (processing 1) to (processing 8) that is being executed.

(Processing 1) The microcomputer 41 performs AD conversion on the electrical signals output from each of the sample-and-hold circuits SHa to SHc, and stores them in an array for each channel. Specifically, the microcomputer 41 sequentially arranges data obtained by AD converting the electrical signals outputted from each of the sample-and-hold circuits SHa to SHc for each channel and stores the data in a built-in memory.

(Processing 2) After acquiring a certain amount of data, the microcomputer 41 performs fast Fourier transform (FFT) for three channels. Specifically, the microcomputer 41 converts the data stored in the memory into data stored in the memory after a predetermined period of time has passed during which the number of data obtained by AD converting the electrical signals output from each of the sample-and-hold circuits SHa to SHc reaches a predetermined number. is fast Fourier transformed for each channel. The predetermined time is approximately 50 ms to 100 ms. For example, if it is longer than 200 ms, there will be a time lag between when a voice is called and when the motion is performed, which will increase the unnaturalness. On the other hand, when the time is shorter than 50 ms, the number of data decreases, resulting in a decrease in direction accuracy. By setting the predetermined time within the above range, communication can be facilitated and accuracy of the sound source angle can be ensured. Additionally, since voices contain a mixture of various wavelengths, frequency analysis is performed using fast Fourier transform. Note that for fast Fourier transform, the number of data should preferably be a power of 2, for example, about 2^8, 2^9, or 2^10. The microcomputer 41 stores the complex number data of each frequency component obtained by fast Fourier transform in a memory in association with a frequency index assigned to each frequency component. In addition, in order to be able to specify one phase when calculating the absolute phase in (Processing 4) described later, in the next processing, we will use frequencies whose half wavelength is longer than the distance between each of the microphones MICa to MICc. The processing target is frequencies lower than 1.7kHz. Note that the calculation here is based on the assumption that the speed of sound is 340 m/s.

(Process 3) Next, the microcomputer 41 calculates power from the complex number data for each frequency index obtained by fast Fourier transform, targeting frequency components lower than 1.7 kHz. Power is the sum of the square of the real value and the square of the imaginary value. Next, the microcomputer 41 stores in memory the frequency index that exceeds a preset threshold. Hereinafter, a frequency index exceeding a preset threshold will be referred to as a voiced frequency index. Here, a frequency index that does not exceed a preset threshold value indicates that there is no audio component at this frequency. Therefore, in the subsequent processing, the microcomputer 41 processes only the voiced frequency index.

(Process 4) Next, the microcomputer 41 calculates the absolute phase from the complex number data for each voice frequency index. As shown below, the formula for calculating the absolute phase targets four quadrants.
Absolute phase = ArcTan [imaginary value, real value]
Note that the absolute phase here is based on the start time at which the sample-and-hold circuits SHa to SHc first hold the real-time data sampled by the sample-and-hold circuits SHa to SHc. Furthermore, since the range of complex number data is four quadrants on the complex number plane, the range of the calculated absolute phase is -π to +π.

(Process 5) Next, the microcomputer 41 determines phase differences for a total of three sets, with two channels as one set, from the absolute phases of three channels for each voiced frequency index. Specifically, the phase difference between channel Ach and channel Bch, the phase difference between channel Bch and channel Cch, and the phase difference between channel Cch and channel Ach are determined. Here, when calculating the phase difference between channel Ach and channel Bch, the absolute phase of channel Ach is subtracted from the absolute phase of channel Bch, and when calculating the phase difference between channel Bch and channel Cch, the absolute phase of channel Cch is calculated. The absolute phase of channel Bch is calculated by subtracting the absolute phase of channel Bch from the absolute phase of channel Bch, and the phase difference between channel Cch and channel Ach is calculated by subtracting the absolute phase of channel Cch from the absolute phase of channel Ach.

Further, the plus and minus polarities of the sound source angle in one set of microphones MIC are defined as shown in FIG. Note that FIG. 11 is a diagram for explaining the set of microphones MICa and MICb among the three sets. The definitions of the sound source angle, front and back, etc. are the same as above. That is, a line perpendicular to the line segment connecting the microphones MICa and MICb that passes through the midpoint of the line segment connecting the microphones MICa and MICb is referred to as a 0° line. Further, the sound source direction is the direction from the projection position of the sound source projected onto the plane including the triangle ABC along a direction perpendicular to the plane including the triangle ABC toward the reference position which is the orthocenter of the triangle ABC. Sound is regarded as a plane wave, and the angle between the 0° line and the direction from the projection position of the sound source to the midpoint of the line segment connecting microphones MICa and MICb is the sound source angle θ. With respect to the line passing through microphones MICa and MICb, the side without microphone MICc is the front side, and the side with microphone MICc is the back side.
When calculating the phase difference between the microphones MICa and MICb with respect to the 0° line, the side of channel Ach, which is the channel to be subtracted, where the microphone MICa is not located is defined as plus, and the side where the microphone MICa is present is defined as minus. In other words, if the phase difference is positive, microphone MICa is farther from the sound source than microphone MICb, whereas if the phase difference is negative, microphone MICb is farther from the sound source than microphone MICa. .
Further, other groups are defined in the same way. That is, when calculating the phase difference between microphones MICb and MICc with respect to the 0° line, the side without microphone MICb of channel Bch, which is the channel to be subtracted, is defined as plus, and the side with microphone MICb is defined as minus. . When calculating the phase difference between microphones MICc and MICa with respect to the 0° line, the side of channel Cch, which is the channel to be subtracted, where there is no microphone MICc is defined as plus, and the side with microphone MICc is defined as minus. In the following description, the sound source angle θ may be referred to as a direction value.

(Process 6) Next, the microcomputer 41 calculates the arrival time difference Tdiff for each voice frequency index from the phase difference and the frequency of the voice frequency index. In this way, by determining the arrival time difference Tdiff from the phase difference, the arrival time difference Tdiff can be determined with high accuracy. For example, the time difference between the times when the two real-time waveforms before the fast Fourier transform exceed a preset volume threshold can be used as the delay time difference. However, in the case of this method using real-time waveforms, it is easily affected by the frequency characteristics of the microphone and the difference in frequency characteristics between the two microphones. For example, if one microphone has poor sensitivity to frequencies in a certain band and the level of the frequencies in this band drops, the real-time waveform will be different from that of the other microphone. Therefore, the delay time difference becomes different from the actual one, and the accuracy of the arrival time difference Tdiff deteriorates. Furthermore, in the case of this method using real-time waveforms, the setting of the threshold greatly affects the delay time difference. As described above, since the two real-time waveforms are different from each other, the delay time difference varies depending on the volume threshold. Furthermore, in the case of this method using real-time waveforms, it is easily affected by the surrounding environment, for example, sound reflected from walls. In this regard, according to the method using the phase difference in this embodiment, compared to the method using real-time waveforms, the frequency characteristics of the microphone and the influence of reflected sound are less likely to be reflected in the arrival time difference Tdiff. The time difference Tdiff can be determined. As described later, the arrival time difference Tdiff is determined for each frequency component of the voice, and the sound source angle θ is determined using the determined arrival time difference Tdiff, so there is no correlation between the frequency components, and the influence of the frequency characteristics of the microphone and environment is eliminated. hard to receive.

After calculating the arrival time difference Tdiff of each voice frequency index, the microcomputer 41 calculates a weighted average of the arrival time differences Tdiff of all voice frequency indexes. The weight (level) used here is √(real value ^2 + imaginary value ^2) of the frequency index. The microcomputer 41 does not use the value at each voice frequency index in subsequent processing, but uses one value determined by weighted average.
After calculating one arrival time difference Tdiff for each group, the microcomputer 41 calculates arrival time difference Tdiff, sound speed, (
The distance difference Ddiff is calculated from Equation 4). Here, it is assumed that the speed of sound is calculated as 340 m/s. Here, it is assumed that the plus/minus polarity of the phase difference is also followed by the arrival time difference Tdiff and the distance difference Ddiff. Therefore, for example, between the microphones MICa and MICb, if the distance difference Ddiff is positive, it means that the microphone MICa is far away, and if the distance difference Ddiff is negative, it means that the microphone MICb is far away.

(Process 7-1) Next, the microcomputer 41 calculates the distance difference Ddiff in FIG. A matching row is selected from among the six rows of the table 51 shown.

Table 51 shown in FIG. 14 is a table summarizing FIG. 8. Table 51 shows which side, front or back, should be adopted for calculating the sound source angle in each set of channels Ach to Cch. Each row of table 51 corresponds to one of the regions R1 to R6 shown in FIG. 8. The columns of the table 51 correspond to the front and back sides of each of the three sets of channels Ach to Cch. In Table 51, the sides that should be adopted in the calculation of the sound source angle are marked with "O", and the sides that should not be adopted are marked with "-".
For example, the first row of Table 51 shows the case where the pair with the largest distance difference Ddiff is the pair of channels Ach and Cch, and the microphone MICa of channel Ach is far from the sound source. This case is a case where the sound source direction belongs to region R4 in FIG. "O" is marked on "Bch-Cch front" and "Cch-Ach back". In this case, as described above, the accuracy of the sound source angle calculated from the distance difference DDca between the pair of channels Ach and Cch is poor, so the microcomputer 41 detects the sound source on either the front or back side of the pair of channels Ach and Cch. It is not used to calculate angles. Therefore, in Table 51, "-" is written on both the "front side of Ach-Cch" and the "back side of Ach-Cch."

The microcomputer 41 refers to Table 51 based on the set of channels with the maximum distance difference Ddiff and the polarity, and marks "〇" for the set of channels other than the set of channels with the maximum distance difference Ddiff. Select which side is on the front or back side. For example, if the set of channels with the maximum distance difference Ddiff is channels Ach and Cch, and the polarity of the distance difference Ddiff is positive, then the first row of Table 51 is applicable, so the microcomputer 41 selects channels Bch, Cch, The front side of Cch and the back side of channels Ach and Bch are selected and stored in the memory. Furthermore, the microcomputer 41 calculates the sound source angle θ for each of the two sets of channels for each sound frequency index using (Equation 5). Here, it is assumed that the plus/minus polarity of the distance difference Ddiff is also followed by the sound source angle θ. As mentioned above, the sound source angle θ calculated using (Equation 5) is based on the projection position of the sound source projected onto the XY plane along the direction perpendicular to the XY plane including the microphones MICa to MICc. This shows the direction of the sound source from the projected position to the reference position, assuming that the distance to the position can be approximated to the distance from the sound source to the reference position.

(Process 7-2) Next, the microcomputer 41 converts the sound source angle θ calculated for each group into an overall sound source angle with the reference direction unified for the three groups. Here, as shown in Fig. 15, the 0° line on the front side of the microphones MICa, MICc is used as the overall reference direction, and the center point of the line connecting the microphones MICa, MICc is used as the fulcrum, and the angle is 0° to 360° clockwise. The range shall indicate the entire sound source angle. FIG. 15 shows a case where the pair with the largest distance difference Ddiff is the pair of channels Bch and Cch, and the microphone MICc of channel Cch is farther from the sound source than the microphone MICb. Here, it is assumed that the sound source angle at the microphones MICa and MICc is an angle +θca, and the sound source angle at the microphones MICa and MICb is an angle +θab. In this case, since the angle +θca is located on the back side and the angle +θab is located on the front side, the overall sound source angles are 180°-θca and 120°+θab, respectively.

(Process 8) Next, the microcomputer 41 statistically calculates the final sound source direction based on the overall sound source angle calculated in (Process 7-2). Specifically, the microcomputer 41 calculates one sound source direction by averaging all the sound source angles calculated in (processing 7-2).

The microcomputer 41 controls the motor that rotates the movable part 3 so that the display device 32 faces the calculated sound source direction. Thereby, the display device 32 of the robot 1 faces the direction of the sound source.

Here, the advantages of this embodiment over other methods of specifying the direction of a sound source will be explained.
Another method is to use a plurality of directional microphone phonons and determine the direction of the sound source from the volume difference or volume ratio. In other methods, the accuracy of sound source position detection depends on the directivity performance of the microphone. In this regard, in this embodiment, an omnidirectional microphone is used and is not dependent on directional performance. Further, in other methods, for example, about 10 directional microphone phonons are required, but in this embodiment, the direction of the sound source can be specified with three microphones. Furthermore, other methods are susceptible to the influence of the surrounding environment. For example, if there are walls nearby, the microphone will pick up indirect sound as the sound reflects off the walls. Therefore, the level difference between the sounds picked up by the plurality of microphones becomes small. In this regard, in this embodiment, since the phase is looked at instead of the volume, the obtained sound source angle can be determined with high resolution and accuracy.

Here, the sound source direction identifying device 10 is an example of a sound source direction identifying device, the microcomputer 41 is an example of a identifying section, (processing 1) is an example of sampling processing, and (processing 2) to (processing 8) are This is an example of specific processing. In addition, the overall sound source angle calculated in (Process 7-2) is "the angle between the sound source direction, which is the determined area among the six areas, and one perpendicular line among the three perpendicular lines. This is an example of "sound source angle".

According to the embodiment described above, the following effects are achieved.
The sound source direction identification device 10 determines the position of the sound source from the equilateral triangle ABC based on the three microphones MICa to MICc placed at the vertices of the equilateral triangle ABC and the arrival time difference Tdiff of the sound from the sound source to each of the three microphones. A microcomputer 41 that specifies the direction of a sound source from a position projected onto a plane containing equilateral triangle ABC along a direction perpendicular to the containing plane toward a reference position located inside an area surrounded by equilateral triangle ABC on the plane containing equilateral triangle ABC. Equipped with. Thereby, the sound source direction identifying device 10 can identify the sound source direction using the three microphones MICa to MICc. Furthermore, since the three microphones MICa to MICc are arranged at the vertices of the equilateral triangle ABC, calculations for deriving the sound source angle θ can be simplified.

In addition, in (processing 7-1), the microcomputer 41 performs a process based on the maximum arrival time difference Tdiff among the three arrival time differences Tdiff calculated from each set, with two microphones among the microphones MICa to MICc as one set. A region in which the direction of the sound source is six regions surrounding a reference position divided by three perpendicular lines PLa to PLc (FIG. 8) passing through the reference position drawn to each of the lines passing through two of the microphones MICa to MICc. Determine which row of Table 21 corresponding to R1 to R6 (FIG. 8) fits, and based on the arrival time difference Tdiff of the remaining two sets excluding the one set with the maximum arrival time difference Tdiff, the area R1 to R6 is determined. The sound source angle θ, which corresponds to the determined region, is calculated by adding information on the front and back sides and polarity. In the embodiment, the range of the sound source angle θ is 0° or more and 90° or less, and 360° is indicated by adding plus/minus polarity and front/back information to the sound source angle θ. In (process 7-2), the microcomputer 41 converts the sound source angle θ calculated in (process 7-1) into an overall sound source angle with the three sets of reference directions unified. Thereby, the sound source direction identifying device 10 can accurately identify the sound source angle θ.

Furthermore, in (process 6), the microcomputer 41 calculates the arrival time difference Tdiff based on the phase difference. Thereby, the sound source direction identifying device 10 can accurately identify the sound source angle θ.

Further, the distance between the microphones MICa to MICc is approximately 100 mm. Thereby, the sound source direction identifying device 10 can accurately identify the direction of the person who is the sound source.

In addition, the microcomputer 41 converts the electrical signals output from each of the microphones MICa to MICc into digital values during a predetermined period (processing 1), and specifies the direction based on the digital values converted in (processing 1). (Processing 2) to (Processing 8) are repeatedly executed. Thereby, the sound source direction specifying device 10 can specify the position of the sound source according to the sound even in response to a short call, and can operate reliably.

Furthermore, it goes without saying that the present invention is not limited to the embodiments described above, and that various improvements and changes can be made without departing from the spirit of the present invention.
For example, although it has been explained above that the microphones MICa to MICc are arranged at the vertices of the equilateral triangle ABC, the present invention is not limited thereto. Instead of being an equilateral triangle, it may be a triangle in which all angles are 90° or less.

Furthermore, although the sound source direction identification device 10 has been described above as having sample and hold circuits SHa to SHc, the present invention is not limited thereto. For example, if the microcomputer 41 has a configuration that can simultaneously sample the output signals from the amplifiers AMPa to AMPc, the microcomputer 41 may have a configuration that does not include the sample and hold circuits SHa to SHc.

Moreover, in the above description, in (Processing 4) and (Processing 5), the phase difference is calculated by subtracting the absolute phase of one channel of the set from the other channel. The calculation is not limited to this, but one channel of the set may be determined as a reference, and the coordinates of the other channel may be rotated so that the absolute phase of the reference channel becomes 0°. The absolute phase of the other channel after coordinate rotation becomes the phase difference.

Further, although it has been described above that the final sound source direction is statistically calculated in (processing 7-2), the present invention is not limited to this. Here, instead of taking an arithmetic average of a plurality of sound source directions, it is preferable to use a weighted average that is weighted the closer the angle is to 0°.

Further, although it has been explained above that the microphones MICa to MICc are omnidirectional microphones, they are not limited to this, and may be unidirectional microphones. Even though condenser microphones are unidirectional, their directivity is not very good. Even with a unidirectional condenser microphone, if you speak from the back, that is, from the side that is not collecting the sound, you will not be unable to pick up the sound, and there is only a small difference between omnidirectional and unidirectional. This is because. However, in this embodiment, each of the microphones MICa to MICc preferably picks up sound equally no matter which direction in the 360° direction the sound source is located in the plane including the microphones MICa to MICc. MICa to MICc are preferably omnidirectional microphones.

Further, in the above description, for example, the definition of front and back sides of the sound source direction and the definition of the polarity of the sound source angle have been described, but the present invention is not limited thereto. These can be arbitrarily defined so that the overall sound source direction is aligned with the calculated phase difference.

Furthermore, although it has been described above that (Processing 1) to (Processing 8) are repeatedly executed while the robot 1 is powered on and the microcomputer 41 is activated, the present invention is not limited to this. For example, the configuration may be such that (processing 1) is started when the volume level exceeds a threshold value as a trigger. According to this configuration, it is possible to reliably capture sound and reliably operate in response to sound.

Moreover, although it has been explained above that the sound source direction identifying device 10 is provided in the robot 1, the present invention is not limited to this. For example, the sound source direction identifying device 10 may be included in a movable surveillance camera. According to this configuration, the camera can be directed in the direction specified by the sound source direction identification device 10. Alternatively, a configuration may be provided that includes a function of recording the sound source direction determined by the sound source direction identification device 10, or a configuration in which the sound source direction determined by the sound source direction identification device 10 is output to a recording device. In addition, the sound source direction identification device 10 may have a configuration in which the sound source direction identifying device 10 records the sounds collected by the microphones MICa to MICc, and a configuration in which the sounds collected by the microphones MICa to MICc are output to a processing device such as a PC. 10 may be provided.

Furthermore, in the above description, in (processing 6), one arrival time difference Tdiff is obtained by weighted averaging for each group, and the subsequent processing is performed. The present invention is not limited to this, and a configuration may be adopted in which the processes after (process 7-1) are performed for each voice frequency index in each group without calculating one arrival time difference Tdiff. In this configuration, in (processing 8), the microcomputer 41 calculates the final sound source direction statistically based on the "number of sound indexes x 2" overall sound source angles calculated in (processing 7-2). Calculate accordingly. Specifically, the microcomputer 41 removes outliers from the total sound source angles calculated in (processing 7-2), excluding outliers, and calculates one sound source direction. calculate. In this way, since the sound source angle is determined for each frequency component and the final sound source angle is determined from the statistics, it is less susceptible to errors caused by frequency components with poor conditions. For example, there are variations in the frequency characteristics of microphones, amplifiers, etc. Therefore, it is conceivable that the calculated phase difference includes a frequency component with a relatively large error and a frequency component with a relatively small error. Therefore, by determining the final sound source angle based on a plurality of frequency components, the accuracy of the sound source angle θ can be made better than when determining the final sound source angle based on one frequency component.

The scope of the present invention is not limited to the configurations explicitly described in the specification, but also includes combinations of various aspects of the invention disclosed herein. Of the present invention, the structure for which a patent is sought has been specified in the attached claims, but currently, even if the structure is not specified in the claims, it is not disclosed in this specification. The applicant intends to make such a configuration the scope of a patent claim in the future.
The present invention is not limited to the configuration described in the embodiments described above. The components of each of the embodiments and modifications described above may be arbitrarily selected and combined. Also, any component of each embodiment or modification, any component described in the means for solving the invention, or a component that embodies any component described in the means for solving the invention. It may be configured in any combination. The applicant intends to obtain rights to these matters through amendments to the application or divisional applications.
In addition, the applicant intends to acquire rights to the entire design or partial design by filing a conversion application to a design application. Although the drawing depicts the entire device using solid lines, the drawing includes not only the overall design but also a partial design that claims some parts of the device. For example, it is a drawing that not only includes some parts of the device as a partial design, but also includes some parts of the device as a partial design regardless of the components. The part of the device may be a part of the device or a part of the device. We intend to obtain rights not only for the entire design, but also for partial designs in which any part of the solid line part of the drawing is a broken line part.

1 Robot 10 Sound source direction identification device 41 Microcomputer
MICa, MICb, MICc Microphone

Claims (4)

A function to record audio collected by multiple microphones,
having a function of identifying a sound source direction based on sounds collected by the plurality of microphones and recording the identified sound source direction ,
The plurality of microphones are three microphones,
Three microphones are placed at the vertices of the triangle.
identifying the direction of the sound source based on the difference in arrival time of sound from the sound source to each of the three microphones;
A camera characterized in that the interior of the casing in which the three microphones are housed has a structure that allows sound to escape, and each of the three microphones is configured to pick up sound from behind as well. The camera according to claim 1, wherein the three microphones arranged at the vertices of the triangle are all omnidirectional microphones. It is characterized by having a function of calculating the difference in arrival time based on a phase difference calculated from each set of two electrical signals out of the three electrical signals output by each of the three microphones. The camera according to claim 1 or 2. The camera according to any one of claims 1 to 3, further comprising a function of specifying a sound source direction based on sounds collected by the plurality of microphones and directing the camera to the specified sound source direction.

Images