JP2020018015A - Acoustic signal processing device, method and program - Google Patents

Abstract

To provide an acoustic signal processing device capable of accurately performing directional sound collection.SOLUTION: The acoustic signal processing device comprises: a direction estimation part 22 configured to estimate an arrival direction of a sound from signals collected by multiple microphones in a manner that a direction closer to the arrival direction which is estimated at a point of time when a predetermined specific sound is detected tends to be estimated as arrival direction; and a first directional sound collection part 23 configured to collect sounds in a manner that sounds from the arrival direction estimated by the direction estimation part 22 is emphasized.SELECTED DRAWING: Figure 6

Description

  The present invention relates to an audio signal processing technique.

  2. Description of the Related Art Directional sound collection techniques described in Patent Literatures 1 and 2 are known (for example, refer to Patent Literatures 1 and 2).

  FIG. 12 shows a configuration of a conventional directional sound collection device disclosed in Patent Documents 1, 2 and the like. The directional sound collecting device of FIG. 12 includes a direction estimating unit 41 and a directional sound collecting unit 42.

  The direction estimation unit 41 estimates the direction of a sound source based on signals collected by a plurality of microphones. Here, the estimation is performed based on the time difference and the amplitude difference generated between the microphones.

  Next, the directional sound collection unit 42 performs directional sound collection such that the sound in the estimated direction is emphasized and collected. The directional sound collection unit 42 can emphasize the sound in the estimated direction by setting the delay time and the filter coefficient so that the sound in the aimed direction is emphasized. According to this directional sound collection technique, if there is one sound source, it is possible to estimate the direction of the sound source and perform sound collection in which the direction of the sound source is emphasized.

JP 2001-309483 A JP 2005-64968 A

  However, in a conventional directional sound collecting device, when both a sound source to be collected and a noise source are present, it is not possible to distinguish which of the sound sources to be collected and to erroneously emphasize the noise source. There was a possibility of operating. For example, assuming a scene in which a robot or a remote control is used in a living room for performing a dialogue or operating a device using voice recognition, there is a possibility that a malfunction may occur due to a reaction to a sound source such as a TV.

  An object of the present invention is to provide an acoustic signal processing device, method, and program for performing highly accurate directional sound collection.

  The sound signal processing device according to an aspect of the present invention is configured such that a direction of arrival of a sound from signals collected by a plurality of microphones is close to an estimated direction of arrival when a specific sound that is a predetermined sound is detected. A direction estimating unit for estimating the direction of arrival of the specific sound so that the direction is more likely to be estimated, and a first directional collecting unit for collecting sound so as to emphasize sounds from the direction of arrival estimated by the direction estimating unit A sound section, and when the direction estimating section estimates a plurality of directions of arrival when a specific sound is detected, a plurality of incoming directions are estimated using the directions of arrival estimated in a predetermined section in the past. Select the arrival direction of the specific sound from the directions.

  By performing the acoustic signal processing based on the information obtained from the specific sound obtained in advance, more accurate directional sound collection can be performed.

FIG. 2 is a block diagram for explaining an example of the acoustic signal processing device according to the first embodiment. FIG. 7 is a block diagram for explaining an example of an acoustic signal processing device according to a first modification of the first embodiment. FIG. 9 is a block diagram for explaining an example of an acoustic signal processing device according to a second modification of the first embodiment. FIG. 9 is a block diagram for explaining an example of an acoustic signal processing device according to a third modification of the first embodiment. 5 is a flowchart for explaining an example of a sound signal processing method. FIG. 6 is a block diagram for explaining an example of an acoustic signal processing device according to a second embodiment. FIG. 9 is a block diagram for explaining an example of a direction estimating unit 22 according to the second embodiment. FIG. 9 is a block diagram for explaining an example of a direction estimating unit 22 according to the second embodiment. FIG. 11 is a block diagram for explaining an example of an acoustic signal processing device according to a second modification of the second embodiment. FIG. 13 is a block diagram for explaining an example of an acoustic signal processing device according to a third modification of the second embodiment. 5 is a flowchart for explaining an example of a sound signal processing method. FIG. 2 is a block diagram illustrating an example of a directional sound collecting device according to the background art. FIG. 9 is a functional block diagram of an acoustic signal processing device according to a third embodiment. The figure showing the example of the processing flow of the acoustic signal processor concerning a third embodiment. FIG. 14 is a functional block diagram of a voice section detection information storage unit according to the third embodiment. The figure for demonstrating a specific sound voice section and a non-voice section. FIG. 13 is a functional block diagram of a voice section detection unit according to the third embodiment. FIG. 13 is a functional block diagram of a first acoustic signal analyzer according to the third embodiment. FIG. 13 is a functional block diagram of a probability estimating unit according to the third embodiment. FIG. 14 is a functional block diagram of a voice section detection unit according to a first modification of the third embodiment. FIG. 14 is a functional block diagram of an acoustic signal processing device according to a third modification and a fourth modification of the third embodiment. The figure showing the example of the processing flow of the acoustic signal processor concerning the 3rd modification and the 4th modification of a third embodiment. FIG. 13 is a block diagram for explaining an example of an acoustic signal processing device according to a modification 4 of the first embodiment. 9 is a flowchart for explaining an example of an acoustic signal processing method according to Modification 4 of the first embodiment.

  In the drawings used in the following description, components having the same functions and steps for performing the same processing are denoted by the same reference numerals, and redundant description is omitted. In the following description, the symbol "^" or the like used in the text should be described immediately above the character immediately after it, but is described immediately before the character due to the limitation of the text notation. In the formula, these symbols are described in their original positions. The processing performed for each element of a vector or matrix is applied to all elements of the vector or matrix unless otherwise specified.

[Technical background]
The sound signal processing device performs sound signal processing using information on a specific sound, which is a predetermined sound, assuming that the information is given. By using information about the specific sound given in advance, usable information is increased, so that more accurate acoustic signal processing can be performed.

  Examples of the acoustic signal processing are estimation of a direction of arrival of sound, directional sound collection, extraction of a target sound, detection of a sound section, and sound recognition.

  For example, by detecting a keyword which is a specific sound for a specific utterance of the user, a signal section of a target voice and a signal section of noise can be accurately grasped, and can be used for subsequent processing.

  Further, when this property is used for voice section detection, the signals in the noise section and the voice section are respectively identified, so that the parameter for voice / non-voice determination can be updated to a value that more closely matches the actually measured value.

  In addition, when sound direction estimation is performed as acoustic signal processing, the direction in which a specific sound is detected is regarded as the direction of the sound, so that even when sound including sound arrives from a direction other than the original direction, the direction estimation is robust. Operate.

  In addition, when the target voice is extracted as the audio signal processing, the signals of the voice section and the non-voice section can be obtained with high accuracy, so that the spatial correlation matrix for calculating the steering vector for voice separation is more accurately obtained. be able to.

  In the case of performing speech recognition as acoustic signal processing, the noise level can be obtained more accurately, so that the accuracy can be improved by selecting an acoustic model.

  Hereinafter, each embodiment will be described with reference to the drawings.

[First embodiment]
The sound signal processing device and method of the first embodiment perform directional sound collection processing as sound signal processing.

  As shown in FIG. 11, the acoustic signal processing device includes, for example, a direction estimation unit 11, a specific sound detection unit 12, a direction storage unit 13, and a first directional sound collection unit 14. The acoustic signal processing device does not need to include the specific sound detection unit 12.

  The acoustic signal processing method is realized, for example, by the acoustic signal processing device performing the processing of FIG. 5 and steps S11 to S14 described below.

  The direction estimating unit 11 estimates a direction of arrival of sound from signals collected by a plurality of microphones (step S11). The direction estimating unit 11 estimates a direction of arrival of a sound at each time. The estimated arrival direction of the sound at each time is output to the direction storage unit 13.

  The method of direction estimation by the direction estimating unit 11 is arbitrary. The direction estimating unit 11 estimates the direction of arrival of the sound using, for example, the direction estimating technology described in Patent Documents 1 and 2. The arrival direction of the sound may be represented not by the direction but by the position.

  The specific sound detector 12 detects a specific sound that is a predetermined sound (step S12). Examples of predetermined sounds are the sound, whistle and clapping of a particular keyword. A predetermined sound other than the above example may be used as the predetermined sound.

  The direction storage unit 13 stores the arrival direction estimated by the direction estimation unit 11 at the time when the specific sound is detected by the specific sound detection unit 12. More specifically, the direction storage unit 13 stores the arrival direction of the sound at the time when the specific sound is detected by the specific sound detection unit 12 among the arrival directions of the sound at each time input from the direction estimation unit 11. .

  The first directional sound collection unit 14 performs sound collection so that the sound from the arrival direction read from the direction storage unit 13 is emphasized (step S14). The method of directional sound collection by the first directional sound collection unit 14 is arbitrary. The first directional sound collection unit 14 performs directional sound collection described in, for example, JP-A-2009-44588.

  As described above, the sound source emitting the specific sound is determined to be a sound source to be collected, and the sound source is subjected to directional sound collection, whereby sound can be collected with a high SN ratio. The user can change the direction of the directivity by emitting a specific sound such as a specific keyword. Even when a sound source such as a television is present, the user directs the directivity to himself and then fixes it. be able to.

  If it takes a long time for the specific sound detection unit 12 to detect the specific sound, a delay unit 15 that delays by a time corresponding to the time may be provided after the direction estimation unit 11. In FIG. 1, the delay unit 15 is indicated by a broken line. The delay unit 15 delays the output from the direction estimation unit 11 by a time corresponding to the time of detection of the specific sound by the specific sound detection unit 12 and then inputs the output to the direction storage unit 13. As a result, even if there is a delay in the detection of the specific sound, it operates normally.

[[Modification 1 of First Embodiment]]
As illustrated in FIG. 2, the acoustic signal processing device may further include an estimated frequency measurement unit 16 and a selection unit 17. In this case, the direction estimating unit 11 may be capable of simultaneous estimation in a plurality of directions. That is, when there is a sound of a noise source at the same time as a specific sound, the direction estimation unit 11 may be able to estimate the directions of both sound sources. In this case, it becomes impossible to determine which sound source produced the specific sound, so the estimation frequency measurement unit 16 makes a determination based on how much direction estimation has been performed in the past. That is, since the sound source such as a TV always outputs sound, it is considered that a large number of direction estimations have been performed in the past, and the estimation frequency measurement unit 16 determines this using the clue as a clue.

  The estimated frequency measuring unit 16 measures the frequency of the arrival direction estimated by the direction estimating unit 11 in a predetermined time section in the past (step S16). That is, the estimated frequency measurement unit 16 measures how frequently the direction has been estimated within a fixed time in the past. Information about the measured frequency is output to the selection unit 17.

  For example, if the time during which the output of the direction estimating unit 11 was in the direction θ during the past T seconds is A (θ) seconds, the frequency of estimation in the θ direction will be the ratio D (θ) = A (θ ) / T. The estimated frequency measurement unit 16 obtains all the frequencies for each direction. Assuming that the noise source is a television or a speaker for listening to music, sound will be emitted from the same direction for a long time with almost no silence. When such a sound source exists in the θ direction, the estimated frequency D (θ) takes a large value close to 1.

  The selecting unit 17 selects the arrival direction with the lowest frequency among the frequencies measured by the estimated frequency measuring unit 16. For example, when the number of estimated directions of the output of the direction estimating unit 11 is two, the selecting unit 17 selects the one with the smaller estimated frequency D (θ). The arrival direction selected by the selection unit 17 at the time when the specific sound is detected by the specific sound detection unit 12 is stored in the direction storage unit 13.

  After that, the first directional sound collection unit 14 performs sound collection so that the sound from the arrival direction read from the direction storage unit 13 is emphasized in the same manner as described above.

  Also in the first modification of the first embodiment, when it takes time to detect the specific sound by the specific sound detection unit 12, the delay unit 15 that delays by a time corresponding to the time is added to the subsequent stage of the direction estimation unit 11. You may put in. In FIG. 2, the delay unit 15 is indicated by a broken line. As a result, even if there is a delay in the detection of the specific sound, it operates normally.

[[Modification 2 of the first embodiment]]
As illustrated in FIG. 3, the acoustic signal processing device may further include a second directional sound collection unit 18.

  By performing the directional sound collection by the second directional sound collection unit 18 before the processing of the specific sound detection unit 12, more specific detection of the specific sound can be performed.

  A signal obtained by delaying a signal collected by a plurality of microphones is input to the second directional sound collection unit 18. This delay has a time length corresponding to the time required for the arrival direction estimation processing by the direction estimation unit 11. This delay is performed by the delay unit 19 shown by a broken line in FIG. The direction of arrival estimated by the direction estimating unit 11 is input to the second directional sound collection unit 18.

  The second directional sound collection unit 18 performs sound collection such that the sound from the arrival direction estimated by the direction estimation unit 11 is emphasized (step S18). More specifically, the second directional sound collection unit 18 emphasizes the sound from the arrival direction estimated by the direction estimation unit 11 by using a signal obtained by delaying the signals collected by the plurality of microphones. So that sound is collected. The signal collected by the second directional sound collection unit 18 is output to the specific sound detection unit 12.

  The specific sound detector 12 detects a specific sound based on the signal collected by the second directional sound collector 18. Subsequent processing is the same as described above.

  As shown in FIG. 3, a plurality of second directional sound collection units 18 may be provided in the acoustic signal processing device. In this case, as many specific sound detectors 12 as the number of the second directional sound collectors 18 are provided in the acoustic signal processing device.

  In this case, when the direction estimating unit 11 estimates a plurality of directions of arrival, the specific sound detecting unit 12 operates to emphasize each of the estimated directions of arrival, and outputs a plurality of outputs each. Is input to the specific sound detection unit 12 to detect the specific sound.

  Thereby, when the specific sounds are detected by the plurality of specific sound detection units 12, it is possible to assign a priority.

  Also in the second modification of the first embodiment, when it takes time for the specific sound to be detected by the specific sound detection unit 12, the delay unit 15 that delays by a time corresponding to the time is added to the downstream of the direction estimation unit 11. You may put in. In FIG. 2, the delay unit 15 is indicated by a broken line. As a result, even if there is a delay in the detection of the specific sound, it operates normally.

[[Modification 3 of First Embodiment]]
As illustrated in FIG. 4, in Modification 2 of the first embodiment, the acoustic signal processing device may further include the estimated frequency measurement unit 16 and the selection unit 17 described in Modification 1 of the first embodiment. . In this case, the direction estimating unit 11 may be capable of simultaneous estimation in a plurality of directions. That is, when there is a sound of a noise source at the same time as a specific sound, the direction estimation unit 11 may be able to estimate the directions of both sound sources.

  The processing of the estimation frequency measurement unit 16 and the selection unit 17 is the same as that described in the first modification of the first embodiment.

  That is, the estimated frequency measuring unit 16 measures the frequency of the arrival direction estimated by the direction estimating unit 11 in a predetermined time section in the past (step S16). That is, the estimated frequency measurement unit 16 measures how frequently the direction has been estimated within a fixed time in the past. Information about the measured frequency is output to the selection unit 17.

  For example, if the time during which the output of the direction estimating unit 11 was in the direction θ during the past T seconds is A (θ) seconds, the frequency of estimation in the θ direction will be the ratio D (θ) = A (θ ) / T. The estimated frequency measurement unit 16 obtains all the frequencies for each direction. Assuming that the noise source is a television or a speaker for listening to music, sound will be emitted from the same direction for a long time with almost no silence. When such a sound source exists in the θ direction, the estimated frequency D (θ) takes a large value close to 1.

  The selecting unit 17 selects the arrival direction with the lowest frequency among the frequencies measured by the estimated frequency measuring unit 16. For example, when the number of estimated directions of the output of the direction estimating unit 11 is two, the selecting unit 17 selects the one with the smaller estimated frequency D (θ). The arrival direction selected by the selection unit 17 at the time when the specific sound is detected by the specific sound detection unit 12 is stored in the direction storage unit 13.

  After that, the first directional sound collection unit 14 performs sound collection so that the sound from the arrival direction read from the direction storage unit 13 is emphasized in the same manner as described above.

  Also in the first modification of the first embodiment, when it takes time to detect the specific sound by the specific sound detection unit 12, the delay unit 15 that delays by a time corresponding to the time is added to the subsequent stage of the direction estimation unit 11. You may put in. In FIG. 4, the delay unit 15 is indicated by a broken line. As a result, even if there is a delay in the detection of the specific sound, it operates normally.

[[Modification 4 of the first embodiment]]
As illustrated in FIG. 23, the acoustic signal processing device may include a third directional sound collection unit 52 instead of the first directional sound collection unit 14, and further include a noise direction storage unit 51.

  The sound signal processing method is realized, for example, by the sound signal processing device performing the processing of FIG. 24 and step S31 described below.

  The direction of arrival estimated by the direction estimating unit 11 except for the time at which the specific sound is detected by the specific sound detecting unit 12 is stored in the noise direction storage unit 51. Here, "excluding the time at which the specific sound is detected" may be a time before the time at which the specific sound is detected or a time after the time at which the specific sound is detected. Both the previous time and the later time may be used. Needless to say, the delay unit 15 may be inserted before the noise direction storage unit 51 and after the direction estimation unit 11.

The third directional sound collection unit 52 performs sound collection such that sound from the arrival direction read from the direction storage unit 13 is emphasized and sound from the arrival direction read from the noise direction storage unit 51 is suppressed. Perform (Step S52). The method of the directional sound collection by the third directional sound collection unit 52 is arbitrary. As a method of the directional sound collection performed by the third directional sound collection unit 52, for example, a method described in Reference Document 5 may be used.
(Reference 5) Tadashi Asano, "Sound Array Signal Processing", pp.82-85, Corona, 2011.

[Second embodiment]
The sound signal processing device and method of the first embodiment perform directional sound collection processing as sound signal processing.

  As shown in FIG. 6, the acoustic signal processing device includes, for example, a specific sound detecting unit 21, a direction estimating unit 22, and a first directional sound collecting unit 23. The acoustic signal processing device does not need to include the specific sound detection unit 12.

  The sound signal processing method is realized, for example, by the sound signal processing device performing the processing of FIG. 11 and steps S21 to S23 described below.

  The specific sound detection unit 21 detects a specific sound that is a predetermined sound (step S21). Examples of predetermined sounds are the sound, whistle and clapping of a particular keyword. A predetermined sound other than the above example may be used as the predetermined sound.

  The direction estimating unit 22 estimates the direction of arrival of the sound from the signals collected by the plurality of microphones (Step S22). At this time, the direction estimating unit 22 determines the direction of arrival of the sound from the signals collected by the plurality of microphones as the direction closer to the direction of arrival estimated at the time when the specific sound is detected by the specific sound detecting unit 21. Is estimated so as to be easily estimated.

  That is, the direction estimating unit 22 sets the ease of detection in each direction according to the result of the detection of the specific sound. In other words, in the direction estimating unit 22, the direction is more likely to be detected as the direction is closer to the direction estimated at the time of detection of the specific sound, and it is more difficult to detect the direction as it is farther. By doing so, the directivity of the user who emits the specific sound is more likely to be directed, and the directivity is less likely to be directed to the noise source. Further, even if the user who emits the specific sound moves, it can follow the movement.

  FIG. 7 shows an example of the configuration of the direction estimating unit 22. As illustrated in FIG. 7, the direction estimating unit 22 includes a direction emphasizing unit 221, a power calculating unit 222, a weight multiplying unit 223, a maximum power direction detecting unit 224, and a weight determining unit 225.

  Each of the signals collected by the plurality of microphones is input to the direction enhancing unit 221.

  The direction emphasizing unit 221 performs a direction emphasizing process on the signals collected by the plurality of microphones so as to emphasize a plurality of directions, respectively (step S221). For example, when N direction emphasizing units 221 are provided, θ1, θ2,..., ΘN are set to directions different from each other, and N direction emphasizing units 221 are arranged in directions of θ1, θ2,. Is performed to emphasize the direction. The emphasized signal is output to power calculation section 222.

  The power calculator 222 calculates the power of the signal emphasized by the direction enhancer 221 (step S222). The calculated power is output to weight multiplying section 223.

  The weight multiplication unit 223 multiplies the power calculated by the power calculation unit 222 by the weight set by the weight setting unit 225 (step S223). The weighted power is output to the maximum power direction detection unit 224. As described later, therefore, the weight multiplying unit 223 multiplies the power of the signal in which each direction of arrival is emphasized by a larger weight as the direction of arrival is closer to the selected direction of arrival, so that the weighted power is increased. Get.

  The maximum power direction detection unit 224 selects the arrival direction of the maximum power from the output of the weight multiplication unit 223. In other words, the maximum power direction detection unit 224 selects the direction of arrival with the largest power after weighting, and sets the selected direction of arrival as the estimated direction of arrival (step S224). The estimated direction of arrival is output to the weight determination unit 225 and the first directional sound collection unit 23 as a direction estimation result.

  The weight setting unit 225 determines the weight corresponding to the direction estimation result output by the maximum power direction detection unit 224 at the time when the specific sound is detected by the specific sound detection unit 21. The determined weight is output to the weight multiplier 223. In other words, the weight setting unit 225 sets a weight corresponding to the direction estimation result when a specific sound is detected.

  The weight corresponding to the direction estimation result is set such that the weight for the estimated direction of arrival increases, and the weight decreases as the distance from the direction of arrival increases. For example, the weight for the estimated direction of arrival is set to 1.0, and a weight that is multiplied by a multiplier less than 1.0 (for example, 0.8) every 10 degrees from the estimated direction of arrival is set.

  The first directivity sound collection unit 23 performs sound collection so that the sound from the arrival direction estimated by the direction estimation unit 22 is emphasized (step S23). The method of the directional sound collection by the first directional sound collection unit 23 is arbitrary. The first directional sound collection unit 23 performs directional sound collection described in, for example, JP-A-2009-44588.

  As described above, the sound source emitting the specific sound is determined to be a sound source to be collected, and the sound source is subjected to directional sound collection, whereby sound can be collected with a high SN ratio. The user can change the direction of the directivity by emitting a specific sound such as a specific keyword. Even when a sound source such as a television is present, the user directs the directivity to himself and then fixes it. be able to.

  If it takes time for the specific sound detection unit 21 to detect a specific sound, a delay unit 226 that delays by a time corresponding to that time may be provided after the maximum power direction detection unit 224. In FIG. 7, the delay unit 226 is indicated by a broken line. The delay unit 226 delays the output from the maximum power direction detection unit 224 by a time corresponding to the time of detection of the specific sound by the specific sound detection unit 21 and then inputs the output to the weight setting unit 225. As a result, even if there is a delay in the detection of the specific sound, it operates normally.

[[Modification 1 of Second Embodiment]]
As illustrated in FIG. 8, the acoustic signal processing device may further include an estimated frequency measurement unit 227 and a selection unit 228.

  In this case, the maximum power direction detection unit 224 may be able to perform simultaneous estimation in a plurality of directions by detecting all power directions exceeding a predetermined threshold. That is, the maximum power direction detection unit 224 detects the direction of the maximum power, and further detects the direction of the maximum power except for the detected direction. The maximum power direction detection unit 224 ends the maximum power detection when the preset maximum number of estimated directions is reached or when the maximum power becomes equal to or less than a preset threshold value. The maximum power direction detection unit 224 may be capable of simultaneously estimating the directions of a plurality of sound sources by such a method, for example. This allows the maximum power direction detection unit 224 to estimate the directions of both sound sources when there is a sound of a noise source at the same time as a specific sound.

  In this case, it becomes impossible to determine which sound source produced the specific sound, so the estimation frequency measurement unit 227 makes a determination based on how much direction estimation has been performed in the past. That is, since the sound source such as a TV always outputs sound, it is considered that a large number of direction estimations have been performed in the past, and the estimation frequency measurement unit 227 determines this using the clue as a clue.

  The estimated frequency measuring unit 227 measures the frequency of the incoming direction estimated by the direction estimating unit 22, in other words, the frequency of the incoming direction selected by the maximum power direction detecting unit 22 in a predetermined time section in the past (step). S16). That is, the estimated frequency measurement unit 227 measures how frequently the direction has been estimated within a fixed time in the past. Information about the measured frequency is output to the selection unit 228.

  For example, if the time during which the output of the maximum power direction detection unit 224 was in the direction θ during the past T seconds is A (θ) seconds, the estimated frequency in the θ direction will be the ratio D (θ) = A (θ) / T. The estimated frequency measurement unit 227 obtains this frequency for each direction. Assuming that the noise source is a television or a speaker for listening to music, sound will be emitted from the same direction for a long time with almost no silence. When such a sound source exists in the θ direction, the estimated frequency D (θ) takes a large value close to 1.

  The selection unit 228 selects the arrival direction with the lowest frequency among the frequencies measured by the estimated frequency measurement unit 227. For example, when the number of estimated directions of the output of the maximum power direction detecting unit 22 is two, the selecting unit 228 selects the one with the smaller estimated frequency D (θ). The selected arrival direction is output to weight setting section 225.

  If it takes time for the specific sound detection unit 21 to detect a specific sound, a delay unit 226 that delays by a time corresponding to that time may be provided after the maximum power direction detection unit 224. In FIG. 8, the delay unit 226 is indicated by a broken line. The delay unit 226 delays the output from the maximum power direction detection unit 224 by a time corresponding to the time of detection of the specific sound by the specific sound detection unit 21 and then inputs the output to the weight setting unit 225. As a result, even if there is a delay in the detection of the specific sound, it operates normally.

[[Modification 2 of Second Embodiment]]
As illustrated in FIG. 9, the acoustic signal processing device may further include a second directional sound collection unit 24.

  By performing the directional sound collection by the second directional sound collection unit 24 before the processing of the specific sound detection unit 21, it is possible to detect the specific sound with higher accuracy.

  A signal obtained by delaying a signal collected by a plurality of microphones is input to the second directional sound collection unit 24. This delay has a time length corresponding to the time required for the arrival direction estimation processing by the direction estimation unit 22. This delay is performed by the delay unit 25 shown by a broken line in FIG. The direction of arrival estimated by the direction estimating unit 22 is input to the second directional sound collecting unit 24.

  The second directivity sound collection unit 24 performs sound collection such that the sound from the arrival direction estimated by the direction estimation unit 22 is emphasized (step S24). More specifically, the second directional sound collection unit 24 emphasizes the sound from the arrival direction estimated by the direction estimation unit 22 using a signal obtained by delaying the signals collected by the plurality of microphones. So that sound is collected. The signal collected by the second directional sound collection unit 24 is output to the specific sound detection unit 21.

  The specific sound detector 21 detects a specific sound based on the signal collected by the second directional sound collector 24. Subsequent processing is the same as described above.

  In addition, as shown in FIG. 9, a plurality of second directional sound collection units 24 may be provided in the acoustic signal processing device. In this case, as many specific sound detectors 21 as the number of the second directivity sound collectors 24 are provided in the acoustic signal processing device.

  In this case, when the direction estimating unit 22 estimates a plurality of directions of arrival, the specific sound detecting unit 21 operates to emphasize each of the estimated directions of arrival, and outputs a plurality of outputs of each. Is input to the specific sound detection unit 21 to detect the specific sound.

  Thereby, when the specific sounds are detected by the plurality of specific sound detection units 21, it is possible to assign a priority.

[[Modification 3 of Second Embodiment]]
As illustrated in FIG. 10, in Modification 2 of the second embodiment, the acoustic signal processing device may further include an estimated frequency measurement unit 26 and a selection unit 27. In this case, the direction estimating unit 22 may be capable of simultaneous estimation in a plurality of directions. In other words, the direction estimation unit 22 may be able to estimate the directions of both sound sources when there is a sound of a noise source simultaneously with the specific sound.

  The processing of the estimation frequency measurement unit 26 and the selection unit 27 is the same as that described in the first modification of the first embodiment.

  That is, the estimated frequency measuring unit 26 measures the frequency of the arrival direction estimated by the direction estimating unit 22 in a predetermined past time section (step S26). That is, the estimated frequency measuring unit 26 measures how frequently the direction has been estimated within a certain period of time in the past. Information about the measured frequency is output to the selection unit 27.

  For example, if the time during which the output of the direction estimating unit 22 is in the direction θ during the past T seconds is A (θ) seconds, the frequency of estimation in the θ direction will be the ratio D (θ) = A (θ ) / T. The estimated frequency measuring unit 26 calculates this frequency for each direction. Assuming that the noise source is a television or a speaker for listening to music, sound will be emitted from the same direction for a long time with almost no silence. When such a sound source exists in the θ direction, the estimated frequency D (θ) takes a large value close to 1.

  The selecting unit 27 selects the arrival direction with the lowest frequency among the frequencies measured by the estimated frequency measuring unit 26 (Step S27). For example, when the number of estimated directions of the output of the direction estimating unit 22 is two, the selecting unit 27 selects the one with the smaller estimated frequency D (θ). The arrival direction selected by the selection unit 27 at the time when the specific sound is detected by the specific sound detection unit 21 is output to the direction estimation unit 22 and is set as the arrival direction estimated by the direction estimation unit 22.

  Thereafter, the first directional sound collection unit 23 performs sound collection in a manner similar to the above, so that the sound from the arrival direction estimated by the direction estimation unit 22 is emphasized.

[Third embodiment]
The acoustic signal processing device and method according to the third embodiment detect a voice section as acoustic signal processing.

<Points of the third embodiment>
In the present embodiment, information on the usage environment (such as noise) is more correctly obtained by narrowing down the utterance contents of the user. For example, a restriction is made so that a specific word (keyword) is uttered before the user starts speaking. At this time, it is assumed that only the specific word voice can be detected with high accuracy, and it is assumed that “the section is voice” and “the previous section is noise”. Then, the information for the determination of “speech / non-speech” is updated using the speech in the noise section and the speech section.

  By doing so, the information of "noise" and "speech" more suitable for the actual use environment can be used when determining the section of the target sound to be emitted thereafter, and the accuracy of section detection is improved.

  Hereinafter, an embodiment of an audio signal processing apparatus and method will be described. The acoustic signal processing device is realized by a computer such as a dedicated machine constituted by dedicated hardware or a general-purpose machine such as a personal computer. Here, a case will be described where the processing is realized by a computer (general-purpose machine).

  An example of a hardware configuration of the acoustic signal processing device will be described.

  The acoustic signal processing device has an input unit to which a keyboard and a pointing device can be connected, an output unit to which a liquid crystal display and a CRT (Cathode Ray Tube) display can be connected, and a communication device ( For example, a communication unit to which a communication cable, a LAN card, a router, a modem, and the like can be connected, and a CPU (Central Processing Unit) [DSP (Digital Signal Processor)] may be used. Further, a cache memory or a register may be provided. And external storage devices such as RAM, ROM, hard disk, optical disk, and semiconductor memory, and data exchange between these input units, output units, communication units, CPU, RAM, ROM, and external storage devices. It has a bus connecting as possible. If necessary, the audio signal processing device may be provided with a device (drive) that can read and write a storage medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), and a DVD (Digital Versatile Disc).

  Further, the acoustic signal processing device can be connected to an acoustic signal collecting means (for example, a microphone) for receiving sounds such as voice, music, noise, etc., and input an (analog) signal obtained by the microphone. A signal input unit for receiving the signal and a sound output device (for example, a speaker) for outputting a reproduced signal as sound, and a signal for outputting a signal to be input to the speaker (D / A converted from the reproduced signal) It is also possible to adopt a configuration in which an output unit is provided. In this case, a microphone is connected to the signal input unit, and a speaker is connected to the signal output unit.

In the external storage device of the acoustic signal processing device, a program for voice section detection and data necessary for processing of the program are stored (not limited to the external storage device, for example, the program is read-only storage device). It may be stored in a certain ROM.
]. Further, data and the like obtained by the processing of this program are appropriately stored in a RAM, an external storage device, or the like. Hereinafter, a storage unit that stores data, an address of a storage area thereof, and the like will be simply referred to as an “「 storage unit ”.

  In this embodiment, in order to acquire a signal in a section chronologically earlier than a speech section included in the sound signal, a sound signal that is a discrete signal is stored in the main storage unit. This storage may be temporary storage such as a buffer.

<Configuration of acoustic signal processing device>
FIG. 13 is a functional block diagram of the acoustic signal processing device according to the third embodiment, and FIG. 14 shows a processing flow thereof.

  The acoustic signal processing device includes a voice section detection section 320 and a voice section detection information storage section 330.

  The audio signal processing device receives the time-series audio signal collected by one microphone 310 and the output value of the specific audio section detection unit 340 as inputs, and determines the audio section and the non-voice section included in the time-series audio signal. At least one is detected and a detection result is output.

Note that the specific sound section detection unit 340 detects that a predetermined sound (hereinafter, also referred to as “specific sound”) has come, and outputs information indicating a detection time of the specific sound. In the present embodiment, the specific sound is a predetermined sound emitted by a person, for example, a sound generated when a person emits a predetermined keyword. For example, the specific voice section detection unit 340 can be implemented using a technique such as “phrase spotting” as in Reference 1.
(Reference 1) "Sensory Company's speech technology description", [online], 2010, [Search on July 24, 2017], Internet <URL: http://www.sensory.co.jp/Parts/Docs /SensoryTechnologyJP1003B.pdf>
Note that the information indicating the detection time of the specific sound is information indicating at least the time at which the specific sound (for example, the keyword) has been completed, and (1-i) the time at which the specific sound has been completed may be output. (1-ii) The frame number of the time-series sound signal corresponding to the time at which the specific sound is finished may be output, or (1-iii) the frame number may be detected at a frame time other than the time at which the specific sound is finished. Outputs information indicating that the specific sound has not been output (for example, "0"), and outputs information indicating that detection has been performed at the time when the specific sound has been completed (for example, "1") to indicate the time at which the specific sound has been completed The information may be information or information indicating the time at which another specific sound is finished. Further, the information indicating the detection time of the specific sound may include information indicating the time at which the specific sound is started, and (2-i) the time at which the specific sound is started and the end of the specific sound may be output. Or (2-ii) the time at which the specific sound started to be spoken and the frame number of the time-series sound signal corresponding to the time at which the specific sound was finished may be output, or (2-iii) the time at which the specific sound began to be spoken To output the information (for example, “1”) indicating that the sound was detected from the time until the end of the word, and output the information (for example, “0”) indicating that the sound was not detected at other times. It may be information indicating the time at which the user has finished speaking, or may be information indicating the time at which another specific sound has been completed.

  Hereinafter, the processing content of each unit will be described.

<Voice section detection information storage section 330>
The voice section detection information storage unit 330 receives information indicating the detection time of the specific sound and the time-series sound signal as inputs, and stores a feature amount of the time-series sound signal corresponding to the specific sound voice section in frame units and a non-voice section. The characteristic amount of the corresponding time-series sound signal is obtained (S330) and output. In each unit including the voice section detection information storage unit 330, each process is performed on a frame basis.

  As shown in FIG. 15, the voice section detection information storage section 330 includes a voice storage section 331, a specific sound voice section calculation section 332, and a feature amount calculation section 333. Hereinafter, the processing content of each unit will be described.

(Voice storage unit 331)
The sound accumulation unit 331 receives and accumulates the time-series sound signal of the sound section detection target.

(Specific sound voice section calculation unit 332)
The specific sound voice section calculation unit 332 receives information indicating the detection time of the specific sound as input, sets a section of the time-series sound signal estimated to be a section corresponding to the specific sound based on the detection time as a specific sound voice section, , A section of the time-series sound signal that is estimated not to be a section corresponding to the specific sound is determined to be a non-voice section, and information indicating a specific sound voice section and information indicating a non-voice section are output. For example, a specific sound voice section is defined as t ₁ second before the specific sound detection time (in this example, the time at which the specific sound is finished), and a non-voice section is determined as t ₂ seconds before the specific sound voice section. (See FIG. 16).

For example, when the information indicating the detection time of the specific sound includes only the information indicating the frame time (for example, t) at which the specific sound is finished, t ₁ and t ₂ are set to predetermined values in advance. Then, a specific sound voice section (from tt ₁ to t) and a non-voice section (from tt ₁ -t ₂ to tt ₁ ) are obtained from the information indicating the detection time of the specific sound. It may be used an average value or the like of the time when triggered by a particular sound as t _1. Further, as information indicating the detection time of a specific sound, if it contains information that indicates a specific sound to say beginning time and Iowa' time (eg, t), the time began to say a particular sound and tt _1, specific and up to time t Iowa' the sound voice interval from the time tt ₁ began to say a specific sound.
Also, previously set to a predetermined value t _2, the predetermined value t _2, determine the non-speech section (from tt ₁ -t ₂ to tt ₁₎ from the time tt ₁ Metropolitan began to say a particular sound.

(Feature calculation unit 333)
The feature amount calculation unit 333 receives the information indicating the specific sound voice section and the information indicating the non-voice section from the specific sound voice section calculation unit 332, and converts the time-series sound signal of the voice section detection target stored in the voice storage unit 331. receive. Then, the feature amount calculation unit 333 associates the time-series sound signal with the specific sound voice section, associates the time-series sound signal with the non-voice section, and extracts the characteristic from the time-series sound signal corresponding to the specific sound voice section. A speech section feature amount as a quantity is calculated, a non-speech section feature quantity as the feature quantity is calculated from a time-series sound signal corresponding to the non-speech section, and a speech section feature quantity and a non-speech section feature quantity are output. As the feature amount, for example, a logarithmic mel spectrum, a cepstrum coefficient, or the like can be used. However, it is preferable to use an acoustic feature amount other than the acoustic feature amount (fundamental frequency) used by the second acoustic signal analysis unit 322. Any method may be used to calculate the feature amount. For example, the method described in Reference 4 is used.
(Reference Document 4) JP 2009-63700 A

<Voice section detection section 320>
The voice section detection unit 320 receives a time-series sound signal from the microphone 310, and receives a voice section feature value and a non-voice section feature value from the feature value calculation unit 333. The voice section detection unit 320 obtains a voice parameter indicating a feature of the voice section from the voice section feature quantity, obtains a non-voice parameter indicating a feature of the non-voice section from the non-voice section feature quantity, and determines the voice parameter and the non-voice parameter. Then, at least one of a voice section and a non-voice section is detected from the time-series sound signal (S320), and a detection result is output.

  For example, the voice section detection unit 320 obtains a voice parameter, which is a parameter of an acoustic model used when estimating a voice section, from the voice section feature amount, and is a parameter of the acoustic model used when estimating a non-voice section. The non-speech parameter is obtained from the non-speech section feature amount.

  For example, the voice section detection device of Reference 4 can be used for the voice section detection unit 320. In this case, the voice parameter is a voice GMM parameter, and the non-voice parameter is a non-voice GMM parameter.

  As shown in FIG. 17, the speech section detection unit 320 includes a first acoustic signal analysis unit 321 that performs a probability calculation on an input time-series audio signal using a parallel Kalman filter / parallel Kalman smoother, and a time-series audio signal. A second acoustic signal analysis unit 322 that calculates a probability using the ratio of the periodic component to the non-periodic component, a weight calculation unit 323 that calculates the weight of each probability, and a time A speech state / non-speech state synthesis probability ratio calculating unit 324 for calculating a synthesis probability that a sequence acoustic signal belongs to a speech state and a synthesis probability belonging to a non-speech state, and a speech state / non-speech state synthesis probability ratio And a voice section estimating unit 325 that performs voice / non-voice recognition based on. Note that the configuration other than the first acoustic signal analysis unit 321 performs the same processing as that in Reference Document 4, and a description thereof will be omitted.

  The time-series sound signal input to the first sound signal analysis unit 321 is a sound signal sampled at a sampling rate of, for example, 8,000 Hz and converted into a discrete signal. This acoustic signal is a sound in which a noise signal is superimposed on a sound signal as a target signal. Hereinafter, the acoustic signal is referred to as “input signal”, the audio signal is referred to as “clean audio”, and the noise signal is referred to as “noise”.

  The voice section detection unit 320 receives the input signal, the voice section feature amount, and the non-voice section feature amount, and outputs a voice section detection result. The voice section detection result takes 1 if the audio signal of each frame belongs to a voice state, and takes 0 if it belongs to a non-voice state. The voice section detection unit 320 may output a signal obtained by multiplying the input signal by the value of the voice section detection result. That is, the value of the input signal of the frame belonging to the voice state is held, and the value of the signal is all replaced with 0 in the frame belonging to the non-voice state.

<First acoustic signal analyzer 321>
As shown in FIG. 18, the first acoustic signal analysis unit 321 receives the input signal, the voice section feature quantity, and the non-voice section feature quantity, and extracts a feature quantity used for voice section detection. 3211 and a probability estimating unit 3212 for estimating a probability model parameter and calculating a probability of an input signal using a probability model constituted by the obtained probability model parameters.

(Feature amount calculation unit 3211)
The feature amount calculation unit 3211 calculates the feature amount from the input signal and outputs the same using the same method as the feature amount calculation unit 333. For example, a vector G _t = {gt _{, 0} ,..., Gt _{, φ} ,..., Gt _{, 23} } having a 24-dimensional logarithmic mel spectrum as an element is calculated and output. Vector G _t is the time of the start point of the cut represents the acoustic features in the frame of t. φ indicates the element number of the vector. Hereinafter, t is referred to as a frame time.

(Probability estimation unit 3212)
The 24-dimensional logarithmic mel spectrum output from the feature amount calculation unit 3211 is input to the probability estimation unit 3212. The probability estimating unit 3212 estimates a noise parameter by applying a parallel nonlinear Kalman filter and a parallel Kalman smoother to the input frame. Using the estimated noise parameters, non-speech (noise + silence) and speech (noise + clean speech) probability models are generated, and the probability when a logarithmic mel spectrum is input to each probability model is calculated.

  As shown in FIG. 19, the probability estimating unit 3212 includes a forward estimating unit 3212-1, a backward estimating unit 3212-2, a GMM (Gaussian Mixture Model) storage unit 3212-3, and a parameter storage unit 3212-4. Note that the backward estimation unit 3212-2 performs the same processing as in Reference Document 4, and a description thereof will be omitted.

The GMM storage unit 3212-3 stores a silence GMM and a clean speech GMM which are acoustic models of a silence signal and a clean speech signal prepared in advance. Hereinafter, the silent GMM and the clean voice GMM are simply referred to as GMM and the like. The method of configuring the GMM is a known technique, and a description thereof will be omitted. Each GMM contains a plurality of normal distributions (for example, 32), each of which has a mixture weight w _{j, k} , a mean μ _{S, j, k, φ} , and a variance Σ _{S, j, k, φ.} Is a parameter, j is the type of GMM (j = 0: silent GMM, j = 1: clean voice GMM), and k is the number of each normal distribution. Each parameter is input to the forward estimation unit 3212-1 and the backward estimation unit 3212-2.

  Parameter storage section 3212-4 includes an initial noise model estimation buffer and a noise model estimation buffer.

[Forward Estimation Unit 3212-1]
The processing content of the forward estimator 3212-1 is different from that of Reference 4.

In Reference 4, in the forward estimating unit, the parameters ^ N _{t, j, k, φ} and ^ Σ _{N, t, j, k, φ} of the noise model are sequentially updated from the processing start time. The parameters of the non-voice / voice GMM are updated without determining whether the sound is voice or non-voice (noise). On the other hand, in the present embodiment, since the non-speech section and the speech section are known, the parameter is updated by utilizing the information more actively. That is, the parameter of the non-voice GMM is updated using the voice feature amount of the non-voice section, and the parameter of the voice GMM is updated using the voice feature amount of the voice section.
The following is an example of processing.

First, the forward estimation unit 3212-1 uses the feature amounts g _{t-t_1-t_2, φ} ,..., G _{t-t_1, φ} from the frame times tt ₁ -t ₂ to tt ₁ corresponding to the non-voice section. , Update the parameters of the non-voice GMM (j = 0). Here, the subscripts t_1 and t_2 mean t ₁ and t ₂ , respectively.

The forward estimating unit 3212-1 stores, in the buffer for initial noise model estimation, the non-speech section feature amount g _{t− of} q frames among the non-speech section feature amounts (log mel spectrum g _{t, φ in} this example). _{t_1-t_2, φ} , ..., g _{t-t_1-t_2-1 + q-1, φ} are stored. However, q is an integer of 1 or more which does not exceed the length t ₂ of the non-speech section, eg, q = 10.

The forward estimating unit 3212-1 extracts feature amounts g _{t-t_1-t_2, φ} ,..., G _{t-t_1-t_2-1 + q-1, φ} for q frames from the initial noise model estimation buffer. Initial noise model parameters N ^init _φ , Σ ^init _{N, φ} are estimated by the following equations, and these are stored in a noise model estimation buffer.

Also, using the feature amounts g _{t-t_1-t_2 + q, φ} ,..., G _{t-t_1, φ} from the frame times tt ₁ -t ₂ + q to tt ₁ , the non-voice GMM (j = 0) Update parameters. The method of updating the parameters of the non-voice GMM and the updating formula are the same as in Reference 4.

Next, the forward estimating unit 3212-1 uses the feature amounts g _{t-t_1 + 1, φ} ,..., G _{t, φ} from the frame times tt ₁ +1 to t corresponding to the voice section to generate the voice GMM ( Update the parameter of j = 1). The parameter updated using the last frame of the non-voice section is set as the first parameter of the voice section. That is,

And Further, the parameters of the speech GMM (j = 1) are updated using the feature amounts g _{t−t — 1 + 1, φ} ,..., G _{t, φ} . The method of updating the parameters of the voice GMM and the updating formula are the same as in Reference 4.

  After the frame time t, the parameters of the voice / non-voice GMM are updated using the feature amount of the input signal, as in the conventional technology.

  The voice section detection unit 320 calculates the frame time t based on the parameters of the non-voice GMM updated using the voice features of the non-voice section and the parameters of the voice GMM updated using the voice features of the voice section. Thereafter, the parameters of the speech / non-speech GMM are updated using the feature amount of the input signal, and speech / non-speech is determined using the parameters obtained as a result. Therefore, the determination accuracy can be improved as compared with the related art in which the parameters of the non-speech / speech GMM are updated without determining whether the speech or non-speech (noise) is used.

Note that the above-described processing may be performed only when the voice section feature amount and the non-voice section feature amount are first received from the feature amount calculation section 333, or may be performed from the feature amount calculation section 333. It may be performed each time the section feature value is received. In addition, every time when the speech section feature amount and the non-speech section feature amount are received from the feature amount calculation unit 333, (a) the process of obtaining the initial noise model parameters N ^init _φ and Σ ^init _{N, φ} b) All the processes including the process of using the parameter updated using the last frame of the non-voice section as the first parameter of the voice section may be repeated, and in the second and subsequent processes, the above-described (a) and Using the parameters at the time of receiving the speech section features and the non-speech section features without performing the processing of (b), the features from the frame times tt ₁ -t ₂ to tt ₁ corresponding to the non-speech section The parameters of the non-speech GMM (j = 0) are updated using the quantities g _{t-t_1-t_2, φ} ,..., G _{t-t_1, φ,} and the frame time tt ₁ +1 to t corresponding to the speech section is updated. The parameters of the speech GMM (j = 1) may be updated using the feature amounts g _{t−t — 1, φ} ,..., G _{t, φ} .

<Effect>
With the above configuration, it is possible to more accurately know information about the surrounding acoustic environment including the target voice using the result of keyword detection for a specific utterance of the target person (user), and Signal processing becomes robust. In particular, even when the speech to be recognized and the noise have characteristics close to each other, it is possible to detect at least one of the speech section and the non-speech section with higher accuracy than before.

  In addition, one microphone 310 and the specific sound section detection unit 340 may be a part of the acoustic signal processing device. Further, in the present embodiment, the GMM is used as an acoustic model used when estimating a speech section and a non-speech section, but another acoustic model such as an HMM (Hidden Markov Model) may be used. Also in this case, similarly to the present embodiment, the voice parameter and the non-voice parameter may be obtained from the voice section feature and the non-voice section feature, respectively.

<First Modification of Third Embodiment>
The description will focus on the differences from the third embodiment.

  In the third embodiment, a logarithmic mel spectrum, a cepstrum coefficient, or the like is used as a feature value, but another feature value may be used. In the present modified example, a case where the level of the voice is more simply used for the determination will be considered.

  In the present embodiment, the average power is used as the feature amount. Therefore, the feature amount calculation unit 333 calculates the average power from the time-series sound signal corresponding to the specific sound voice section, outputs the average power as the voice section feature amount, and calculates the average power from the time-series sound signal corresponding to the non-voice section. It is calculated and output as a non-voice section feature amount.

<Voice section detection section 320>
The voice section detection unit 320 receives the time-series sound signal of the voice section detection target stored in the voice storage unit 331, and receives the voice section feature value and the non-voice section feature value from the feature value calculation unit 333. The voice section detection unit 320 obtains a voice parameter indicating a feature of the voice section from the voice section feature quantity, obtains a non-voice parameter indicating a feature of the non-voice section from the non-voice section feature quantity, and determines the voice parameter and the non-voice parameter. Then, at least one of a voice section and a non-voice section is detected from the time-series sound signal (S320), and a detection result is output.

  As shown in FIG. 20, the voice section detection unit 320 includes a voice power calculation unit 326, a voice / non-voice determination unit 327, a non-voice level storage unit 328, and a voice level storage unit 329.

  The audio power calculation unit 326 receives the time-series audio signal of the audio section detection target stored in the audio storage unit 331, calculates the average power P (n) of the time-series audio signal for each frame n, and outputs the average power P (n).

For example,
P (n)> γV, and P (n)> δN
If the condition is satisfied, a method of determining that section as a voice section may be considered. n is an index indicating the frame time, N and V are the power threshold of the non-voice section and the power threshold of the voice section stored in the non-voice level storage section 328 and the voice level storage section 329, respectively. δ is a real number of 1 or more. A voice section is determined to be a voice section if it is larger than a value (γV) that is somewhat close to the level of a signal in a voice section and is larger than a value (ΔN) that is sufficiently larger than a signal level in a non-voice section (for example, noise). In this case, if the information (V, N) of the non-speech and the speech stored in advance and the signal levels of the actual speech section and the non-speech section are different, the operation does not work properly. It is also conceivable to update each information (V, N) sequentially according to the time-series sound signal, but the value is updated in the wrong direction because it is updated without knowing which section is non-voice or voice. Risk.

  In the present embodiment, the power thresholds V and N are changed using the voice section feature (the average power of the voice section) and the non-voice section feature (the average power of the non-voice section).

  The voice / non-voice determination unit 327 extracts the power thresholds V and N from the non-voice level storage unit 328 and the voice level storage unit 329, respectively, receives the average power P (n) from the voice power calculation unit 326, and From 333, the average power Pv of the time-series sound signal corresponding to the specific sound voice section and the average power Pn of the time-series sound signal corresponding to the non-voice section are received.

The voice / non-voice determination unit 327 replaces the power thresholds V and N with power thresholds V ′ and N ′ in consideration of the average powers Pv and Pn, respectively, by the following equations.
N '= (1-α) N + αPn
V '= (1-β) V + βPv
Note that α and β represent parameters (0 <α <1, 0 <β <1) that determine the contribution rate of the detected voice / non-voice section. The voice / non-voice determination unit 327
P (n)> γV 'and P (n)>δN'
If the condition is satisfied, the section corresponding to the frame n is detected as a voice section. If the condition is not satisfied, the section corresponding to the frame n is detected as a non-voice section, and a detection result is output.

  In the case of the present embodiment, V ′ corresponds to a voice parameter indicating a feature of a voice section, and N ′ corresponds to a non-voice parameter indicating a feature of a non-voice section.

<Effect>
With the above configuration, it is possible to perform the level determination more in accordance with the actual situation, and it is possible to obtain the same effect as in the third embodiment.

<Second Modification of Third Embodiment>
The description will focus on the differences from the third embodiment.

  FIG. 13 is a functional block diagram of the acoustic signal processing device according to the third embodiment, and FIG. 14 shows a processing flow thereof.

  The acoustic signal processing device includes a voice section detection section 320, a voice section detection information storage section 330, and a preprocessing section 350.

<Pre-processing unit 350>
The preprocessing unit 350 receives the time-series sound signal as input, performs processing (voice enhancement processing) for emphasizing the sound included in the time-series sound signal (S350), and outputs the emphasized time-series sound signal. Any method may be used as the voice enhancement processing. For example, a noise suppression method described in Reference Document 2 is used.
(Reference Document 2) JP-A-2009-110011

<Effect>
With the above configuration, the same effects as in the third embodiment can be obtained. Furthermore, the detection accuracy can be improved by performing the subsequent processing (S330, S320) using the time-series sound signal subjected to the voice enhancement processing.

<Third Modification of Third Embodiment>
The description will focus on the differences from the third embodiment.

  The acoustic signal processing apparatus includes M time-series acoustic signals collected by M microphones 310-m (m = 1, 2,..., M, where M is any integer of 2 or more). And L (L is any integer of 2 or more) output values of the specific voice section detection unit 340 as inputs, and detects at least one of a voice section and a non-voice section included in the time-series sound signal. And outputs the detection result.

  FIG. 21 is a functional block diagram of an acoustic signal processing device according to a third modification, and FIG. 22 shows a processing flow thereof.

  The acoustic signal processing device includes a beam forming section 360, a voice section detection section 320, and a voice section detection information storage section 330.

<Beam forming unit 360>
The beamforming unit 360 receives the M time-series sound signals as input, and increases the directivity of the M time-series sound signals in the L directions in L time-series signals (a time-series sound signal, For example, a beamforming output signal is converted (S360) and output to the specific voice section detection section 340, the voice section detection information storage section 330, and the voice section detection section 320. For example, the signal is converted into L time-series beamforming output signals using a beamforming technique. Any method may be used as the beam forming technique. For example, the method described in Reference Document 3 is used.
(Reference 3) JP-A-2017-107141

  The specific voice section detection unit 340 detects that a specific sound has come for each of the L time-series signals, and outputs information indicating the detection time of the specific sound to the voice section detection information storage unit 330. In addition, it is assumed that a specific sound is detected in at least one of the L time-series signals, and the information indicating the detection time of the specific sound is information indicating one or more detected channels. And information indicating detection times of one or more specific sounds respectively corresponding to the detected one or more channels. The information indicating the detection time of each specific sound is as described in the third embodiment.

<Voice section detection information storage section 330>
The voice section detection information storage unit 330 receives information indicating the detection time of the specific sound and L time-series signals, and obtains voice section feature amounts and non-voice section feature amounts of the channel in which the specific sound is detected. (S330), and output. Note that feature amounts are obtained for all channels in which the specific sound is detected.

<Voice section detection section 320>
The voice section detection unit 320 receives the L time-series signals, and receives, from the feature amount calculation unit 333, the voice section feature amount and the non-voice section feature amount of the channel in which the specific sound is detected. The voice section detection unit 320 obtains one voice parameter indicating a feature of the voice section from the voice section feature amounts of all the channels in which the specific sound is detected, and obtains a non-voice section feature amount of all the channels in which the specific sound is detected. One non-speech parameter indicating a feature of the speech section is obtained, and at least one of the speech section and the non-speech section is detected from each of the L time-series signals using the speech parameter and the non-speech parameter (S320). And outputs the detection result. The detection method is as described in the third embodiment. In this modification, one (common) speech parameter and one (common) non-speech parameter are used for L time-series signals.

<Effect>
With such a configuration, the same effect as in the third embodiment can be obtained. Note that the beamforming unit 360 may be a separate device, and the acoustic signal processing device may have a configuration in which L time-series signals are input. In addition, sound is picked up by L directional microphones 310-m (m = 1, 2,..., L, where L is any integer of 2 or more), each of which has increased directivity in L directions. A configuration may be adopted in which the input L time-series sound signals are input and the beamforming unit 360 is not used.

<Fourth Modification of Third Embodiment>
The following description focuses on the differences from the third modification.

<Voice section detection section 320>
The voice section detection unit 320 receives the L time-series signals, and receives, from the feature amount calculation unit 333, the voice section feature amount and the non-voice section feature amount of the channel in which the specific sound is detected. The voice section detection unit 320 obtains one voice parameter indicating a feature of the voice section from the voice section feature value of one channel in which the specific sound is detected, and obtains a non-voice section feature value of one channel in which the specific sound is detected. One non-speech parameter indicating the feature of the non-speech section is calculated from the time-series sound signal in which the specific sound is detected using the sound parameter and the non-speech parameter obtained for each channel in which the specific sound is detected. At least one of the section and the non-voice section is detected (S320), and the detection result is output. The detection method is as described in the third embodiment.

  In this modification, L speech parameters and L non-speech parameters respectively corresponding to L time-series signals are used. Note that the voice section detection unit 320 receives the voice section feature amount and the non-voice section feature amount of the channel in which the specific sound is detected, and obtains only the non-voice parameter and the voice parameter of the channel. The non-speech parameter and the speech parameter are not determined for the channel in which the specific sound is not detected, and the non-speech parameter and the speech parameter corresponding to the channel are determined at the timing when the specific sound is detected.

<Effect>
With such a configuration, the same effect as in the third embodiment can be obtained, and detailed voice parameters and non-voice parameters can be obtained for each channel.

[Supplement]
The sound signal processing device receives a sound signal including a specific sound which is a predetermined sound as an input, and sets the sound signal obtained by removing the sound signal corresponding to the specific sound from the sound signal as a noise sound signal, the noise sound signal And an audio signal processing unit that performs audio signal processing in which the audio signal corresponding to the specific sound is associated.

  Alternatively, the acoustic signal processing device receives an acoustic signal including a specific sound that is a predetermined sound as an input, and sets an acoustic signal corresponding to the specific sound as a target acoustic signal, the target acoustic signal, and the sound signal. It can be said that the apparatus includes an audio signal processing unit that performs audio signal processing in which an audio signal other than the target audio signal is associated with the audio signal.

  Alternatively, the acoustic signal processing apparatus receives an acoustic signal including a specific sound that is a predetermined sound as an input, and sets an acoustic signal obtained by removing an acoustic signal corresponding to the specific sound from the acoustic signal as a noise acoustic signal, It can be said that the audio signal processing unit includes an audio signal processing unit that performs audio signal processing in which the audio signal corresponding to the sound is set as the target audio signal and the target audio signal is associated with the noise audio signal.

  An example of the acoustic signal processing unit is the third directional sound collection unit 52 of Modification 4 of the first embodiment. In this case, the target acoustic signal is a sound signal from the arrival direction read from the direction storage unit 13, and the noise acoustic signal is a sound signal from the arrival direction read from the noise direction storage unit 51.

  Another example of the audio signal processing unit is the voice section detection information storage unit 330 and the voice section detection unit 320 of the third embodiment. In this case, the target acoustic signal is a time-series sound signal corresponding to the specific sound voice section, and the noise sound signal is a time-series sound signal corresponding to the non-voice section.

[Program and recording medium]
When the processing in each unit of each acoustic signal processing device is realized by a computer, the processing content of the function that each unit of these devices should have is described by a program. By executing this program on a computer, the processing of each unit is realized on the computer.

  A program describing this processing content can be recorded on a computer-readable recording medium. As a computer-readable recording medium, for example, any recording medium such as a magnetic recording device, an optical disk, a magneto-optical recording medium, and a semiconductor memory may be used.

  Further, the processing of each unit may be configured by executing a predetermined program on a computer, or at least a part of these processing may be realized by hardware.

  In addition, it goes without saying that changes can be made as appropriate without departing from the spirit of the present invention.

Claims (7)

The direction of arrival of sound from signals collected by a plurality of microphones is estimated as the direction of arrival of the specific sound is closer to the direction of arrival estimated when a specific sound that is a predetermined sound is detected. A direction estimating unit for estimating so as to be easily performed,
A first directional sound collection unit that performs sound collection so that the sound from the arrival direction estimated by the direction estimation unit is emphasized,
When the direction estimating unit estimates a plurality of directions of arrival when the specific sound is detected,
Utilizing the direction of arrival estimated in the past predetermined section, to select the direction of arrival of the specific sound from the plurality of directions of arrival,
Sound signal processing device. The acoustic signal processing device according to claim 1,
The selection is performed such that, in the predetermined section in the past, the arrival direction of the sound source that continuously outputs sound is not selected as the arrival direction of the specific sound.
Sound signal processing device. The acoustic signal processing device according to claim 1,
The selection is performed such that, in the past predetermined section, the arrival direction having a high frequency of the estimated arrival direction is not selected as the arrival direction of the specific sound.
Sound signal processing device. The acoustic signal processing device according to claim 1,
The selection is performed such that, in the predetermined section in the past, the direction of arrival with the lowest frequency of estimated directions of arrival is selected as the direction of arrival of the specific sound.
Sound signal processing device. The acoustic signal processing device according to any one of claims 1 to 4,
The first directional sound collection unit performs sound collection so that a sound from an arrival direction closer to the arrival direction estimated to be the arrival direction of the specific sound is more emphasized.
Sound signal processing device. The direction estimating unit determines the direction of arrival of the sound from the signals collected by the plurality of microphones, the closer the arrival direction of the specific sound is to the estimated arrival direction when a specific sound that is a predetermined sound is detected. A direction estimating step of estimating so as to be easily estimated as a direction;
The first directional sound collection unit includes a first directional sound collection step of performing sound collection so that sound from the arrival direction estimated by the direction estimation unit is emphasized,
In the direction estimating step, when a plurality of arrival directions are estimated when the specific sound is detected,
Utilizing the direction of arrival estimated in the past predetermined section, to select the direction of arrival of the specific sound from the plurality of directions of arrival,
Sound signal processing method.   A computer-readable program for causing a computer to function as each unit of the acoustic signal processing device according to claim 1.

Images