JP2009109868A - Sound source localization apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a sound source localization apparatus advantageous in hardware mounting. <P>SOLUTION: The sound source localization apparatus 1 includes: two microphones 2 and 3, each of which has front directivity, and which are horizontally arranged with an interval, with one facing forward, while the other facing backward; a time difference detecting section 8 for detecting time difference information of sound collected by the microphones 2 and 3, by a pulse neuron model; a sound pressure difference detecting section 9 for detecting sound pressure difference information of the sound collected by the microphones 2 and 3, by the pulse neuron model; a horizontal direction detecting section 10 for detecting sound source direction information in a horizontal direction based on the time difference information of the sound; and a back and forth direction detecting section 11 for detecting sound source direction information in a back and forth direction based on the sound pressure information of the sound. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a sound source localization apparatus that identifies a sound source direction, and more particularly, to a sound source localization apparatus that uses a pulse neuron model that is a neuron model that inputs and outputs a pulse signal (also simply referred to as “pulse”).

  Techniques using a pulse neuron model (hereinafter also referred to as “PN model” or “neuron”) for sound source localization are disclosed in Patent Document 1 and Non-Patent Documents 1 to 3 below. Patent Document 1 discloses a time difference detector for sound source localization, and Non-Patent Document 1 discloses a sound source direction perception model that extracts a time difference and a sound pressure difference of sounds coming into both ears using a PN model. Non-Patent Document 2 discloses a competitive learning neural network using a PN model for an auditory information processing system, and Non-Patent Document 3 discloses a method of mounting a PN model on hardware. ing.

FIG. 1 shows a schematic diagram of the PN model. Since the same PN model is also shown in Non-Patent Documents 1 to 3, it will not be described in detail. However, in the PN model, from the nth input channel at time t (t is a discrete value, here, dt = 1). When the pulse i _n (t) = 1 is input to the n-th synapse portion, the local membrane potential p _n (t) of the n-th synapse portion is increased by the connection weight (also simply referred to as “weight”) w _n . After that, it decays to a static potential with a time constant τ. The internal potential I (t) of the PN model at time t is expressed as the sum of the local membrane potentials p _n (t) at that time. The PN model ignites (that is, generates an output pulse “1”) when the internal potential I (t) becomes equal to or greater than the threshold θ. However, since there is a refractory period RP related to firing in nerve cells, even in the PN model, even if the internal potential exceeds a threshold value during a certain firing to RP, firing does not occur.

  The PN model can be implemented by hardware using a digital circuit. FIG. 2 shows a configuration example of a PN model digital circuit. A similar configuration example is also described in Non-Patent Document 3 and will not be described in detail. However, in this configuration example, an addition process is usually performed, and a value obtained by bit-shifting the register value is subtracted after several addition processes ( That is, attenuation is approximately realized by bit shift and complement expression), and a multiplier is not used for the attenuation processing mechanism, so that it is suitable for realization by a digital circuit.

  By the way, as shown in FIG. 3, two microphones 101 and 102 corresponding to both ears are arranged on the left and right in the same direction, and the sound collected by the microphone 101 and the sound collected by the microphone 102 are used. Thus, when trying to identify the direction of the sound source, the left and right direction can be identified by the time difference of the sound resulting from the difference in the distance from the sound source to the microphones 101 and 102, but the sound source connects the sound collection parts of the microphones 101 and 102. It was impossible to discriminate whether it was in the front-rear direction across the line (dashed line in the figure). If it says in FIG. 3, it is because the same time difference will arise even if a sound source exists in the position of P symmetrical about the dashed-dotted line in a figure, and the position of P '.

For this reason, in the sound source localization system described in Non-Patent Document 4 below, as described in section 3.4 of the same document, the right microphone is disposed forward and the left microphone is disposed rearward ( Since the sound from the back is more muffled than the sound from the front, calculate the centroid of the frequency spectrum of the sound coming from the left and right microphones, and the right centroid is more than the left centroid. If it is larger, it is determined to be forward, and if the left centroid is larger than the right centroid, it is determined to be rearward.
JP 2007-164027 A Kuroyanagi Shu, Akira Iwata, "Sound source direction perception by pulse transmission type auditory neural circuit model-Extraction of time difference and sound pressure difference-" Technical report of IEICE, IEICE, March 1993, NC92- 149, p. 163-170 Kuroyanagi Shu, Akira Iwata, “Competitive Learning Neural Network Using Pulsed Neuron Model for Auditory Information Processing System”, IEICE Transactions (D-II), July 2004, Vol. J87-D-II No. 7, p. 1496-1504 Noriyoshi Futaki, Susumu Kuroyanagi, Akira Iwata, “Implementation Method of Pulsed Neuron Model for FPGA”, IEICE NC Research Technical Report, The Institute of Electronics, Information and Communication Engineers, March 2002, NC2001-211, p . 121-128 Schauer, Gross, “Model and Application of a Binaural 360 ° Sound Localization System”, Proceedings of the International Joint Conference on Neural Networks), IEEE Computer Society, 2001, Vol. 2, p. 1132-1137

  However, the sound source localization system of Non-Patent Document 4 is disadvantageous in terms of hardware implementation because it performs FFT (Fast Fourier Transform) to calculate the center of gravity of the frequency spectrum and requires an FFT calculator.

  The present invention solves the above-described problems, and an object thereof is to provide a sound source localization apparatus that is advantageous in terms of hardware implementation.

  Each of the sound source localization apparatuses of the present invention has a front directivity, and is disposed with a space between left and right, one of which is directed forward and the other of which is disposed rearward, Time difference detection means for detecting the time difference information of the sound collected by the microphone using a pulse neuron model, and sound pressure difference detection means for detecting the sound pressure difference information of the sound collected by each microphone using the pulse neuron model; Based on the time difference information of the sound detected by the time difference detection means, the left / right direction detection means for detecting the direction information of the sound source in the left / right direction by a pulse neuron model, and the sound pressure difference information of the sound detected by the sound pressure difference detection means And a front-rear direction detecting means for detecting the direction information of the sound source in the front-rear direction by using a pulse neuron model. To.

  In addition, based on the direction information of the sound source in the left-right direction detected by the left-right direction detection unit and the direction information of the sound source in the front-rear direction detected by the front-rear direction detection unit, the direction information of the sound source in a plurality of surrounding directions It is preferable to include a surrounding direction detecting means for detecting the signal using a pulse neuron model.

  In the sound source localization apparatus of the present invention, the time difference detecting means detects sound time difference information using a pulse neuron model, the sound pressure difference detecting means detects sound pressure difference information using a pulse neuron model, and the left-right direction detecting means detects left and right information using a pulse neuron model. Since the direction information of the sound source in the direction is detected and the direction detection means detects the direction information of the sound source in the front and back direction by the pulse neuron model, and the pulse neuron model can be realized by a simple digital circuit, hardware implementation This is advantageous.

  Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

  As shown in FIG. 4, the sound source localization apparatus 1 includes left and right microphones 2 and 3 and a main body 4 to which the microphones 2 and 3 are connected. The main body 4 is connected to a display device 5 that displays the localization result.

  The microphones 2 and 3 have forward directivity (that is, unidirectional with good forward sensitivity). The microphones 2 and 3 are arranged so that the sound collecting portions 2a and 3a of the microphones 2 and 3 are arranged in the left and right direction with an interval left and right, and the left microphone 2 faces the front and the right microphone 3 Are arranged facing backwards. The microphones 2 and 3 convert the sound collected by the sound collection units 2a and 3a into electric signals. The greater the volume, that is, the greater the sound pressure, the higher the voltage value of the converted electrical signal. Note that the sound collection units 2a and 3a do not necessarily have to be arranged on a straight line in the left-right direction. In such a case, an appropriate correction may be made on the obtained data.

  As shown in FIG. 5, the main unit 4 includes a left input signal processing unit 6 connected to the left microphone 2, a right input signal processing unit 7 connected to the right microphone 3, and an input signal processing unit. 6 and 7, a time difference detection unit (corresponding to a time difference detection unit) 8, and a sound pressure difference detection unit (corresponding to a sound pressure difference detection unit) 9 connected to both of the input signal processing units 6 and 7. And a left-right direction detection unit (corresponding to the left-right direction detection means) 10 connected to the time difference detection unit 8, and a front-rear direction detection unit (corresponding to the front-rear direction detection means) 11 connected to the sound pressure difference detection unit 9. , An eight-direction detection unit (corresponding to a surrounding direction detection means) 12 connected to both the left-right direction detection unit 10 and the front-rear direction detection unit 11.

  The input signal processing units 6 and 7 convert each of the left and right input signals into a pulse train having a pulse frequency corresponding to the signal intensity, that is, the sound pressure, for each frequency band. The time difference detection unit 8 detects time difference information of left and right sounds (sounds from the left and right microphones 2 and 3) from the left and right pulse trains output from the input signal processing units 6 and 7. The sound pressure difference detection unit 9 detects sound pressure difference information of left and right sounds from the left and right pulse trains. When the sound source is on the left side, the left / right direction detection unit 10 receives sound from the left microphone 2 faster than the right microphone 3 and when it is on the right side, the sound enters from the right microphone 3 faster than the left microphone 2. Using this, the direction information of the sound source in the left-right direction is detected from the time difference information detected by the time difference detection unit 8. The front-rear direction detection unit 11 detects the difference in sound pressure by using the fact that the sound pressure from the microphone not facing the sound source is smaller than the sound entering from the microphone facing the sound source. The sound source direction information in the front-rear direction is detected from the sound pressure difference information detected by the unit 9. The eight-direction detection unit 12 is based on the information output from the left-right direction detection unit 10 and the front-rear direction detection unit 11 and the direction information of the sound source in the surrounding eight directions (whether the sound source is in one of the eight directions). Information) is output.

  The display unit 5 displays the direction of the sound source based on the direction information output from the 8-direction detection unit 12.

  As shown in FIG. 6, the input signal processing units 6 and 7 include AD conversion units 14L and 14R, frequency resolving units 15L and 15R corresponding to human cochleas, and nonlinear conversion units corresponding to hair cells. 16L and 16R, and pulse converters 17L and 17R corresponding to the cochlear nerve. The AD converters 14L and 14R AD convert the signals input from the microphones 2 and 3. The frequency resolving units 15L and 15R are configured by a band pass filter (BPF) group, and decompose the AD-converted signals into signals of a plurality of (N) frequency bands (frequency channels) on a logarithmic scale for a predetermined frequency range. . The non-linear transformation units 16L and 16R take out only the positive components by performing non-linear transformation on the signals of the respective frequency bands input from the frequency decomposition units 15L and 15R, and also by a low-pass filter (LPF). Perform envelope detection. The pulse converters 17L and 17R convert the signals in the respective frequency bands input from the nonlinear converters 16L and 16R into pulse trains each having a pulse frequency proportional to the signal intensity. Through these processes, the input signal processing units 6 and 7 convert each of the left and right input signals into a pulse train having a pulse frequency corresponding to the signal intensity for each frequency band.

  When the input signal units 6 and 7 are implemented as hardware, the AD conversion units 14L and 14R are AD conversion circuits, and the frequency resolution units 15L and 15R, the nonlinear conversion units 16L and 16R, and the pulse conversion units 17L and 17R are digital, respectively. It can be configured with a circuit.

  Detection of time difference information in the time difference detection unit 8, detection of sound pressure difference information in the sound pressure difference detection unit 9, detection of direction information in the left-right direction detection unit 10, detection of direction information in the front-rear direction detection unit 11, and 8-direction detection unit The detection of the direction information in 12 is performed by a pulse neural network constituted by a plurality of PN models. The pulse neural network is realized by mounting a plurality of PN models that can operate independently and asynchronously in parallel as an electronic circuit, and can perform high-speed processing.

  The time difference detection unit 8 is composed of a time difference detection model composed of a PN model as shown in FIG. 7 and a sequence of time delay elements 19 (see FIG. 8) for inputting to the time difference detection model while shifting the pulse train. Yes. Since the time difference detection model is the same as that described in Non-Patent Document 1, etc., it will not be described in detail. However, as shown in FIG. 8, a plurality of time difference detection neurons (hereinafter also referred to as “MSO neurons”) 20 are provided. , An odd number) of MSO neuron rows arranged for each frequency channel. Each MSO neuron 20 includes a left input terminal 21 to which a left pulse signal is input, a right input terminal 22 to which a right pulse signal is input, and an output terminal 23. When the pulse signal is input almost simultaneously from the left and right by setting the weight to the common fixed value and the threshold value to be twice the weight or the value obtained by adding the reference value of the internal potential to the double weight, etc. 23 is configured to output a pulse signal. Note that “substantially simultaneous” includes, of course, simultaneous.

  Then, the time difference detector 8 shifts the left pulse train to the right and the right pulse train to the left and shifts the left and right pulse trains to the corresponding frequency channel for each clock (unit time) by the time delay element 19. To the MSO neuron array. That is, the left pulse signal is sequentially input to each MSO neuron 20 while being shifted from one end (left end in FIG. 8) to the other end (right end) of the MSO neuron train for each unit time, and the right pulse signal is input to the MSO neuron train. Are sequentially input to each MSO neuron 20 while being shifted every other unit time from the other end (same right end) to one end (same left end).

For example, if the number of MSO neurons 20 in each MSO neuron array is 2J + 1 and each MSO neuron 20 is assigned a number from −J to J, at time t, each MSO neuron 20 has an internal potential I ^MSO according to the following [Equation 1]. _ji (t) is calculated, and y _ji (t) = 1 is output when the internal potential exceeds a predetermined threshold, and y _ji (t) = 0 is output when it does not exceed the predetermined threshold. Note that j is the number of the MSO neuron 20, and i is the frequency channel number (i = 1 to N). In the following [Equation 1], p ^left _ji (t) is a local membrane potential for the left input signal, p ^right _ji (t) is a local membrane potential for the right input signal, and w is a common connection in all neurons 20. The weight, τ, is the decay time constant.

Thus, in the time difference detection model, the neuron 20 near the center in the MSO neuron array fires when a pulse signal enters from the left and right substantially simultaneously, and when the pulse signal enters earlier from the left than the right, the MSO The right neuron 20 in the neuron array fires, and when the pulse signal comes in from the right earlier than the left, the left neuron 20 in the MSO neuron array fires. The firing pattern is output as sound time difference information.

As described above, when each MSO neuron 20 in each MSO neuron train is _assigned a number from −J to J, the output pulse train (y _−Ji (t)) is output from the MSO neuron train corresponding to the frequency channel i at time t. ,..., Y _0i (t),..., Y _Ji (t)) are output, and the following vector y _MSO (t) is output as time difference information as a whole from the time difference detection model.

y _MSO (t) = (y _-J1 (t), ..., y ₀₁ (t), ..., y _J1 (t),
y _-J2 (t), ..., y ₀₂ (t), ..., y _J2 (t),
…,
y _-JN (t), ..., y _0N (t), ..., y _JN (t))
The time difference detection unit 8 can be configured with a digital circuit, for example, as shown in FIG. FIG. 9A is a diagram for explaining the operation of the first half of one clock, and FIG. 9B is a diagram for explaining the operation of the second half. This example is also described in Chapter 5 of Non-Patent Document 3 and will not be described in detail. However, each MSO neuron 20 of the time difference detection unit 8 includes AND circuits 24L and 24R, adders 25 and 26, and a register 27. And a comparator 28 and an attenuation generator 29. As shown in FIG. 9A, the attenuation generation unit 29 generates an attenuation amount of the internal potential by inputting a bit shift and a complement expression to the internal potential (internal potential), and inputs it to the adder 26. The adder 26 adds the internal potential and its attenuation, and updates the internal potential in the register 27. Then, as shown in FIG. 9B, the AND circuits 24L and 24R input the coupling weight to the adder 25 only when the input signal is “1”, and the adder 25 inputs the input coupling weight and the register 27. Is added to the internal potential held in the register 27 to update the internal potential in the register 27. The comparator 28 compares the threshold value held by itself with the internal potential held by the register 27, and outputs a signal “1” if the internal potential is equal to or greater than the threshold value. Signal “0” is output. Note that the implementation of the refractory period can be realized by providing a counter that counts the refractory period and not firing during the period from ignition to resetting the counter along with the ignition.

  The sound pressure difference detection unit 9 is composed of a sound pressure difference detection model composed of a PN model as shown in FIG. The sound pressure difference detection model is the same as that described in Non-Patent Document 1 and the like and will not be described in detail. However, as shown in FIG. 11, a sound pressure difference detection neuron (hereinafter also referred to as “LSO neuron”) 40 is used. A plurality (however, odd number) of LSO neuron rows arranged are provided for each frequency channel. Each LSO neuron 40 includes a left input terminal 41 to which a left pulse signal is input, a right input terminal 42 to which a right pulse signal is input, and an output terminal 43.

As shown in FIG. 11, at time t, the left input terminal 41 and the right input terminal 42 of each LSO neuron 40 in each LSO neuron string are respectively connected to the corresponding frequency channel i ( The left pulse signal x ^left _i (t) of i = 1 to N) and the right pulse signal x ^right _i (t) are input. Note that k is a number assigned to each neuron 40 in each LSO neuron array, and −K ≦ k ≦ K.

Then, each LSO neuron 40 calculates an internal potential I ^LSO _ki (t) according to the following [Equation 2], and outputs y _ki (t) = 1 when the internal potential exceeds a predetermined threshold value. If it is below the threshold, y _ki (t) = 0 is output. The threshold value is a value common to each LSO neuron 40. In [Equation 2] below, p ^left _ki (t) is the local membrane potential for the left input signal, p ^right _ki (t) is the local membrane potential for the right input signal, and w ^left _ki is for the left input signal. The coupling weight, w ^right _ki, is the coupling weight for the right input signal. Further, τ is an attenuation time constant, and b, α, and β are constants.

As shown in the above [Equation 2], in the sound pressure difference detection model, the coupling weight with respect to the left and right input signals gradually changes, and when the left and right sound pressures are substantially equal, the central portion (from number -b to b) (However, the neuron number 0 does not fire.) If only the neuron 40 is fired and the left sound pressure is greater than the right sound pressure, the right sound pressure is from the center to the left neuron 40. If the sound pressure is larger than the sound pressure, the neuron 40 is fired from the center to the right neuron 40, and as the sound pressure difference between the left and right is larger, the neuron far from the center is fired. It should be noted that by appropriately determining the connection weight, the central LSO neuron 40 in each frequency channel may not fire at all as described above, and some neurons 40 on both sides of the central LSO neuron 40 You may make it always ignite.

As described above, when each LSO neuron 40 in each LSO neuron train is _assigned a number from −K to K, the output pulse train (y _−Ki (t)) is output from the LSO neuron train corresponding to the frequency channel i at time t. ,..., Y _0i (t),..., Y _Ki (t)) are output, and the following vector y _LSO (t) is output as sound pressure difference information as a whole from the sound pressure difference detection model.

y _LSO (t) = (y _−K1 (t),..., y ₀₁ (t),..., y _K1 (t),
y _-K2 (t), ..., y ₀₂ (t), ..., y _K2 (t),
…,
y _-KN (t), ..., y _0N (t), ..., y _KN (t))
Each LSO neuron 40 can be realized by a digital circuit by appropriately changing the connection weight, threshold value, and the like in the same configuration as the MSO neuron 20 of FIG.

  The left-right direction detection unit 10, the front-rear direction detection unit 11, and the 8-direction detection unit 12 are all configured by a competitive learning neural network (hereinafter referred to as “CONP”) described in Non-Patent Document 2. Yes. The CONP is a pulse neural network configured so that only one competitive learning neuron fires each time by adjusting the threshold value of each competitive learning neuron, and is intended for quantization of an input vector. The configuration of CONP is shown in FIG. The CONP is composed of a competitive learning neuron group 50 and a control neuron group 60. The competitive learning neuron group 50 is composed of a plurality of competitive learning neurons (hereinafter also referred to as “CL neurons”) 51, and the control neuron group is CL. Multiple firing detection neurons (hereinafter, referred to as “NFD neurons”) 61 and multiple firing detection neurons (hereinafter referred to as “NFD neurons”) 61 that fire when no neurons 51 are fired. It is also referred to as “MFD neuron”) 62.

  The NFD neuron 61 and the MFD neuron 62 change the threshold value of each CL neuron 51 uniformly according to their firing status (actually, the internal potential of each CL neuron 51 is changed uniformly), so that the CL neuron This is a PN model for maintaining a situation where only one CL neuron 51 is fired in the group 50. Each of the NFD neuron 61 and the MFD neuron 62 includes an input terminal corresponding to the number of CL neurons 51 in the CL neuron group 50 and an output terminal. The pulse signal output from each CL neuron 51 is received by each input terminal. The NFD neuron 61 outputs “1” from the output terminal only when the signals from all the CL neurons 51 are “0”, and the MFD neuron 62 receives the signal “1” from the plurality of CL neurons 51. Only in this case, “1” is output from the output terminal.

Each CL neuron 51, as shown in FIG. 13, the input pulse _{x 1 (t), x 2} (t), ..., x i (t), ..., the input terminal x _n (t) are respectively input 551 , 552, ..., 55i, ..., 55n, and input terminals 56 and 57 to which pulse signals y _nfd (t) and y _mfd (t) output from the NFD neuron 61 and the MFD neuron 62 are input, respectively, and output terminals 58. Each input terminal 55i (i = 1 to n) branches into two, one connected to the synapse unit 53i having a variable coupling weight w _hi and the other connected to the synapse unit 54i having a fixed coupling weight “1”. Has been. Note that h is a number assigned to each CL neuron 51 in the CL neuron group 50, and h = 1 to M.

The operation of CONP will be described based on FIGS. 14-1 and 14-2. Each CL neuron 51 in the CL neuron group 50 has an input vector x (t) = (x ₁ (t), x ₂ (t) _,. t),..., x _n (t)) (t: time) are input (S101). Then, the NFD neuron 61 and the MFD neuron 62 respectively output the output value y _{nfd at the} time t based on the output y _h (t−1) from each CL neuron 51 at the held time (t−1). (T), y _mfd (t) are calculated and output to each CL neuron 51 (S102, S103). Note that, in the NFD neuron 61 and the MFD neuron 62, output values y _nfd (t) and y _mfd (t) using the output y _h (t−1) from each CL neuron 51 at time (t−1), respectively. _{May be} calculated and held, and y _nfd (t) and y _mfd (t) may be output to each CL neuron 51 at time t.

Next, each CL neuron 51 calculates an internal potential I _h (t) (h = 1 to M) (S104) (see [Expression 6] below), and the internal potential I _h (t) is a threshold TH. Is exceeded and y _h (t) = 1 is output if the refractory period has elapsed since the previous ignition, and y _h (t) = 0 is output otherwise (S105) .

At the time of learning, for the CL neuron 51 that has output “1”, the connection weight w _i is updated using the local membrane potential pcw _i in the synapse 54 _i (S 106), and the CL neurons 51 around the CL neuron 51 are also updated. Similarly, the connection weight is updated (S107). As a method of determining the peripheral CL neurons 51 (that is, the connection weight update range), for example, all CL neurons 51 are initially set as the update range, the range is linearly reduced, and finally the connection weights of the winner neurons are determined. There is a way to gradually reduce, such as updating only. Then, the norm of the connection weight (norm of the reference vector) is normalized to 1 for the CL neuron 51 whose connection weight has been updated (S108). That is, in this CONP, a self-organizing map (SOM) algorithm is realized by learning not only the winner neuron but also the neighboring neurons.

  On the other hand, when it is not at the time of learning (at the time of recognition), the connection weight is not updated. Then, the coefficient α for updating the connection weight is updated by multiplying by a constant γ (0 ≦ γ) (S109), and the processing of steps S101 to S108 is performed for the next input vector.

Here, a method of calculating the internal potential I _h (t) in CONP will be described. First, as an argument, a function F having four parameters of time t, decay time constant τ, coupling weight w, and input signal x (t) at time t is introduced and defined as [Equation 3] below. Note that Δt = 1 / Fs (Fs: sampling frequency).

Then, the internal potential I (t) of the PN model at time t can be described as the following [Equation 4] as the sum of the local membrane potentials p _i (t) (i = 1 to n). τ is the decay time constant of p _i (t).

If the refractory period of the PN model is RP, the elapsed time from the previous firing at time t is ET (t), and ET (0)> RP, the output value y (t) of the PN model is calculated by the following algorithm: The

if I (t) ≧ TH and ET (t)> RP
then y (t) = 1, ET (t) = 0
else y (t) = 0, ET (t) = ET (t−Δt) + Δt
Parameters τ, w ₁ , w ₂ ,..., W _n , TH are variable values depending on each PN model, and the operation of each PN model is determined by this combination.

Here, the outputs of the NFD neuron 61 and the MFD neuron 62 at time t are y _nfd (t) and y _mfd (t), respectively, and the connection weights of the CL neurons 51 to the NFD neuron 61 and the MFD neuron 62 are w _fd and − _Assuming w _fd (where w _fd > 0), the internal potential I _h (t) of the CL neuron 51 of number h at time t can be described using the function F as shown in [Formula 5] below. In CONP, p _nfd (t) and p _mfd (t) are treated as dynamic variations of the threshold (however, instead of changing the threshold TH, the internal potential I _h (t) to be compared with the threshold TH is changed to p _nfd ( t) and p _mfd (t) to maintain the state in which only one CL neuron 51 is fired. For this reason, the decay time constant τ _fd is sufficiently large with respect to the time constant τ.

By the way, when the total amount of the internal potential generated by the input pulse train fluctuates greatly, a change in the threshold value occurs to absorb this fluctuation amount, and the change in the threshold value may not follow the change in the direction of the input vector. Therefore, in the CONP, the total sum of local membrane potentials pcw _i (t) in the synapse portions i (i = 1 to n) with the coupling weight fixed to 1 is set to a constant ratio β _pcw (where 0 ≦ β _By subtracting in advance with _pcw ≦ 1), the change in the internal potential with respect to the norm fluctuation of the input signal is suppressed. As a result, I _h (t) in the above [Equation 5] is corrected as shown in the following [Equation 6], and each CL neuron 51 calculates the internal potential I _h (t) according to [Equation 6]. Note that pcw _i (t) = F (t, τ, 1, x _i (t)).

CONP can also be implemented by hardware using a simple digital circuit, and an example is shown in FIG. In this example, CONP includes M CL neuron units 51H corresponding to CL neurons 51, one NFD neuron unit 61H corresponding to NFD neuron 61, and one MFD neuron unit corresponding to MFD neuron 62, respectively. 62H, and further includes one threshold change amount generators 63 and 64 and one internal potential suppression amount generator 65.

Each CL neuron unit 51H includes n input terminals corresponding to the input terminals 511,..., 51n of the CL neuron 51 and n input pulses x ₁ (t), x ₂ ( t),..., x _n (t), each of which is multiplied by n AND circuits 71, an adder 72 for adding the output from each AND circuit 71 to the internal potential, and bit shift and complement expression. An attenuation generation unit 73 that attenuates the internal potential and outputs the same to the adder 72 and a comparator 74 that compares the internal potential output from the adder 72 with a threshold value are provided. If the refractory period has passed since the last firing, y _h (t) = 1 is output, and y _h (t) = 0 is output otherwise. As will be described later, the comparator 74 _receives p _nfd (t) and p _mfd (t) as dynamic threshold change amounts and S _pcw (t) as a suppression amount of the internal potential. In 74, after adjusting the internal potential with these values as in [Formula 6], it is compared with the threshold value.

The NFD neuron unit 61H includes M input terminals respectively connected to the output terminals of the M CL neuron units 51H, and M input pulses y ₁ (t), y ₂ ( t),..., y _M (t) are multiplied by M AND circuits 76, an adder 77 for adding the output from each AND circuit 76 to the internal potential, and bit shift and complement expression. The attenuation generation unit 78 that attenuates the internal potential and outputs it to the adder 77 is compared with the internal potential output from the adder 77 and the threshold value, and the internal potential exceeds the threshold value and has not been detected since the previous ignition. The comparator 79 outputs 1 when the deadline has passed and 0 otherwise, and is configured to ignite when all the M input pulses are 0.

  The MFD neuron unit 62H has the same configuration as that of the NFD neuron unit 61H, but is configured to fire when a plurality of M input pulses are 1 by changing weights and thresholds.

The threshold value change amount generation units 63 and 64 are portions that generate local membrane potentials p _nfd (t) and p _mfd (t) for the output from the NFD neuron unit _61H in each CL neuron unit 51H. Although the neuron unit 51H is provided in common, the weight and the decay time constant are not changed by the CL neuron unit 51H. Therefore, the neuron unit 51H is extracted from each CL neuron unit 51H to be one unit as a whole.

The threshold value change amount generation unit 63 includes an AND circuit 81 that multiplies the output from the NFD neuron unit 61H by a weight w _fd , an adder 82 that adds the output from the AND circuit 81 to the local membrane potential, bit shift and complement An attenuation generation unit 83 that attenuates the local membrane potential according to the expression and outputs the attenuated local membrane potential to the adder 82. The CL membrane unit 51H _{receives the} local membrane potential p _nfd (t) from the adder 82 as the dynamic change amount of the threshold. Output to the comparator 74.

The threshold variation generator 64 has the same configuration as the threshold variation generator 63, generates a local membrane potential p _mfd (t) for the output from the MFD neuron 62H in each CL neuron 51H, Is output to the comparator 74 of each CL neuron unit 51H.

The internal potential suppression amount generation unit 65 is a portion that generates the suppression amount S _pcw (t) of the change in internal potential with respect to the norm fluctuation of the input signal described above. Originally, each CL neuron unit 51H has a fixed weight of 1 The total sum of local membrane potentials pcw _i (t) in the synapse part 54i is generated by multiplying by a constant ratio β _pcw , but the weight and the decay time constant are not changed by the CL neuron part 51H, so each CL neuron part 51H It is taken out from the whole to make one. The internal potential suppression amount generation unit 65 includes an AND circuit 86 that multiplies each of n input pulses by a fixed weight, an adder 87 that adds the output from the AND circuit 86 to the internal potential, bit shift, and complement expression. And an attenuation generation unit 88 that attenuates the internal potential and outputs it to the adder 87, and outputs the internal potential as a suppression amount S _pcw (t) from the adder 87 to the comparator 74 of each CL neuron unit 51H. . Note that multiplication by the ratio β _pcw is realized by setting the weight in each AND circuit 86 to the ratio β _pcw .

  Note that the CONNP hardware configuration example shown in FIG. 15 does not include a learning mechanism (a weight update mechanism in each CL neuron unit 51H). This is because learning may be performed by software simulation as will be described later, weights are determined, and the weights are set on the hardware. Of course, the learning mechanism can be implemented in hardware, but in order to facilitate the circuit configuration and reduce the circuit size, it is better to perform learning on software and determine the weight.

As shown in FIG. 16, the left-right direction detection unit 10 includes a CONP having a plurality (16 in this case) of CL neurons 51. The 16 CL neurons 51 are arranged in a line from number 1 to number 16, and the closer the number, the closer the distance. Each CL neuron 51 receives time difference information (here, vector y _MSO (t)) output from the time difference detection unit 8. As a result of learning, the left-right direction detection unit 10 can quantize the input vector y _MSO (t) while maintaining the similarity relationship. That is, at the time of recognition, the left and right direction detection unit 10 fires CL neurons 51 that are close to each other when vectors having a high degree of similarity are input, and CL neurons 51 that are far from each other when vectors having a low degree of similarity are input. Will be ignited. Thereby, at the time of recognition, the direction information in the left-right direction of the sound source is indicated from the left-right direction detection unit 10 according to which CL neuron 51 fires.

As shown in FIG. 16, the front-rear direction detection unit 11 is also composed of a CONP having 16 CL neurons 51 arranged in a line, like the left-right direction detection unit 10, and each CL neuron 51 includes a sound pressure difference. The sound pressure difference information (here, vector y _LSO (t)) output from the detector 9 is input. As in the case of the left / right direction detection unit 10, the front / rear direction detection unit 11 can quantize the input vector y _LSO (t) while maintaining the similarity as a result of learning. Is indicated by which CL neuron 51 fires.

  As shown in FIG. 16, the eight-direction detection unit 12 includes a CONP having a plurality of CL neurons 51 (here, eight in accordance with the direction to be identified). The eight CL neurons 51 are arranged in a line from number 1 to number 8, and the closer the number, the closer the distance. Each CL neuron 51 is input with a vector whose elements are 32 pulses including 16 pulses output from the left-right direction detection unit 10 and 16 pulses output from the front-rear direction detection unit 11. The As a result of learning, the 8-direction detection unit 12 can quantize the input vector while maintaining the similarity, and at the time of recognition, the 8-direction detection unit 12 receives the direction information of the sound source in the surrounding 8 directions from which CL. This is indicated by whether the neuron 51 fires.

  Since the CONP can be realized with a simple digital circuit as described above, the left-right direction detection unit 10, the front-rear direction detection unit 11, and the 8-direction detection unit 12 can also be realized with a simple digital circuit.

  Below, the result of having performed the simulation by realizing the main body 4 of the sound source localization apparatus 1 on a computer with software will be described. The experiment was performed in an anechoic chamber with the sound source S as a speaker arranged as shown in FIG. It should be noted that the distance between the microphones 2 and 3 is 30 cm, the distance from the midpoint C of the microphones 2 and 3 to the sound source S when placed as shown in FIG. 17A is 100 cm, and the direction of the arrow in FIG. It was the front for 1. Then, the white noise generated by the computer is emitted from the sound source S at each position where the position of the sound source S with respect to the microphones 2 and 3 is changed by 45 ° as shown in FIGS. The sound collected from 3 was input to the main unit 4. Experimental parameters are shown in Tables 1-5.

Further, the output results of the time difference detection unit 8 when the sound source is placed at each position of 0 ° to 315 ° are shown in FIGS. 18-1 to 18-8, and the output results of the sound pressure difference detection unit 9 are shown in FIGS. It is shown in FIG. In these output results, the shade of each square represents the firing frequency of neurons.

  The learning in the left-right direction detection unit 10 and the front-rear direction detection unit 11 performs unsupervised learning based on the SOM as shown in FIG. 14-2, and the learning in the 8-direction detection unit 12 is general supervised learning. Learning based on LVQ was performed. That is, in the 8-direction detection unit 12, the CL neurons 51 of numbers 1 to 8 are determined as the CL neurons 51 indicating directions of 0 ° to 315 °, respectively, and if the CL neurons 51 fired when data is input are correct ( For example, if the CL neuron 51 fired when data of 0 ° is input is number 1), the reference vector of the CL neuron 51 is brought close to the input vector, and if it is incorrect, the reference vector of the CL neuron 51 is input. We learned to keep away from the vector. Learning was performed only by the winner neuron.

  After learning, white noise is emitted from the sound source S at each position in FIGS. 17A to 17H, and the left / right direction detection unit 10 and the front / rear direction detection units 11 and 8 when the sound source localization apparatus 1 performs recognition are used. The output results from the detection unit 12 are shown in Tables 6, 7, and 8, respectively.

Tables 6 to 8 show the firing rate of each CL neuron 51 when an input signal (white noise) is continuously emitted from each direction for a certain period of time. According to Tables 6 and 7, although it seems that identification in the entire circumferential direction is possible only with the output of either the left-right direction detection unit 10 or the front-rear direction detection unit 11, this is an artificially generated white noise Is used as an input sound, and in an actual sound, it is difficult to identify only one of the outputs.

  In Table 8, Nos. 1 to 8 are the numbers of the CL neurons 51 of the eight-direction detection unit 12. For example, when an input signal is emitted from the direction of 0 ° (position shown in FIG. 17A), the firing rate of the CL neuron 51 is 99.4% for No. 1, 0.0% for No. 2 to 7, and No. 8 for No. 8. 0.6%. Similarly, in the case of 45 ° direction (position shown in FIG. 17B), the firing rate of No. 2 is 86.1%, and in the case of 90 ° direction (position shown in FIG. 17C). It can be seen that the firing rate of the CL neuron 51 corresponding to the direction of the input signal is the highest in the 8-direction detection unit 12 such that the firing rate of No. 3 is 92.6%. It can be seen from the firing of the CL neuron 51 whether the input signal has come from the direction of.

  As described above, in the sound source localization apparatus 1, the time difference detection unit 8 detects the time difference of the sound, the sound pressure difference detection unit 9 detects the sound pressure difference, and the left / right direction detection unit 10 performs vector quantization of the time difference information. Are output as direction information in the left-right direction, the front-rear direction detection unit 11 performs vector quantization of the sound pressure difference information and outputs it as direction information in the front-rear direction, and the 8-direction detection unit 12 is a direction in the left-right direction and the front-rear direction. Vector quantization of the information is performed and output as direction information in the surrounding eight directions. These detections are performed using a PN model, and vector quantization is performed using a CONP including a PN model. Since the PN model can be realized by a simple digital circuit as described above, the time difference detection unit 8, the sound pressure difference detection unit 9, the left / right direction detection unit 10, the front / rear direction detection unit 11, and the eight direction detection unit 12 are also simple. It can be realized with a simple digital circuit and can be easily mounted on an FPGA. Therefore, the sound source localization apparatus 1 is advantageous in terms of hardware implementation. If the time difference detection unit 8 or the like is realized by a digital circuit, the calculation in each PN model is executed in parallel on the digital circuit, so that a practical calculation speed can be realized.

  In the sound source localization apparatus 1, an 8-direction detection unit 12 is provided, and outputs from the left-right direction detection unit 10 and the front-rear direction detection unit 11 are input to the 8-direction detection unit 12. Although the direction information of the sound source in the direction is output, whether or not the eight-direction detection unit 12 is provided is arbitrary. For example, without providing the 8-direction detection unit 12, the direction in the left-right direction is estimated based on the direction information output from the left-right direction detection unit 10, and in the front-rear direction based on the direction information output from the front-rear direction detection unit 11. The direction may be estimated, and those estimation results may be directly output to the display device 5 and displayed. However, if a surrounding direction detection unit such as the 8-direction detection unit 12 is provided, the direction of the sound source can be easily identified.

It is a schematic diagram of a PN model. This is an example in which the PN model is configured by a digital circuit. It is a figure for demonstrating the method of the conventional sound source localization. It is a top view of the sound source localization apparatus which concerns on one Embodiment of this invention. It is a block diagram which shows the structure of the sound source localization apparatus. It is a block diagram which shows the structure of an input signal processing part. It is a schematic diagram of a time difference detection model. It is a figure which shows the structure of a MSO neuron row | line | column. It is the example which comprised the time difference detection part with the digital circuit, (a) is the figure for demonstrating operation | movement of the first half of 1 clock, (b) is operation | movement for the latter half. It is a schematic diagram of a sound pressure difference detection model. It is a figure which shows the structure of a LSO neuron row | line | column. It is a schematic diagram of CONP. It is a schematic diagram of CL neuron in CONP. It is a flowchart which shows the operation | movement of CONP. It is a flowchart which shows the operation | movement of CONP. This is an example in which CONP is configured by a digital circuit. It is a schematic diagram of a time difference detection part, a sound pressure difference detection part, a left-right direction detection part, a front-back direction detection part, and an 8-direction detection part. It is a figure which shows the position of the sound source in experiment. It is a figure which shows the output of the time difference detection part at the time of putting a sound source in the 0 degree position. It is a figure which shows the output of the time difference detection part at the time of putting a sound source in the 45 degree position. It is a figure which shows the output of the time difference detection part at the time of putting a sound source in the 90-degree position. It is a figure which shows the output of the time difference detection part at the time of putting a sound source in the position of 135 degrees. It is a figure which shows the output of the time difference detection part at the time of putting a sound source in the position of 180 degrees. It is a figure which shows the output of the time difference detection part at the time of putting a sound source in the position of 225 degrees. It is a figure which shows the output of the time difference detection part at the time of putting a sound source in the position of 270 degrees. It is a figure which shows the output of the time difference detection part at the time of putting a sound source in the position of 315 degrees. It is a figure which shows the output of the sound pressure difference detection part at the time of putting a sound source in the 0 degree position. It is a figure which shows the output of the sound pressure difference detection part at the time of putting a sound source in the 45-degree position. It is a figure which shows the output of the sound pressure difference detection part at the time of putting a sound source in the 90 degree position. It is a figure which shows the output of the sound pressure difference detection part at the time of putting a sound source in the position of 135 degrees. It is a figure which shows the output of the sound pressure difference detection part at the time of putting a sound source in the position of 180 degrees. It is a figure which shows the output of the sound pressure difference detection part at the time of putting a sound source in the position of 225 degrees. It is a figure which shows the output of the sound pressure difference detection part at the time of putting a sound source in the position of 270 degrees. It is a figure which shows the output of the sound pressure difference detection part at the time of putting a sound source in the position of 315 degrees.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Sound source localization apparatus 2, 3 ... Microphone 8 ... Time difference detection part 9 ... Sound pressure difference detection part 10 ... Left-right direction detection part 11 ... Front-back direction detection part 12 ... 8 direction detection part

Claims (2)

Two microphones each having front directivity, spaced apart from each other on the left and right, one facing forward and the other facing backward;
Time difference detection means for detecting time difference information of the sound collected by each microphone using a pulse neuron model;
Sound pressure difference detection means for detecting the sound pressure difference information of the sound collected by each microphone using a pulse neuron model;
Based on the time difference information of the sound detected by the time difference detection means, left and right direction detection means for detecting the direction information of the sound source in the left and right direction by a pulse neuron model,
Based on the sound pressure difference information of the sound detected by the sound pressure difference detection means, the front-rear direction detection means for detecting the direction information of the sound source in the front-rear direction by a pulse neuron model;
A sound source localization apparatus comprising: Based on the direction information of the sound source in the left-right direction detected by the left-right direction detection unit and the direction information of the sound source in the front-rear direction detected by the front-rear direction detection unit, the direction information of the sound source in a plurality of surrounding directions is pulsed 2. The sound source localization apparatus according to claim 1, further comprising a surrounding direction detection means for detecting by a neuron model.