JP2020038284A - Acoustic processing device, acoustic processing method and program - Google Patents

Abstract

To provide an acoustic processing device, an acoustic processing method and a program that can easily obtain sound collected in a situation in which a sound collection position changes.SOLUTION: A sound collection position discretization unit discretizes a sound collection position as a position of a sound collection unit which moves, at predetermined time intervals, and a simulation unit acquires an impulse response indicative of transfer characteristics from a sound source position to the sound collection position. The impulse response includes N (N: an integer larger than 1) response coefficients from a "0"th response coefficient to an "N-1"th response coefficient at respective points of time. Then signal values from a signal value at current time t to a signal value at time t-(N-1) are used to perform convolutional operation on response coefficients from a "0"th response coefficient at the current time t to an "N-1"th response coefficient at time t-(N-1) which is an N-1 time before the current time t, and the signal values obtained by discretizing an acoustic signal that a sound source generates, at the predetermined intervals of time, thereby calculating a signal value at the current time t which indicates an acoustic signal at the sound collection position.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a sound processing device, a sound processing method, and a program.

  Speech recognition is a process for specifying the content of an uttered voice, and is applied in various environments as an elemental technology of artificial intelligence (AI). In speech recognition, generally, an acoustic model indicating a statistical relationship between an acoustic feature indicating a physical characteristic of a speech and pronunciation is used. Conventionally, acoustic model learning has been performed on the premise of a static environment in which the positional relationship between a speaker and a microphone is fixed.

JP 2008-219884 A

  With the spread of AI, speech recognition may be applied in a dynamic environment. For example, a voice recognition engine may be mounted on a moving object such as a robot. In such a case, a microphone for collecting sound is also installed on the moving body. Since the positional relationship between the speaker and the microphone changes, the acoustic feature of the collected sound also changes. Therefore, when speech recognition is performed in a dynamic environment, the recognition rate tends to decrease if an acoustic model learned in a static environment is used as it is.

  When speech recognition is performed in a dynamic environment, it is expected to improve the recognition rate by using an acoustic model corresponding to the environment. The positional relationship between the speaker and the microphone can change each time, but the method described in Patent Document 1 assumes that the microphone is stationary. However, it is complicated to acquire the audio data for learning while actually changing the position of the microphone, that is, the sound pickup position, for learning the acoustic model. Therefore, it is expected that the sound data to be picked up can be easily obtained assuming an environment in which the picked-up position changes.

  The present invention has been made in view of the above points, and it is an object of the present invention to easily obtain a sound picked up in a situation where a sound pickup position changes.

(1) The present invention has been made to solve the above-described problem, and one embodiment of the present invention is a sound pickup position that digitizes a sound pickup position that is a position of a moving sound pickup unit at predetermined time intervals. A discretization unit and an impulse response indicating a transfer characteristic from the sound source position to the sound pickup position are acquired, and the impulse response is N (N is from 0th response coefficient to N-1th response coefficient for each time). Response coefficients from the 0th response coefficient at the current time t to the (N-1) th response coefficient at a time t- (N-1) immediately before the current time t. Convolution operation using a response coefficient and a signal value obtained by discretizing an acoustic signal emitted from a sound source at the predetermined time interval from a signal value at the current time t to a signal value at a time t− (N−1). To obtain an acoustic signal at the sound pickup position. A simulation unit for calculating a signal value at to the current time t, a sound processing apparatus comprising a.

(2) Another embodiment of the present invention is the above-described sound processing device, further comprising a sound source position discretization unit that discretizes the moving sound source position at predetermined time intervals, and wherein the simulation unit performs discretization. An impulse response indicating a transfer characteristic from the sound source position to the sound pickup position is obtained.

(3) Another aspect of the present invention is the above-described sound processing device, wherein the simulation unit performs a simulation of T + N−1 rows and T columns (T is an integer greater than N) including the response coefficient as an element value. A t-th row from the 0th row to the (N-2) th row of the simulation matrix includes a 0th response coefficient based on the sound pickup position at time t to a 0th response coefficient based on the sound pickup position at time t. , And T- (t + 1) 0s as element values of each column, and the t-th row from the (N−1) th row to the T−1-th row of the simulation matrix has T−N + 1 0, the response coefficients from the (N−1) th response coefficient based on the sound pickup position at time t to the 0th response coefficient based on the sound pickup position at time t, and T− (t + 1) 0s are the element values of each column. As the simulation line The t-th row from the T-th row to the (T + N-2) -th row has t-N + 1 zeros and the (n−1) th response coefficient based on the sound pickup position at time t, and the t-th line based on the sound pickup position at time t. A response signal up to −T + 1 response coefficient is included as an element value of each column, and an acoustic signal vector including a signal value from the signal value at time 0 to the signal value at time T−1 as an element value of each row is generated. The simulation matrix is multiplied by the acoustic signal vector.

(4) Another aspect of the present invention is a sound processing method in a sound processing apparatus, wherein the sound processing apparatus discretizes a sound collection position, which is a position of a moving sound collection unit, at predetermined time intervals. A position discretization process and an impulse response indicating a transfer characteristic from the sound source position to the sound pickup position are acquired. , Which is an integer greater than 1), from the 0th response coefficient at the current time t to the (N-1) th response coefficient at a time t- (N-1) which is N-1 times before the current time t. And a signal value obtained by discretizing the acoustic signal emitted from the sound source at the predetermined time interval from the signal value at the current time t to the signal value at the time t− (N−1). Perform the calculation to find the sound pickup position Kicking with a simulation step of calculating the signal value at the current time t showing an acoustic signal.

(5) In another aspect of the present invention, a computer of the sound processing apparatus includes a sound collecting position discretizing step of discretizing a sound collecting position, which is a position of a moving sound collecting unit, at predetermined time intervals, and An impulse response indicating a transfer characteristic up to the sound pickup position is obtained, and the impulse response is N (N is an integer greater than 1) response coefficients from a 0th response coefficient to an N-1th response coefficient at each time. And a response coefficient from the 0th response coefficient at the current time t to an N-1th response coefficient at a time t- (N-1) immediately before the current time t and an N-1 time point, and an acoustic signal emitted by the sound source. Is convolved using the signal values from the signal value at the current time t to the signal value at the time t− (N−1) for the signal value obtained by discretizing the signal at the predetermined time interval. At the current time t indicating the sound signal, And simulation procedure for calculating the that signal value, which is a program for causing the execution.

According to the aspects (1), (4) and (5) of the present invention, it is possible to easily obtain a synthesized signal that is similar to a sound pickup signal picked up by a moving sound pickup unit.
According to the aspect (2) of the present invention, it is possible to easily obtain a synthesized signal that is similar to a sound pickup signal picked up in accordance with a sound emitted from a moving sound source.
According to the aspect (3) of the present invention, a sound pickup signal is obtained by multiplying an acoustic signal vector based on a sound source signal by an impulse response matrix including, as element values, a response coefficient of an impulse response corresponding to a sound source position and a moving sound source position. The vector is obtained. Therefore, the signal value of the picked-up signal can be easily obtained without requiring a complicated operation.

1 is a schematic block diagram illustrating a configuration example of a sound processing device according to a first embodiment. FIG. 3 is an explanatory diagram for describing a simulation method according to the first embodiment. 5 is a flowchart illustrating an example of a synthesized signal generation process according to the first embodiment. FIG. 7 is a schematic block diagram illustrating a configuration example of a sound processing device according to a second embodiment. FIG. 9 is an explanatory diagram for describing a simulation method according to a second embodiment.

(First embodiment)
Hereinafter, a first embodiment of the present invention will be described with reference to the drawings.
FIG. 1 is a schematic block diagram illustrating a configuration example of a sound processing device 1 according to the present embodiment.
The sound processing device 1 includes a sound source signal acquisition unit 11, a sound collection position acquisition unit 12, a sound collection position discretization unit 13, a simulation unit 14, and a composite signal generation unit 15.

The sound source signal acquisition unit 11 acquires a sound source signal as a sound signal to be processed. The sound source signal is a digital sound signal composed of a time series of signal values for each time sampled at time intervals corresponding to a predetermined sampling frequency (for example, 8 kHz to 48 kHz). The signal value at each sample time indicates the sound intensity at that time. The sound source signal acquisition unit 11 outputs the acquired sound source signal to the simulation unit 14.
The sound source signal acquisition unit 11 includes, for example, an AD (Analog / Digital) converter for converting an analog audio signal input from a microphone (not shown) into a digital audio signal. The microphone may be built in the sound processing device 1 or may be separate from the sound processing device 1. The sound source signal acquisition unit 11 may be an input / output interface for inputting an acoustic signal from another device separate from the own device.
Further, the sound source signal acquisition unit 11 may read a data file storing an acoustic signal indicated by a command input to the own unit from a storage unit (not shown) of the own device. The command input to the sound source signal acquisition unit 11 may be a command input from another device or a command transmitted by an operation signal input from an operation unit (not shown).

  The sound collection position acquisition unit 12 acquires a sound collection position signal indicating a sound collection position, which is one element of the simulation conditions. The sound collection position is a virtual position of a sound collection unit (for example, a microphone) that collects sound. The sound pickup position generally moves, that is, can fluctuate as time passes. The sound pickup position acquisition unit 12 generates, for example, a sound pickup position signal indicating a movement pattern of a predetermined sound pickup position. The sound collection position acquisition unit 12 may generate a sound collection position signal indicating a sound collection position sequentially specified by an operation signal input from an operation unit (not shown). The sound collection position acquisition unit 12 outputs the generated sound collection position signal to the sound collection position discretization unit 13.

  The sound collection position discretization unit 13 discretizes the sound collection position indicated by the sound collection position signal input from the sound collection position acquisition unit 12 by sampling at a time interval corresponding to a predetermined sampling frequency. This sampling frequency is a frequency equal to the sampling frequency of the acoustic signal. The input sound pickup position signal may be a digital signal indicating a sound pickup position discretized at each discretization time different from the sampling time of the acoustic signal. In that case, the sound pickup position discretizing unit 13 interpolates the sound pickup position for each time indicated by the sound pickup position signal, and collects the sound pickup position for each time discretized at a time interval corresponding to the sampling frequency. Is calculated. The sound collection position discretization unit 13 outputs the discretized sound collection position signal to the simulation unit 14.

The simulation unit 14 receives a sound source signal from the sound source signal acquisition unit 11 and a sound collection position signal from the sound collection position discretization unit 13.
The simulation unit 14 includes an impulse response acquisition unit 142. In the impulse response acquisition unit 142, model data indicating a generation model of an impulse response indicating a sound transmission characteristic from a sound source position to a sound pickup position is set in advance. In the present embodiment, it is assumed that the sound source position is stationary at a predetermined position.
The impulse response acquisition unit 142 generates an impulse response from the sound source position to the sound pickup position indicated by the sound pickup position signal at each time using the model data. Each impulse response includes N (N is an integer of 2 or more) response coefficients. In the following description, each response coefficient is referred to as an n-th (n is an integer from 0 to N-1). The response period, which is the length of the impulse response, may be, for example, about the same as the reverberation time (for example, 0.1 s to 2.0 s) of the space including the simulation target sound source position and the sound pickup position. Therefore, the order N of the impulse response may be, for example, a value obtained by dividing a real value obtained by dividing a response period by a sampling interval to an integer.

The simulation unit 14 performs a convolution operation on the sound source signal using the impulse response generated at each discrete time t. Here, the simulation unit 14 _{calculates the} N-th response coefficient h ′ _{q (t)} (0) at the current time t from the N-th response coefficient at the time t− (N−1) before (N−1) time before the current time t. -1 response coefficient _{h′q (t− (N−1))} (N−1) is calculated from the signal value s (t) of the sound source signal at the current time t to the (N−1) th time from the current time t. The sum of the multiplied values obtained by multiplying the previous signal value s (t− (N−1)) by the signal value x ′ (t of the sound that can be collected at the sound collection position q (t) at the current time t. ). The simulation unit 14 outputs the calculated signal value to the composite signal generation unit 15.

  The synthesized signal generation unit 15 generates a synthesized signal indicating a time series of signal values input from the simulation unit 14. The composite signal indicates a time series of signal values calculated by the simulation. The composite signal generation unit 15 outputs the generated composite signal to another device, for example. The output destination device is, for example, a voice recognition device, a speaker (not shown), or the like. The speech recognition device includes an acoustic model learning unit (not shown), and can generate an acoustic model using the synthesized signal input from the synthesized signal generation unit 15. The speaker reproduces a sound based on the composite signal input from the composite signal generation unit 15. The sound arriving at the moving sound pickup position is reproduced by the speaker. Further, the composite signal generation unit 15 may store the generated composite signal in a storage unit (not shown) of the own device without outputting the generated composite signal to another device.

(Simulation method)
Next, a simulation method according to the present embodiment will be described.
FIG. 2 illustrates the impulse response h ′ _{q (t)} at time t. The impulse response h ′ _{q (t)} indicates a sound transmission characteristic from the sound source position p, which is the position of the sound source Sr, to the sound collecting position q (t), which is the position of the sound collecting unit Mc. The impulse response h ′ _{q (t) at} time t is obtained by calculating the 0th-order response coefficient _{h′q (t)} (0) to the N−1th-order response coefficient _{h′q (t)} (N−1), respectively. N-dimensional vector [h ′ _{q (t)} (0), h ′ _{q (t)} (1), h ′ _{q (t)} (2),..., H _{q (t)} (N−1) ] ^Represented as ^T. Here, [...] ^T indicates transposition of a vector or matrix [...]. In the present application, the first row and column of a vector or matrix are referred to as a 0th row and a 0th column, respectively.

When both the sound source position p and the sound collection position q (t) are stationary, the signal value x ′ _{q (t)} (t) of the acoustic signal collected at the sound collection position q (t) at time t is In a manner similar to the conventional method, the sound source signal s (t) is calculated by performing a convolution operation on the impulse response h ′ _{q (t)} . The convolution operation is performed at the current time t of the signal value s (t-τ) of the sound source signal at the past time t−τ before a predetermined sample τ (τ is an integer of 1 to N−1) before the current time t. It can also be regarded as a mathematical model in which the contribution rate to the signal value x ′ _{q (t)} (t) is the τ-th order response coefficient h ′ _{q (t)} (τ).

However, in the present embodiment, the impulse response h ′ _{q (t)} changes due to a change in the sound collection position q (t) with the passage of the time t. Therefore, in the convolution operation, the simulation unit 14 calculates the contribution ratio of the signal value s (t−τ) of the sound source signal at the past time t−τ to the signal value x ′ _{q (t)} (t) at the current time t. The τ-th order response coefficient h ′ _{q (t−τ)} (τ) at time t−τ is used. In other words, the simulation unit 14 calculates the N-th response coefficient _{hq (t)} (0) at the current time t from the N-th response coefficient at the time t- (N-1) at a time (N-1) time before the current time t. One response coefficient h ′ _{q (t− (N−1))} (N−1) is calculated based on the signal value s (t) of the sound source signal at the current time t and before the (N−1) th time before the current time t. Is multiplied by the signal value s (t− (N−1)) at the current time t, and the signal value x ′ of the sound that can be collected at the sound collection position q (t) at the current time t _{q (t)} is calculated as (t).

Simulation unit 14, as shown in Equation (1) can be calculated to the sound source signal vector s, 'multiplied by _{q (t)} synthesized signal vector x' impulse response matrix H _q a _(t).

The sound source signal vector s is represented as [s (0), s (1), s (2), ..., s (t), ..., s (T-1)] ^T. That is, the sound source signal vector s is a T-dimensional column vector including the signal value s (t) of the sound source signal at the time t as the t-dimensional element. T indicates the number of samples (period) of the sound source signal to be calculated. T is an integer greater than the order N of the impulse response.
The composite signal vector x ′ _{q (t)} is [x ′ _{q (0)} (0), x ′ _{q (1)} (1), x ′ _{q (2)} (2),..., X ′ _{q (t)} (t), ..., x ' q (T + N-2) (T + N-2)] it is expressed as ^T. That is, the synthesized signal vector x ′ _{q (t)} is a T + N−1-dimensional vector including the signal value x ′ _{q (t)} (t) of the synthesized signal at the time t as the t-dimensional element.
Impulse response matrix H _{'q (t) is, [h' 0, h '} 1, h' 2, ..., h 't, ..., h' T + N-2] is expressed as ^T. That is, the impulse response matrix H ′ _{q (t)} is a matrix of T + N−1 rows and T columns including a T-dimensional element vector h ′ _t as an element of the t-th row.

Each of the element vectors h ′ _t is a T-dimensional row vector represented by the following equation.

The simulation unit 14 calculates the signal value of the composite signal according to the procedure described below.
FIG. 3 is a flowchart illustrating an example of the composite signal generation process according to the present embodiment.
(Step S102) The simulation unit 14 configures the sound source signal vector s by arranging the sound source signal from the signal value s (0) at time 0 to the signal value s (N-1) at time N-1 in that order. .
(Step S104) The impulse response acquisition unit 142 uses the model data set in advance to collect sound at time T + N-2 from the impulse response h ′ _{q (0)} corresponding to the sound collection position q (0) at time 0. An impulse response h ′ _{q (T + N−2) up} to a position q (T + _N−2) is generated.

(Step S106) The simulation unit 14 constructs an impulse response matrix H ′ from the generated impulse responses h ′ _{q (0)} −h ′ _{q (T + N−2)} . When constructing the impulse response matrix H ′, in the t-th row from the 0-th row to the (N−2) -th row, the simulation unit 14 calculates the t-th response coefficient h ′ _{q (t)} (t) at the time t from the 0-th to the 0-th. T + 1 response coefficients up to a response coefficient h ′ _{q (t)} (0) and T− (t + 1) 0s (zero; scalar value) are arrayed in that order as element values of each column.
In the t-th row from the (N−1) -th row to the (T−1) -th row, the simulation unit 14 calculates t−N + 1 zeros and the (N−1 _{) -th} response coefficient h ′ _{q (t)} (N−1 _{) at} time t. ) To the 0th response coefficient h ′ _{q (t)} (0) and T− (t + 1) 0s are arranged in that order as element values of each column. In the t-th row from the T-th row to the (T + N−2) -th row, the simulation unit 14 calculates t−N + 1 zeros and the (N−1) th response coefficient h ′ _{q (t)} (N−1) at time t. T + N- (t + 1) response coefficients up to the (t-T + 1) th response coefficient _{h'q (t)} (t-T + 1) are arranged in that order as element values of each column.
(Step S108) The simulation unit 14 calculates a synthesized signal vector _{x'q (t)} by multiplying the sound source signal vector s by the impulse response matrix H '. The simulation unit 14 outputs the element value x ′ _{q (t)} (t) of the composite signal vector x ′ _{q (t) to} the composite signal generation unit 15 as the signal value of the composite signal at time t.

(Impulse response generation model)
Next, an example of an impulse response generation model will be described.
As a generation model of the impulse response, any generation model can be used as long as it is a mathematical model that can uniquely determine an impulse response according to a sound source position and a sound pickup position (or a sound pickup direction based on the sound source position). It is.
The impulse response acquisition unit 142 can use, for example, a geometric sound propagation model as a generation model of the impulse response. A spherical wave model can be used as one of the simple acoustic propagation models. Spherical wave model, sound pressure at the sound collecting position q is attenuated in inverse proportion to the distance r from the sound source position to the sound collection position, a model representing the delaying from the time at the sound source position by the propagation time t _p. Propagation time t _p is obtained by dividing the distance r at the speed of sound v.

  Further, the impulse response acquisition unit 142 may calculate the transfer function at the sound collection position by interpolating the transfer function actually measured for each of the plurality of sound receiving points in advance. By performing an inverse Fourier transform on the transfer function calculated in the frequency domain, an impulse response in the time domain can be obtained. As a method of interpolating a plurality of transfer functions, an FDLI (Frequency Domain Linear or bi-linear Interpolation) method, a TDLI (Time Domain Linear Interpolation) method, and an FTDLI (Frequency Timeline Inner method) May be used. The FDLI method is a method of linearly interpolating a transfer function for each sound receiving point in a frequency domain between two or more sound receiving points and calculating a transfer function for a sound pickup position. The TDLI method is a method of linearly interpolating a transfer function for each sound receiving point in a time domain between two or more sound receiving points in a time domain and calculating a transfer function for a sound pickup position. The FTDLI method is a method of linearly interpolating the phase of a transfer function for each sound receiving point in the frequency domain between two or more sound receiving points in the time domain and linearly interpolating the amplitude in the time domain.

Further, the impulse response acquisition unit 142 may use a model derived from a wave equation representing the propagation of a sound wave radiated from the sound source position, as a model for generating the impulse response. The Green function derived from the wave equation can be used as an impulse response indicating a transfer characteristic from a sound source position to a sound pickup position. In the room impulse response generation method proposed by Habats, a Green function derived as a boundary condition using sound reflection characteristics on a wall of a room having a rectangular parallelepiped shape is adopted as an impulse response. The method proposed by Havets is described in detail in, for example, the following document.
Havets, E .; A. (2006). Room impulse response generator. Technische Universitysite Eindhoven, Tech. Rep. 2 (2.4), 1.

(Second embodiment)
Next, a second embodiment of the present invention will be described. In the following description, differences from the first embodiment will be mainly described. The same processes and configurations as those in the first embodiment are denoted by the same reference numerals, and the description will be referred to unless otherwise specified.
FIG. 4 is a schematic diagram illustrating a configuration of the sound processing device 1 according to the present embodiment.
The sound processing device 1 includes a sound source signal acquisition unit 11, a sound collection position acquisition unit 12, a sound collection position discretization unit 13, a simulation unit 14, a synthesized signal generation unit 15, a sound source position acquisition unit 16, and a sound source position discretization unit 17 It is comprised including. That is, the sound processing device 1 illustrated in FIG. 4 further includes a sound source position acquisition unit 16 and a sound source position discretization unit 17 in addition to the sound processing device 1 illustrated in FIG.

  The sound source position acquisition unit 16 acquires a sound source position signal indicating a sound source position which is another element of the simulation condition. In this embodiment, in addition to the sound pickup position, the sound source position can also change over time. The sound source position acquisition unit 16 generates a sound source position signal indicating a movement pattern of a predetermined sound source position, for example. The sound source position acquisition unit 16 may generate a sound source position signal indicating a sound source position sequentially designated by an operation signal input from an operation unit (not shown). The sound source position acquisition unit 16 outputs the generated sound source position signal to the sound source position discretization unit 17.

  The sound source position discretization unit 17 discretizes the sound source position indicated by the sound source position signal input from the sound source position acquisition unit 16 by sampling at a time interval corresponding to a predetermined sampling frequency. This sampling frequency is a frequency equal to the sampling frequency of the acoustic signal. The input sound source position signal may be a digital signal indicating the sound source position at each discrete time different from the sampling time of the acoustic signal. In that case, the sound source position discretizing unit 17 interpolates the sound pickup position for each time indicated by the sound source position signal, and calculates the sound pickup position for each time discretized at a time interval corresponding to the sampling frequency. I do. The sound source position discretization unit 17 outputs the discretized sound source position signal to the simulation unit 14.

The simulation unit 14 receives a sound source signal from the sound source signal acquisition unit 11, a sound collection position signal from the sound collection position discretization unit 13, and a sound source position signal from the sound source position discretization unit 17. .
The impulse response acquisition unit 142 generates an impulse response from the sound source position to the sound pickup position indicated by the sound pickup position signal at each discretized time using the model data. In the present embodiment, the sound source position and the sound pickup position for each time are indicated by the input sound source position signal and the sound pickup position signal, respectively. Therefore, the generated impulse response h ″ _{q (t) p (t)} depends on the sound pickup position q (t) and the sound source position p (t).
The simulation unit 14 performs a convolution operation on the sound source signal using a response coefficient included in the impulse response generated at each time t according to a simulation method described later. The simulation unit 14 outputs the signal value x ″ (t) of the sound that can be collected at the sound collection position q (t) at each time t obtained by the convolution operation to the composite signal generation unit 15.

(Simulation method)
Next, a simulation method according to the present embodiment will be described.
FIG. 5 shows an impulse response h ″ _{p (t) q (t)} at time t. The impulse response h ″ _{p (t) q (t)} is shifted from the sound source position p (t) to the sound pickup position q (t). This shows the transmission characteristics of sound up to this point. The impulse response h ″ _{p (t) q (t) at} time t is calculated from the 0th-order response coefficient h ″ _{p (t) q (t)} (0) to the (N−1) th-order response coefficient h ″ _{p (t ) Q (t)} (N−1) as an N-dimensional vector [h ″ _{p (t) q (t)} (0), h ″ _{p (t) q (t)} (1), h ″ _{p (t) q (t)} (2),..., h ″ _{p (t) q (t)} (N−1)] ^T.

In this embodiment, since both the sound source position p (t) and the sound pickup position q (t) can change over time, the impulse response h ″ _{p (t) q (t)} also changes over time. With respect to the fluctuation of the sound pickup position q (t), the simulation unit 14 performs, in the convolution operation, the signal value at the current time t of the signal value s (t−τ) of the sound source signal at the past time t−τ. As the contribution rate to x ″ _{q (t)} (t), the τ-th order response coefficient h ″ _{p (t) q (t−τ)} (τ) for the sound pickup position q (t−τ) at time t−τ. Here, regarding the fluctuation of the sound source position p (t), the sound source Sr disposed at the sound source position p (t) at each time t emits a sound based on the signal value s (t), and the like. The sound source located at the sound source position p (t−τ) at time t−τ does not emit sound. Assume that.

Therefore, the simulation unit 14 calculates the 0th response coefficient h ″ _{p (t) q (t)} (0) of the impulse response corresponding to the sound source position p (t) and the sound pickup position q (t) at the current time t. The (N−1) th response coefficient h of the impulse response corresponding to the sound source position p (t−N + 1) at time t−N + 1 before (N−1) time before the current time t and the sound pickup position q (t) at current time t " _{P (t-N + 1) q (t))} Each of the N response coefficients up to (N-1) is calculated from the signal value s (t) of the sound source signal at the current time t to the N-th The sum of the multiplied values obtained by multiplying each of the signal values up to the signal value s (t−N + 1) one time before is represented by the signal value x ″ _{q of the} sound that can be collected at the sound collection position q (t) at the current time t. _(T) Calculated as (t).

In the present embodiment, the simulation unit 14 calculates the composite signal vector x ″ _{q (t)} by multiplying the sound source signal vector s by the impulse response matrix H ″ _{q (t)} as shown in Expression (3). Can be.

The composite signal vector x ″ _{q (t)} is [x ″ _{q (0)} (0), x ″ _{q (1)} (1), x ″ _{q (2)} (2),..., X ″ _{q (t) (t), ..., x "} q (T + N-2) (T + N-2)] it is expressed as ^T.
The impulse response matrix H " _{q (t)} is represented as [h" ₀ , h " ₁ , h" ₂ , ..., h " _t , ..., h" _{T + N-2} ] ^T.
Each of the element vectors h ″ _t is a T-dimensional row vector represented by the following equation.

Thus, the impulse response obtaining unit 142, at step S104 (FIG. 3), using the model data, each time _t 1 _{(t 1} is an integer from 0 to T-1) sound source position p _{(t 1)} in the If an impulse response h ″ _{p (t1) q (t2)} corresponding to each pair with the sound pickup position q (t ₂ ) at each time t ₂ (t ₂ is an integer from 0 to T + N−2 ₎ , Good.

The simulation unit 14 constructs an impulse response matrix H ″ from the generated impulse response h ″ _{p (t1) q (t2)} in step S106 (in FIG. 3). When constructing the impulse response matrix H ″, in the t th row from the 0 th row to the N−2 th row, the simulation unit 14 sets the sound source position p (0) at time 0 and the sound pickup position q (t )), The impulse response corresponding to the sound source position p (t) at time t and the sound pickup position q (t) at time t from the t-th response coefficient h ″ _{p (0) q (t)} (t) of the impulse response corresponding to The t + 1 response coefficients up to the 0th response coefficient h ″ _{p (t) q (t)} (0) and the T− (t + 1) 0s are arrayed in that order as element values of each column.
In the t-th row from the (N−1) -th row to the (T−1) -th row, the simulation unit 14 collects t−N + 1 zeros, the sound source position p (t−N + 1) at time t−N + 1, and the sound pickup at time t. The sound source position p (t) at time t and the sound pickup at time t from the (N−1) th response coefficient h ″ _{p (t−N + 1) q (t)} (N−1) of the impulse response corresponding to the position q (t) The 0th response coefficient h ″ of the impulse response corresponding to the position q (t), N response coefficients up to _{p (t) q (t)} (0), and T− (t + 1) 0s are elements of each column. The values are arranged in that order.
In the t-th row from the T-th row to the (T + N-2) -th row, the simulation unit 14 calculates T−N + 1 0s, the sound source position p (t−N + 1) at time t−N + 1, and the sound pickup position q at time t. From the (N−1) th response coefficient h ″ _{p (t−N + 1) q (t)} (N−1) of the impulse response corresponding to (t), the sound source position p (T−1) at time T−1 and the time t T + N- (t + 1) response coefficients up to t-T + 1th response coefficient h " _{p (T-1) q (t)} (t-T + 1)) of the impulse response corresponding to the sound pickup position q (t) Arrange in that order as the element value of each column.

(Evaluation experiment)
In order to verify the effectiveness of the sound processing method according to the above-described embodiment, the applicant performed two items of evaluation experiments. In Experiment 1, the reproducibility of the Doppler effect of the synthesized signal was verified. In Experiment 1, the following movement patterns (a) to (c) were set as the positional relationship between a speaker serving as a sound source and a microphone serving as a sound pickup unit.
Pattern (a) Initially, the distance from the sound source position to the sound pickup position is set to 18.74 m, and the sound source position is approached to the sound pickup position at a speed of 40 m / s per second while the sound pickup position is stationary. Was.
Pattern (b) Initially, the distance from the sound source position to the sound pickup position was set to 8.5 m, and the sound pickup position was approached to the sound source position at a speed of 40 m / s while the sound source position was kept still. .
Pattern (c) Initially, the distance from the sound source position to the sound pickup position was set to 26.74 m, and the sound source position and the sound pickup position were approached each other at a speed of 40 m / s per second. Therefore, in the generation of the composite signal, the second embodiment is applied to the patterns (a) and (c), and the first embodiment is applied to the pattern (b).
To generate the composite signal, a sound source signal having a period of 0.2 s and an impulse response having a length of 0.256 s were used. In generating the impulse response, the method proposed by Havets was used. However, it was assumed that the sound velocity was 340 m / s, the sampling frequency was 8 kHz, the reverberation time was 0.2 s, and the reflection order was 0. In addition, omnidirectionality was assumed as the directional characteristic of the microphone.
In order to evaluate the effectiveness of the verification result, the frequency of the synthesized signal was compared with the theoretical value of the frequency of the collected signal. According to the Doppler effect, the theoretical value f of the frequency of the sound collection signal ', as shown in equation (5), with respect to the frequency f of the excitation signal, for the difference between the moving velocity v _s of sound velocity V and the sound source position The frequency is obtained by multiplying the ratio of the sum of the sound speed V and the moving speed _vo of the sound pickup position.

  In the pattern (a), the theoretical value was 1133.33 Hz, whereas the frequency of the synthesized signal was 1133.42 Hz. In the pattern (b), the theoretical value was 1117.65 Hz, whereas the frequency of the combined signal was 1117.71 Hz. In the pattern (c), the theoretical value was 1266.67 Hz, whereas the frequency of the synthesized signal was 1266.64 Hz. In all of the patterns (a) to (c), the difference between the frequency of the synthesized signal and the theoretical value is only 0.14 Hz or less. Therefore, the result of Experiment 1 shows that the change in frequency due to the movement of the sound source position or the sound pickup position can be sufficiently reproduced.

In Experiment 2, the volume of the synthesized signal was verified. In the verification, the volume of the collected signal obtained by actually collecting the sound emitted from the sound source was compared with the volume of the synthesized signal. As a sound source signal, a voice having utterance contents of ten sentences in a manuscript of a Wall Street Journal (WSJ) was used.
The picked-up signal was recorded while moving one or both of the speaker and the microphone in the anechoic room. The interior of the anechoic room is a rectangular parallelepiped space having a length of 6.2 m, a width of 4.8 m, and a height of 5.1 m. The loudspeaker was moved along a range of 4.0 m in the vertical direction with the center of the anechoic room as the center position. However, when the speaker was stopped, the speaker was stopped at the center position of the path. The microphone was moved along a range of 4.0 m in the vertical direction with the center position being 1.0 m away from the center of the anechoic chamber in the horizontal direction. However, when the microphone was stopped, the microphone was stopped at the center position of the path. The following movement patterns (i) to (v) were set as the positional relationship between the speaker and the microphone.

Pattern (i) Both the sound source position and the sound pickup position were stopped.
Pattern (ii) The sound pickup position was moved from one end of the path to the other end at a constant speed of 1.8 m / s while the sound source position was kept still.
Pattern (iii) The sound source position was moved from one end of the path to the other end at a constant speed of 1.8 m / s while the sound collection position was stopped.
Pattern (iv) The sound source position and the sound pickup position were moved at a constant speed from one end to the other end on each path in the same direction. However, the moving speed of the sound source position was 1.8 m / s, and the moving speed of the sound pickup position was 1.7 m / s.
Pattern (v) The sound source position and the sound pickup position were moved at the same speed on each route at a constant speed of 1.8 m / s. However, the moving directions of the sound source position and the sound pickup position are opposite to each other. The movement start position of the sound source is one end of the route, whereas the movement start position of the sound pickup position is the other end of the route. Therefore, the composite signal for the pattern (i) is obtained by performing a convolution operation on the sound source signal with the impulse response for the sound source position and the sound pickup position, as in the conventional method. A composite signal for the pattern (ii) is obtained by executing the method of the first embodiment. The composite signals for the patterns (iii) to (v) are obtained by executing the method of the second embodiment.

Before the evaluation, the amplification factor A for the combined signal is determined. The amplification factor A is a parameter for adjusting the volume of the entire synthesized signal to the volume of the entire collected signal.
The amplification factor A can be calculated based on equation (6).

x _s (f, t) indicates the signal value of the composite signal at time t of the f-th frame. _xr (f, t) indicates the signal value of the collected sound signal at time t of the f-th frame. F and N indicate the number of frames and the number of samples in the frame, respectively. Therefore, an amplification factor A ′ that makes the entire volume of the amplified composite signal equal to the volume of the collected signal as a whole is calculated as the amplification factor A.

  That is, the amplification factor A shown in Expression (6) is given under the condition that the function C (A) shown in Expression (7) is minimized.

  The derivative of the function C (A) with respect to the amplification factor A is given by equation (8).

  If both sides of equation (8) are set to 0, the relationship of equation (9) is obtained.

  By transforming equation (9), equation (10) is obtained. The amplification factor A is calculated using the equation (10).

Then, the composite signal x _s (f, t) is multiplied by the amplification factor A to calculate a corrected composite signal x ′ _s (f, t). Then, as a measure of the similarity of the sound volume of the synthesized signal and the collected signal calculates the distance D _s using the equation (11).

The distance D _s indicates the magnitude of the difference for each frame of the signal value of the composite signal and the collected sound signal. In the evaluation, using the distance D _s as a measure indicating the magnitude of the difference of the change in amplitude between the two signals due to the time change of the sound source position and sound pickup position.
For comparison, it was calculated distance _{D o} of using Equation (12) the original signal x _'o (f, t) and the picked-up signal.

In the evaluation, for each pattern (i) ~ (v), was calculated distance _{D s} and the distance _{D o.} Next, an example of calculation of the distance _{D s} and the distance _{D o.} However, the following calculation example is an average value between 10 utterances for each movement pattern.
The distance D _s, the pattern (i), for each of the (ii), (iii), (iv), (v), a 0.0110,0.0147,0.096,0.0120,0.0089 Was.
The distance D _o, a pattern (i), for each of the (ii), (iii), (iv), (v), a 0.0108,0.0302,0.0335,0.0139,0.0372 Was.
Calculated distance _{D s,} the pattern (v), (iii), (i), (iv), but increases in the order of (ii), irrespective of any pattern, about 0.01, and the sound source position and the yield There is no correlation with the relative speed of the sound position. The most relative velocity is large moving pattern even (v) the distance _{D S} is only 0.0089.
On the other hand, the distance D _o, a pattern (i), tends to become larger in the order of (iv), (ii), (iii), (v). This confirms that the greater the relative speed, the more significant the change in the sound volume due to the movement. In the pattern (i), the relative speed between the sound source position and the sound pickup position is 0, and in the movement pattern (iv), the relative speed between the sound source position and the sound pickup position is 0.1 m / s, and the movement patterns (ii), ( In iii), the relative speed between the sound source position and the sound pickup position is 1.8 m / s, and in the movement pattern (v), the relative speed between the sound source position and the sound pickup position is 3.6 m / s. Movement pattern (i), when close to the relative speed is 0 and 0 of the sound source position and the voice collecting position as (iv), the distance D _o and the distance D _s is only approximate.
Therefore, the result of Experiment 2 shows that even if the relative speed between the sound source position and the sound pickup position is increased, the change in the volume due to the movement can be reproduced.

The sound processing apparatus 1 according to the embodiment described above includes a sound pickup position discretizing unit 13 that discretizes a sound pickup position, which is a position of a moving sound pickup unit, at predetermined time intervals, and a sound pickup position to a sound pickup position. The simulation unit 14 acquires an impulse response indicating the transfer characteristic of The impulse response includes N response coefficients from the 0th response coefficient to the (N-1) th response coefficient at each time. The simulation unit 14 generates a response coefficient from a 0th response coefficient at the current time t to an N-1th response coefficient from time t- (N-1), and a signal obtained by discretizing an acoustic signal emitted from the sound source at predetermined time intervals. The convolution operation is performed on the value using the signal value from the signal value at the current time t to the signal value at the time t− (N−1), and the signal value indicating the synthesized signal that is the acoustic signal at the sound collection position is obtained. calculate.
With this configuration, it is possible to easily obtain a synthesized signal that is similar to a sound pickup signal picked up by the moving sound pickup unit.

In addition, the sound processing device 1 may further include a sound source position discretization unit 17 that discretizes a moving sound source position at predetermined time intervals. The simulation unit 14 acquires an impulse response indicating a transfer characteristic from the discretized sound source position to the sound pickup position.
With this configuration, it is possible to easily obtain a synthesized signal that is similar to a sound pickup signal picked up in accordance with a sound emitted from a moving sound source.

Further, in the sound processing device 1, the simulation unit 14 generates a simulation matrix of T + N−1 rows and T columns including a response coefficient as an element value, and generates a t-th row from the 0th row to the N-2th row of the simulation matrix. Includes the response coefficients from the t-th response coefficient based on the sound collection position at time t to the 0th response coefficient based on the sound collection position at time t, and T− (t + 1) 0s as element values of each column. The t-th row from the (N-1) -th row to the T-1-th row of the simulation matrix has T-N + 1 zeros and the N-1th response coefficient based on the pickup location at the time t, and the pickup location at the time t. , And T- (t + 1) zeros as element values of each column. The t-th row from the T-th row to the (T + N−2) -th row of the simulation matrix includes t−N + 1 zeros and the N−1 response coefficient based on the sound collection position at time t, and the sound collection position at time t. Are included as the element values of each column up to the t-T + 1th response coefficient. Then, the simulation unit 14 generates an acoustic signal vector including signal values from the signal value at time 0 to the signal value at time T-1 as element values of each row, and multiplies the generated simulation matrix by the generated acoustic matrix. .
According to this configuration, a sound pickup signal vector is obtained by multiplying the acoustic signal vector based on the sound source signal by the impulse response matrix including, as element values, the response coefficients of the impulse response corresponding to the sound source position and the moving sound source position. Therefore, the signal value of the picked-up signal can be easily obtained without requiring a complicated operation.

Note that a part of the sound processing apparatus 1 in the above-described embodiment, for example, the sound collection position discretization unit 13, the simulation unit 14, the synthesized signal generation unit 15, and the sound source position discretization unit 17 may be realized by a computer. Good. In this case, a program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read and executed by a computer system. Here, the “computer system” is a computer system including one or more processors such as a CPU built in the sound processing apparatus 1 and includes an OS and hardware such as peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in a computer system. Further, the "computer-readable recording medium" is a medium that dynamically holds the program for a short time, such as a communication line for transmitting the program through a network such as the Internet or a communication line such as a telephone line, In this case, a program holding a program for a certain period of time, such as a volatile memory in a computer system serving as a server or a client, may be included. Further, the above-mentioned program may be for realizing a part of the above-mentioned functions, and may be for realizing the above-mentioned functions in combination with a program already recorded in the computer system.
Further, a part or all of the sound processing device 1 in the above-described embodiment may be realized as an integrated circuit such as an LSI (Large Scale Integration). Each functional block of the sound processing device 1 may be individually implemented as a processor, or a part or all of the functional blocks may be integrated into a processor. The method of circuit integration is not limited to an LSI, and may be realized by a dedicated circuit or a general-purpose processor. Further, in the case where a technology for forming an integrated circuit that replaces the LSI appears due to the advance of the semiconductor technology, an integrated circuit based on the technology may be used.

As described above, one embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the gist of the present invention. It is possible to
For example, the sound processing device 1 may be configured as a part of an acoustic model learning unit (not shown) used for speech recognition. The acoustic model learning unit generates an acoustic model using the composite signal generated by the composite signal generation unit 15 for each movement pattern indicating a time series of the sound source position. The acoustic model learning unit calculates an acoustic feature value (for example, MFCC (Mel-frequency Cepstrum Coefficients; mel frequency cepstrum coefficients) for each frame having a predetermined time length (for example, 10 to 50 ms) for the generated synthesized signal, The calculated acoustic signal is used to perform an update process by a maximum likelihood linear regression (MLLR) for an existing acoustic model generated in advance. GMM (Gaussian Mixture Model; Gaussian Mixture Distribution Model), Hidden Markov Model (Hidden Markov Model) are used as acoustic models learned using uttered voices collected in a static environment where the position is fixed. Model; HMM), etc. Thereby, an acoustic model for each movement pattern can be acquired with a relatively small amount of synthesized signal.The speech recognition device uses the acoustic model generated for each movement pattern for speech recognition. By using this, it is possible to improve the recognition of the uttered voice according to the utterer and the movement pattern of the sound pickup unit.
Further, the sound processing device 1 may be configured as a sound simulator for generating and audible a synthesized signal indicating a sound propagating from a sound source position to a sound collection position in a virtual sound environment.

DESCRIPTION OF SYMBOLS 1 ... Sound processing apparatus, 11 ... Sound source signal acquisition part, 12 ... Sound collection position acquisition part, 13 ... Sound collection position discretization part, 14 ... Simulation part, 15 ... Synthetic signal generation part, 16 ... Sound source position acquisition part, 17 … Sound source position discretization unit

Claims (5)

A sound collecting position discretizing unit that discretizes a sound collecting position, which is a position of a moving sound collecting unit, at predetermined time intervals;
Obtain an impulse response indicating a transfer characteristic from a sound source position to the sound pickup position,
The impulse response includes N (N is an integer greater than 1) response coefficients from a 0th response coefficient to an N-1th response coefficient at each time,
The response coefficient from the 0th response coefficient at the current time t to the (N-1) th response coefficient at a time t- (N-1) immediately before the current time t and the sound signal emitted by the sound source is determined by the predetermined value. Convolution operation is performed on the signal values discretized at the time intervals using the signal values from the signal value at the current time t to the signal value at the time t− (N−1), and the acoustic signal at the sound collection position is obtained. A simulation unit for calculating a signal value at the current time t shown,
A sound processing device comprising: A sound source position discretization unit that discretizes the moving sound source position at predetermined time intervals,
The simulation unit includes:
The acoustic processing apparatus according to claim 1, wherein an impulse response indicating a transfer characteristic from the discretized sound source position to the sound pickup position is acquired. The simulation unit includes:
Generating a simulation matrix of T + N−1 rows and T columns (T is an integer greater than N) including the response coefficient as an element value;
The t-th row from the 0th row to the N-2th row of the simulation matrix includes response coefficients from the t-th response coefficient based on the sound collection position at time t to the 0th response coefficient based on the sound collection position at time t. , T- (t + 1) 0s as element values of each column,
The t-th row from the (N−1) -th row to the T−1-th row of the simulation matrix includes the sound pickup at the time t based on the (T−N + 1) 0s and the (N−1) th response coefficient based on the sound pickup position at the time t. Including response coefficients up to the 0th response coefficient based on the position and T- (t + 1) 0s as element values of each column,
The t-th row from the T-th row to the (T + N−2) -th row of the simulation matrix includes t−N + 1 zeros and the (N−1) th response coefficient based on the sound pickup position at the time t to the sound pickup position at the time t. Response factors up to the (t-T + 1) th response factor based on
Generating an acoustic signal vector including a signal value from the signal value at time 0 to the signal value at time T-1 as an element value of each row;
The acoustic processing device according to claim 1, wherein the acoustic matrix is multiplied by the simulation matrix. A sound processing method in a sound processing device,
The sound processing device,
A sound collection position discretization process of discretizing a sound collection position, which is a position of a moving sound collection unit, at predetermined time intervals;
Obtain an impulse response indicating a transfer characteristic from a sound source position to the sound pickup position,
The impulse response includes N (N is an integer greater than 1) response coefficients from a 0th response coefficient to an N-1th response coefficient at each time,
The response coefficient from the 0th response coefficient at the current time t to the (N-1) th response coefficient at a time t- (N-1) immediately before the current time t and the sound signal emitted by the sound source is determined by the predetermined value. Convolution operation is performed on the signal values discretized at the time intervals using the signal values from the signal value at the current time t to the signal value at the time t− (N−1), and the acoustic signal at the sound collection position is obtained. A simulation process of calculating a signal value at the current time t shown,
A sound processing method comprising: In the computer of the sound processing device,
A sound collection position discretization procedure for discretizing a sound collection position, which is a position of a moving sound collection unit, at predetermined time intervals;
Obtain an impulse response indicating a transfer characteristic from a sound source position to the sound pickup position,
The impulse response includes N (N is an integer greater than 1) response coefficients from a 0th response coefficient to an N-1th response coefficient at each time,
The response coefficient from the 0th response coefficient at the current time t to the (N-1) th response coefficient at a time t- (N-1) immediately before the current time t and the sound signal emitted by the sound source is determined by the predetermined value. Convolution operation is performed on the signal values discretized at the time intervals using the signal values from the signal value at the current time t to the signal value at the time t− (N−1), and the acoustic signal at the sound collection position is obtained. A simulation procedure for calculating a signal value at the current time t shown,
A program for executing

Images