JP2016148818A - Signal processor - Google Patents

Abstract

PROBLEM TO BE SOLVED: To perform signal processing of, when a plurality of acoustic signals are input to a signal processor, giving an acoustic effect to an acoustic signal while keeping the balance of all of the sounds issued from the plurality of acoustic signals.SOLUTION: A signal as the object of signal processing of giving an acoustic effect is set as a main acoustic signal, and the other signals are set as sub acoustic signals. The intensity of the acoustic effect is determined in accordance with the correlation values between a plurality of signals, namely between the main acoustic signal and one or the plurality of sub acoustic signals, and the signal processing is applied to the main acoustic signal. Therefore, after that, even when compression processing of a dynamic range is applied and added to the main acoustic signal and one or the plurality of sub acoustic signals, the balance between the main acoustic signal and one or the plurality of sub acoustic signals of an issued sound is not lost.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a signal processing apparatus.

  In recent years, various types of audio systems such as car stereos and home karaoke apparatuses have become popular. When the volume of the sound reproduced by this type of audio system is small and difficult to hear, the sound may be broken, that is, the sound may be distorted by simply increasing the volume. In particular, when the difference in the level of the sound signal level (that is, the dynamic range) is wide, the occurrence of sound distortion is significant. Therefore, a signal processing technique that compresses the dynamic range prior to volume adjustment is generally used so that sound distortion does not occur even when the volume is increased. Such a technique is disclosed in, for example, Patent Document 1, Patent Document 2, and Non-Patent Document 1.

  In these techniques, dynamic range compression is performed for each frame obtained by dividing an acoustic signal by a certain length of time. By performing such dynamic range compression prior to volume adjustment, if the dynamic range changes gradually between frames over time, a natural sound can be heard. However, if the dynamic range suddenly changes between frames, it will sound like an unnatural sound. For this reason, for example, in the signal processing technique disclosed in Patent Document 3, when the dynamic range of the sound signal level changes suddenly, the sound signal is heard in a natural sound by shortening the time interval for dividing the acoustic signal into frames. To be able to.

  Furthermore, in Patent Document 4, when the sound is emitted from the car stereo in the car, the sound volume of the car stereo is reduced when a siren sound that has been stored in advance is detected around the car. A signal processing technique is disclosed.

Japanese Patent No. 3123052 Japanese Patent No. 2996646 JP 2002-232247 A Japanese Patent Laid-Open No. 11-136059

Dynamic Range Control® Digital Audio Signals / G. W. McNally, Internet <URL: http: // www. cs. tut. fi / ~ sgn14006 / PDF / L05-dynamics. pdf>

  When the signal processing techniques disclosed in Patent Document 1, Patent Document 2, and Non-Patent Document 1 are applied to a karaoke apparatus, a problem may occur. The reason is as follows. The karaoke apparatus receives an acoustic signal representing a vocal sound and an acoustic signal sharing a time axis with the acoustic signal (that is, an acoustic signal representing a sound reproduced in synchronization with the vocal sound, for example, an acoustic signal representing an accompaniment sound). The karaoke apparatus adds these acoustic signals and emits the sound. When the signal processing technology disclosed in Patent Document 1 is applied to a karaoke device and the dynamic range compression is performed independently for vocal sound and accompaniment sound signal, for example, the signal level of the sound signal of vocal sound peaks. However, the balance of the entire song composed of the vocal sound and the accompaniment sound may be lost.

  Further, when the signal processing disclosed in Patent Document 3 is applied to the vocal sound and the accompaniment sound signal of the karaoke apparatus, even if the dynamic range of the signal level of the vocal sound or the accompaniment sound signal changes suddenly, it is natural. Hearing can be maintained. However, even if this signal processing technique is applied to a karaoke apparatus, it is impossible to keep the balance of the entire song composed of vocal sounds and accompaniment sounds.

  Similarly, even if the signal processing technique of Patent Document 4 is applied to a karaoke apparatus, the balance of the entire music composed of vocal sounds and accompaniment sounds cannot be maintained. In the first place, the signal processing technique of Patent Document 4 is not a technique that takes into account the sense of music. The balance between the vocal sound and accompaniment sound emitted from the karaoke device is not only lost when performing signal processing to compress the dynamic range for the vocal sound and the accompaniment sound signal. The same applies to the case of performing signal processing that gives a reverb effect.

  The present invention has been made in view of the problems described above, and when there are a plurality of sound signals input to the signal processing device, the balance of the whole sound emitted by the plurality of sound signals is maintained. An object of the present invention is to provide a technique that enables compression of a dynamic range of an acoustic signal and imparting a reverb effect to the acoustic signal.

  In order to solve the above-described problems, the present invention provides a cross-correlation for calculating a correlation value between a plurality of signals among a main acoustic signal to be subjected to signal processing that gives an acoustic effect and one or more sub-acoustic signals. There is provided a signal processing device comprising: a detection unit; and a signal processing unit that determines the intensity of the acoustic effect according to the correlation value calculated in the cross-correlation detection unit.

  According to the present invention, the intensity of the sound effect is determined according to the correlation value between a plurality of signals in the main sound signal and one or a plurality of sub sound signals, and signal processing is performed on the main sound signal. For this reason, even if the main acoustic signal and one or more sub-acoustic signals are subsequently subjected to compression processing of the dynamic range and added, the balance between the main acoustic signal and the one or more sub-acoustic signals in the sound to be emitted Will not collapse.

  In a more preferred aspect, there is provided a weighting unit that weights an audible range of at least one of the plurality of signals and gives the audible range to the cross-correlation detection unit. Such weighting increases the contribution of at least one audible range component of the main sound signal and the one or more sub sound signals to the correlation value between the main sound signal and the one or more sub sound signals. For this reason, although the correlation between the main acoustic signal and one or more sub-acoustic signals in the audible range is high, the correlation between both signals is low due to the low correlation in the other bands. Even in such a case, it is possible to perform dynamic range compression processing or the like without losing the balance of both signals in the sense of hearing.

  In a more preferred aspect, the cross-correlation detection unit calculates a plurality of correlation values between one signal of the plurality of signals and one or more signals of the plurality of signals that are different in time from the signal. Then, one correlation value is obtained from the plurality of correlation values. Even when the time of one signal among a plurality of signals and one or more signals among the plurality of signals are different, the time of each signal is different when obtaining the correlation value of each signal. Since the influence is alleviated, the balance between the main sound signal of the sound to be emitted and the one or more sub sound signals is not lost.

  In a more preferred aspect, there is provided a band dividing unit that performs band division on at least one of the main acoustic signal and the one or more sub-acoustic signals, and the cross-correlation detecting unit includes the band dividing unit The correlation value is calculated using at least one of the signals in the band divided by. According to such an aspect, it becomes possible to finely perform signal processing corresponding to the correlation with the sub-acoustic signal on the main acoustic signal for each frequency band.

  In a more preferred aspect, the main acoustic signal is a synthesized signal including at least one of the one or more sub-acoustic signals. According to such an aspect, an effect similar to the case where the signal processing is performed by performing sound source separation that separates the sub sound signal from the main sound signal can be obtained.

1 is a block diagram of a signal processing device 101 according to a first embodiment of the present invention. It is a graph which shows the relationship between the normalization value x of the signal processing apparatus 101, and the multiplier y. It is a block diagram of the signal processing apparatus 102 which is 2nd Embodiment of this invention. It is a graph which shows the loudness curve which the weighting part 50 of the signal processing apparatus 102 refers. It is a block diagram of the signal processing apparatus 103 which is 3rd Embodiment of this invention. It is a block diagram of the signal processing apparatus 104 which is 4th Embodiment of this invention. It is a graph which shows the relationship between the normalization value x in the same embodiment, and the reverb mix value g _mix . It is a block diagram of the signal processing apparatus 106 which is 6th Embodiment of this invention. It is a block diagram of the signal processing apparatus 108 which is 8th Embodiment of this invention. It is a block diagram of the signal processing apparatus 110 which is 10th Embodiment of this invention.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<First Embodiment>
FIG. 1 is a block diagram of a signal processing apparatus 101 according to the first embodiment of the present invention. The signal processing apparatus 101 is a DSP (Digital Signal Processor) and receives two types of acoustic signals. Each of these two types of acoustic signals is a digital signal representing a time waveform of sound. In the present embodiment, among the digital signals input to the signal processing device 101, a signal that is processed by the signal processing device 101 is called a main acoustic signal, and shares a time axis with the main acoustic signal, but is processed by the signal processing device 101. A signal that is not subjected to the processing is referred to as a sub acoustic signal (hereinafter, a signal to be processed is also referred to as a main acoustic signal in other embodiments). The signal processing device 101 calculates the correlation between the main sound signal and the sub sound signal, performs amplification processing of the signal level of the main sound signal based on the calculation result, and outputs it as an output signal. Signal processing such as dynamic range compression and volume amplification is performed on the output signal that has been output. These signal processing may be performed using known signal processing techniques. Therefore, in this embodiment, illustration and description of these signal processes are omitted.

  The signal processing apparatus 101 includes a time frequency conversion unit 10, a cross correlation detection unit 20, a parameter calculation unit 30, and a multiplication unit 40. The time frequency conversion unit 10 and the like are a part of functions realized by a microprogram executed by the signal processing apparatus 101. Note that the DSP may not be used as the signal processing device 101, and the time frequency conversion unit 10 and the like may be configured by an electronic circuit, and the signal processing device 101 may be realized by combining these electronic circuits.

  The time-frequency converter 10 receives two types of digital signals, a main sound signal and a sub sound signal, and divides each signal into frames of a certain time length (for example, divides into T frames), and for each frame. Perform time-frequency transform, that is, Fourier transform. The time frequency conversion unit 10 outputs the Fourier transform data calculated by the Fourier transform to the cross correlation detection unit 20.

なお、Ｃ_ｍｓは次式（２）を用いて算出される。

Ｃ_ｍｍとＣ_ｓｓは、メイン音響信号とサブ音響信号の自己相関を各々示し、次式（３）〜（４）を用いて算出される。

ｎは各フレームを一意に示すフレーム番号であり、１≦ｎ≦Ｔである。ｆ_ｍ（ｎ）とｆ_ｓ（ｎ）は、メイン音響信号とサブ音響信号のｎ番目のフレームのフーリエ変換データである。相関値Ｒ_ｍｓは−１≦Ｒ_ｍｓ≦１を満たし、相関値Ｒ_ｍｓが１の時にはメイン音響信号とサブ音響信号は完全相関であり、相関値Ｒ_ｍｓが小さくなるほどメイン音響信号とサブ音響信号の相関は低下する。相互相関検出部２０は、相関値Ｒ_ｍｓをパラメータ算出部３０に出力する。 The cross-correlation detection unit 20 detects the correlation between the main sound signal and the sub sound signal input from the time-frequency conversion unit 10. Specifically, the cross-correlation detection unit 20 calculates the correlation value R _ms of each signal based on the Fourier transform data of the main acoustic signal and the sub-acoustic signal using the cross-correlation function shown in the following equation (1).

C _ms is calculated using the following equation (2).

C _mm and C _ss indicate autocorrelations of the main acoustic signal and the sub acoustic signal, respectively, and are calculated using the following equations (3) to (4).

n is a frame number uniquely indicating each frame, and 1 ≦ n ≦ T. f _m (n) and f _s (n) are Fourier transform data of the nth frame of the main sound signal and the sub sound signal. The correlation value R _ms satisfies −1 ≦ R _ms ≦ 1, and when the correlation value R _ms is 1, the main acoustic signal and the sub acoustic signal are completely correlated, and the main acoustic signal and the sub acoustic signal decrease as the correlation value R _ms decreases. The correlation decreases. The cross-correlation detection unit 20 outputs the correlation value R _ms to the parameter calculation unit 30.

相関値Ｒ_ｍｓは、−１≦Ｒ_ｍｓ≦１を満たすので、ノーマライズ値ｘは、０≦ｘ≦１となる。図２は、ノーマライズ値ｘと増幅率ｙの関係を示すグラフである。図２のｔｈは閾値である。ノーマライズ値ｘが閾値ｔｈよりも小さければ、パラメータ算出部３０は増幅率ｙを０ｄＢとし、ノーマライズ値ｘが閾値ｔｈよりも大きければ、パラメータ算出部３０はノーマライズ値ｘが１に近づくにつれて増幅率ｙを線形に小さくする。図２では、ノーマライズ値ｘが１の時に増幅率ｙは−１０ｄＢとなるが、この増幅率ｙの値は−１０ｄＢに限られるものではなく、実験により好適な値を定めればよい。さらに、図２では、ノーマライズ値ｘが閾値ｔｈよりも大きくなると増幅率ｙが線形に小さくなる場合について例示されているが、ノーマライズ値ｘと増幅率ｙの関数関係は非線形でもよく、実験により好適な態様を取ればよい。 The parameter calculation unit 30 calculates the amplification factor y based on the correlation value R _ms input from the cross correlation detection unit 20 and outputs it to the multiplication unit 40. The amplification factor y is a value indicating the strength of amplification processing performed on the signal level of the main acoustic signal. More specifically, the parameter calculation unit 30 first calculates the normalized value x using the following equation (5).

Since the correlation value R _ms satisfies −1 ≦ R _ms ≦ 1, the normalized value x satisfies 0 ≦ x ≦ 1. FIG. 2 is a graph showing the relationship between the normalized value x and the amplification factor y. In FIG. 2, th is a threshold value. If the normalization value x is smaller than the threshold th, the parameter calculation unit 30 sets the amplification factor y to 0 dB. If the normalization value x is larger than the threshold th, the parameter calculation unit 30 determines that the amplification factor y increases as the normalization value x approaches 1. Is reduced linearly. In FIG. 2, when the normalized value x is 1, the amplification factor y is −10 dB. However, the amplification factor y is not limited to −10 dB, and a suitable value may be determined by experiment. Further, FIG. 2 illustrates the case where the amplification factor y decreases linearly when the normalization value x becomes larger than the threshold th, but the functional relationship between the normalization value x and the amplification factor y may be non-linear and is suitable by experiment. What is necessary is just to take this aspect.

  The multiplier 40 amplifies the signal level of the main acoustic signal based on the amplification factor y input from the parameter calculator 30, and outputs the amplified main acoustic signal as an output signal. That is, the multiplication unit 40 functions as a signal processing unit that performs amplification processing on the main acoustic signal.

If the main sound signal and the sub sound signal are completely correlated, that is, if the correlation value R _ms is 1, the normalized value x is 1 from the equation (5). In this case, the amplification factor y is −10 dB from FIG. 2, and the signal processing apparatus 101 outputs the main acoustic signal whose signal level is lower than the input level as an output signal. Since the main acoustic signal and the sub acoustic signal are completely correlated, the main acoustic signal output from the signal processing apparatus 101 is subjected to signal processing such as dynamic range compression and added to the sub acoustic signal. The decrease in the signal level is supplemented by the sub-acoustic signal. Therefore, even after the main sound signal is output from the signal processing device 101, the main sound signal and the sub sound signal are added, and even if the sound is finally emitted, the balance between the main sound signal and the sub sound signal will not be lost. .

When the correlation between the main acoustic signal and the sub-acoustic signal becomes low, that is, when the correlation value R _ms approaches −1, the normalized value x approaches 0 according to the equation (5). In this case, the amplification factor y is 0 from FIG. 2, and the signal processing apparatus 101 outputs the output signal as an output signal without amplifying the signal level of the input main sound signal. Since the correlation between the main sound signal and the sub sound signal is low, the peak (or dip) of the main sound signal and the peak (or dip) of the sub sound signal do not collide with each other. Therefore, the main acoustic signal output from the signal processing device 101 is subjected to signal processing such as compression of the dynamic range, added to the sub-acoustic signal, and finally output the sound even if the main acoustic signal and the sub-acoustic signal are emitted. There is no loss of balance.

  By using the signal processing device 101 in this way, it is possible to prevent the balance between the main sound signal and the sub sound signal that is generated by simply adding the main sound signal and the sub sound signal.

  For example, application of the signal processing device 101 to a karaoke device is assumed. The karaoke apparatus receives an acoustic signal representing a vocal sound and an acoustic signal representing an accompaniment sound. In the following, the drum sound is considered among the accompaniment sounds, and the sound signal of the vocal sound whose frequency is concentrated in the middle range (200 to 1000 Hz) is set as the main acoustic signal, and the drum whose frequency is concentrated in the high range (1000 Hz or more). The sound signal of sound is defined as a sub sound signal. In the prior art, since the correlation between the vocal sound and the drum sound is not taken into account, if the vocal sound signal and the accompaniment sound signal are subjected to processing such as compression of the dynamic range and added, the sound of the vocal sound and the drum sound is added. In some cases, the peak (or dip) of the signal collides, and the balance between the vocal sound and the drum sound is lost in the sound finally emitted. On the other hand, when the signal processing device 101 is used, if the correlation between the vocal sound and the drum sound is high, the dynamic range is compressed after the signal level of the vocal sound is lowered, and the dynamic range is also compressed. Addition with drum sound is performed. Therefore, the balance between the vocal sound and the drum sound is not lost in the sound finally emitted.

  The frequency at which the peak (or dip) of the main sound signal and the peak (or dip) of the sub sound signal collide with each other so that the balance between the main sound signal and the sub sound signal does not collapse in the sound that is finally emitted. A method is conceivable in which a band signal is subtracted from the main sound signal, and then the main sound signal is subjected to signal processing such as compression of a dynamic range and added to the sub sound signal to be emitted. However, in this method, an artificial noise is generated at the time of spectral subtraction in which a signal in a frequency band in which the main acoustic signal and the sub acoustic signal collide are subtracted from the main acoustic signal, and noise is finally generated in the sound to be emitted. Sometimes. Since the signal processing apparatus 101 does not use spectral subtraction, artificial noise is generated, and noise is not generated in the sound finally emitted.

Second Embodiment
FIG. 3 is a block diagram of the signal processing apparatus 102 according to the second embodiment of the present invention. In FIG. 3, the same components as those in FIG. 1 are denoted by the same reference numerals. As apparent from a comparison between FIG. 1 and FIG. 3, the difference between the signal processing device 102 and the signal processing device 101 is that a weighting unit 50 is provided between the time-frequency conversion unit 10 and the cross-correlation detection unit 20. It is. Below, it demonstrates centering on the weighting part 50 which is a difference with 1st Embodiment.

  The weighting unit 50 weights the Fourier transform data of the main sound signal and the sub sound signal input from the time frequency conversion unit 10 based on the loudness curve. Specifically, the weighting unit 50 stores in advance data representing a 40 phon curve of the loudness curve shown in FIG. The weighting unit 50 refers to data having a frequency of 20 Hz to 10 kHz, which is a frequency band (hereinafter, audible range) that is easy for humans to listen to, and sets the maximum sound pressure of the waveform of the referenced data to 1, and the minimum sound pressure. Weighted multiplication normalized as 0 is performed on the Fourier transform data of the main sound signal and the sub sound signal. By this weighting multiplication, the Fourier transform data of the sound included in the audible range becomes larger, and the Fourier transform data of the sound included in the frequency band difficult to hear becomes smaller. The weighting unit 50 outputs the Fourier transform data of the main sound signal and the sub sound signal subjected to weighting multiplication to the cross correlation detection unit 20.

The cross-correlation detection unit 20 calculates a correlation value R _ms from the Fourier transform data of the main sound signal and the sub sound signal input from the weighting unit 50 based on the equations (1) to (4), and the correlation value R _ms. Is output to the parameter calculation unit 30. After that, since it is the same as that of 1st Embodiment, description is abbreviate | omitted.

In the signal processing device 102, since the weighting unit 50 performs weighting based on the loudness curve, the balance of each sound emitted by the main sound signal and the sub sound signal in the sound finally emitted is lost. In addition, the contribution of audible sound to the correlation value R _ms increases. For example, it is assumed that the signal processing device 102 is applied to a karaoke device, the main sound signal is a vocal sound sound signal, and the sub sound signal is a drum sound sound signal. In this case, although the correlation over the entire frequency of the vocal sound and the drum sound is low, the balance between the vocal sound and the drum sound of the sound finally emitted is lost even if the correlation between both signals in the audible range is high. There is nothing. Of course, the weighting unit 50 may be used only for either the main sound signal or the sub sound signal.

<Third Embodiment>
FIG. 5 is a block diagram of a signal processing apparatus 103 according to the third embodiment of the present invention. In FIG. 5, the same components as those in FIG. 1 are denoted by the same reference numerals. As apparent from a comparison between FIG. 5 and FIG. 1, the difference between the signal processing device 103 and the signal processing device 101 is that a cross-correlation detection unit 21 and an average calculation unit 90 are provided instead of the cross-correlation detection unit 20. It is. Below, it demonstrates centering on the cross correlation detection part 21 and the average calculation part 90 which are the different points from 1st Embodiment.

τは整数値であり、Ｃ_ｍｓ（τ）は次式（７）を用いて算出される。

ｎはフレーム番号であり、１≦ｎ≦Ｔである。すなわち、ｆ_ｍ（ｎ）はメイン音響信号のｎ番目のフレームのフーリエ変換データであり、ｆ_ｓ（ｎ＋τ）はサブ音響信号のｎ＋τ番目のフレームのフーリエ変換データである。相関値Ｒ_ｍｓ（τ）は−１≦Ｒ_ｍｓ（τ）≦１を満たし、相関値Ｒ_ｍｓ（τ）が１の時にはメイン音響信号とメイン音響信号からτだけ遅れたサブ音響信号とは完全相関であり、相関値Ｒ_ｍｓ（τ）が小さくなるほどメイン音響信号とサブ音響信号の相関は低下する。相互相関検出部２１は、τ＝０、１、２の場合の相関値Ｒ_ｍｓ（τ）を平均算出部９０に出力する。なお、時間周波数変換部１０と相互相関検出部２１の間に遅延器を設け、遅延器は、時間周波数変換部１０からメイン音響信号とサブ音響信号のフーリエ変換データを入力され、相互相関検出部にメイン音響信号とメイン音響信号からτだけ遅れたサブ音響信号とのフーリエ変換データを出力する態様であってもよい。この態様では、相互相関検出部２１は、遅延器から入力されたメイン音響信号とサブ音響信号から相関値Ｒ_ｍｓ（τ）を算出し平均算出部９０に出力するだけで、サブ音響信号をメイン音響信号からτだけ遅れたようにする処理は行わない。 Based on the following equation (6), the cross-correlation detection unit 21 calculates a correlation value from Fourier transform data of the main acoustic signal input from the time-frequency conversion unit 10 and a sub acoustic signal that is a signal other than the main acoustic signal. R _ms (τ) is calculated. The correlation value R _ms (τ) is a value representing the correlation between the main acoustic signal and the sub acoustic signal delayed by τ from the main acoustic signal.

τ is an integer value, and C _ms (τ) is calculated using the following equation (7).

n is a frame number, and 1 ≦ n ≦ T. That is, f _m (n) is the Fourier transform data of the nth frame of the main acoustic signal, and f _s (n + τ) is the Fourier transform data of the n + τth frame of the sub acoustic signal. The correlation value R _ms (τ) satisfies −1 ≦ R _ms (τ) ≦ 1, and when the correlation value R _ms (τ) is 1, the main acoustic signal and the sub acoustic signal delayed by τ from the main acoustic signal are completely The correlation between the main acoustic signal and the sub acoustic signal decreases as the correlation value R _ms (τ) decreases. The cross-correlation detector 21 outputs the correlation value R _ms (τ) when τ = 0, 1, and 2 to the average calculator 90. A delay unit is provided between the time-frequency conversion unit 10 and the cross-correlation detection unit 21, and the delay unit receives the Fourier transform data of the main acoustic signal and the sub-acoustic signal from the time-frequency conversion unit 10, and the cross-correlation detection unit Alternatively, Fourier transform data of the main acoustic signal and the sub acoustic signal delayed by τ from the main acoustic signal may be output. In this aspect, the cross-correlation detection unit 21 calculates the correlation value R _ms (τ) from the main acoustic signal and the sub acoustic signal input from the delay unit, and outputs the correlation value R _ms (τ) to the average calculation unit 90. No processing is performed to delay the acoustic signal by τ.

The average calculation unit 90 calculates an arithmetic average of the correlation values R _ms (0), R _ms (1), and R _ms (2) input from the cross correlation detection unit 21, and calculates a parameter as the average correlation value R _ms. To the unit 30.

The parameter calculation unit 30 calculates the amplification factor y from the average correlation value R _ms input from the average calculation unit 90 as in the first embodiment. The subsequent steps are the same as those in the first embodiment, and the description thereof is omitted.

In the present embodiment, the main acoustic signal is amplified with an amplification factor corresponding to the arithmetic average of R _ms (τ) calculated using f _m (n) and f _s (n + τ). The time lag of the acoustic signal can be saved. More specifically, for example, when the signal processing apparatus 101 according to the first embodiment is applied to a karaoke apparatus, although there is originally a high correlation between the vocal sound and the accompaniment sound, there is a time lag between the accompaniment sound and the vocal sound. If so, the correlation between the two is judged to be low. For this reason, the balance between the vocal sound and the accompaniment sound of the sound finally emitted may be lost. On the other hand, when the signal processing device 103 of the present embodiment is applied to a karaoke device, the influence of the time lag between the accompaniment sound and the vocal sound is alleviated, and the balance between the vocal sound and the accompaniment sound of the finally emitted sound is lost. There is nothing. That is, according to the present embodiment, even if there is a time lag between the main sound signal and the sub sound signal, it is possible to perform signal processing on the main sound signal while reducing the influence of the time lag.

In this embodiment, the average correlation value R _ms is calculated from the arithmetic average of the three values of correlation values R _ms (0), R _ms (1), and R _ms (2) with τ set to 0, 1, and 2. However, τ is not limited to 0, 1, and 2. For example, the average correlation value R _ms may be calculated by setting τ to another integer value such as 0, 3, and 6, for example. Furthermore, the correlation value R _ms (τ) used for calculating the average correlation value R _ms is not limited to three values, and the number of correlation values R _ms (τ) may be increased. For example, an average correlation value from an arithmetic mean of four values of correlation values R _ms (0), R _ms (1), R _ms (2), and R _ms (3) with τ set to 0, 1, 2, 3 R _ms may be calculated. Moreover, in the present embodiment, the average calculation unit 90 uses the arithmetic average of the correlation values R _ms (τ) for calculating the average correlation value R _ms , but the largest value may be used as the average correlation value R _ms. . For example, when the value of R _ms (1) is the largest among the correlation values R _ms (0), R _ms (1), and R _ms (2) with τ set to 0, 1, and 2, R _ms (1 ) _May be calculated as the average correlation value R _ms . Of course, the calculation method of the average correlation value R _ms may be other than these, for example, a geometric average or a weighted average. What is necessary is just to determine the suitable calculation method of average correlation value _Rms according to the precision calculated _| required by the signal processing apparatus 103. FIG.

<Fourth embodiment>
FIG. 6 is a block diagram of a signal processing apparatus 104 according to the fourth embodiment of the present invention. In FIG. 6, the same components as those in FIG. 1 are denoted by the same reference numerals. 6 and 1, the difference between the signal processing device 104 and the signal processing device 101 is that the parameter calculation unit 31, the multiplication unit 41, and the parameter calculation unit 30 are replaced with the parameter calculation unit 30 and the multiplication unit 40. The reverberation signal generator 60 and the adder 70 are provided. Below, it demonstrates centering on the parameter calculation part 31, the multiplication part 41, the reverberation signal production | generation part 60, and the addition part 70 which are the different points from 1st Embodiment. The multiplication unit 41, the reverberation signal generation unit 60, and the addition unit 70 function as a signal processing unit that gives a reverb effect to the main acoustic signal.

  In the first embodiment, the main acoustic signal input to the signal processing device 101 is input to the time-frequency conversion unit 10 and the multiplication unit 40. However, in the signal processing device 104 of the present embodiment, the main acoustic signal is converted to the time-frequency conversion. Input to unit 10, reverberation signal generation unit 60, and addition unit 70. In the present embodiment, the main acoustic signal input to the reverberation signal generation unit 60 is referred to as a Wet signal, and the main acoustic signal input to the addition unit 70 is referred to as a Dry signal.

  The reverberation signal generation unit 60 generates a reverberation signal based on the Wet signal and outputs the reverberation signal to the multiplication unit 41. The existing reverberation signal generation algorithm of the reverberation signal generation unit 60 may be used. By adding a reverberation signal to the Dry signal, that is, by adding a reverb effect, the signal after the addition becomes an acoustic signal indicating a sound with a sense of depth in terms of hearing.

ｘ_０は閾値であり、図７は、式（８）をグラフ化した図である。ノーマライズ値ｘがｘ_０よりも小さいとミックス値ｇ_ｍｉｘはａｘ＋ｂとなり、ノーマライズ値が増加するに連れてミックス値ｇ_ｍｉｘは線形に大きくなる。ノーマライズ値ｘがｘ_０以上であるとミックス値ｇ_ｍｉｘは一定値ｃ（ｃ＝ａｘ_０＋ｂ）となる。なお、ミックス値ｇ_ｍｉｘがノーマライズ値が増加するにつれて非線形に大きくなってもよく、実験により好適な態様を取ればよい。 The parameter calculation unit 31 calculates a normalized value x from the correlation value R _ms input from the cross correlation detection unit 20 based on Expression (3), and outputs it to the multiplication unit 41. 0 ≦ x ≦ 1 is as described above. The parameter calculation unit 31 calculates the mix value g _mix based on the following equation (8). The mix value g _mix is a value indicating the strength of amplification processing applied to the signal level of the reverberation signal, and corresponds to the mixing ratio of the reverberation signal to the Dry signal.

x ₀ is the threshold, Fig. 7 is a diagram showing a graph of the equation (8). Normalize value x is small, the mix value _{g mix} than _{x 0} Mix value _{g mix} him to ax + b, and the is normalized value increases increases linearly. Normalize value x _x Mixed value _{g mix} When it is ₀ or more becomes a constant value _{c (c = ax 0 + b} ). Note that the mix value g _mix may increase non-linearly as the normalization value increases, and a suitable aspect may be taken by experiment.

The multiplication unit 41 receives an input of a reverberation signal from the reverberation signal generation unit 60 and further receives an input of a mix value g _mix from the parameter calculation unit 31. The multiplier 41 amplifies the signal level of the reverberation signal based on the mix value g _mix and outputs the amplified signal level to the adder 70.

  The adder 70 adds the Dry signal and the reverberation signal amplified by the multiplier 41 and outputs the result as an output signal.

When the correlation between the main sound signal and the sub sound signal is high, if the reverb effect added to the main sound signal is too strong, the reverberant sound becomes noticeable, and the sound finally emitted by the main sound signal and the sub sound signal The balance will be lost. If the signal processing apparatus 104, when normalized value x is larger than the threshold value x ₀ is the c-mix value g _mix is constant value. If the value of c is set to a value such that the reverberant sound is inconspicuous by experiment or the like, even if the correlation between the main acoustic signal and the sub-acoustic signal is high, the reverberation effect so that the reverberant sound is conspicuous is given. Therefore, the balance of each sound emitted by the main sound signal and the sub sound signal in the sound finally emitted is not lost.

<Fifth Embodiment>
The signal processing device 105 of this embodiment is a combination of the signal processing device 103 and the signal processing device 104. More specifically, the signal processing device 105 is provided with a cross-correlation detection unit 21 and an average calculation unit 90 of the signal processing device 103 instead of the cross-correlation detection unit 20 of the signal processing device 104 with respect to the signal processing device 104. It corresponds to.

In the signal processing device 105, the cross-correlation detection unit 21 and the average calculation unit 90 calculate the average correlation value R _ms from the arithmetic average of the correlation values R _ms (τ) when τ = 0, 1, and 2, Determines the strength of the reverb effect. Therefore, when the signal processing device 105 is used, even if there is a time lag between the main sound signal and the sub sound signal that is a signal other than the main sound signal, the sound that is finally emitted is the main sound signal. In addition, the reverb effect with which the sub-acoustic signal is well-rounded is applied, and the balance between the sounds emitted by the main acoustic signal and the sub-acoustic signal is not lost by the reverb effect.

Also in the present embodiment, as in the third embodiment, τ is not limited to 0, 1, and 2, and the number of τ used to calculate the average correlation value R _ms is not limited to three. Furthermore, not only an arithmetic mean is used as a method for calculating the average correlation value R _ms from a plurality of correlation values R _ms (τ), but other calculation methods may be used. In short, the average correlation value R _ms can be calculated by any method as long as the main sound signal and sub-acoustic signal can be given a good reverb effect to the sound finally emitted. Good.

<Sixth Embodiment>
FIG. 8 is a block diagram of a signal processing device 106 according to the sixth embodiment of the present invention. In FIG. 8, the same components as those in FIG. 1 are denoted by the same reference numerals. As apparent from a comparison between FIG. 8 and FIG. 1, the difference between the signal processing device 106 and the signal processing device 101 is that a high-pass filter (hereinafter, HPF) 81, a band-pass filter (hereinafter, “HPF”) 81 , BPF) 82 and a low-pass filter (hereinafter referred to as LPF) 83, and an addition unit 71 subsequent to the multiplication unit 40. Below, it demonstrates centering on HPF81, BPF82, LPF83, and the addition part 71 which are the different points from 1st Embodiment.

The HPF 81 is a filter that outputs only a high frequency frequency band that is equal to or higher than the frequency f _HL of the input signal, and the BPF 82 is a filter that outputs only an intermediate frequency band from the frequency f _BL to the frequency f _BH (frequency f _BL <frequency f _BH ), LPF 83 is a filter that outputs only a low frequency band of frequency f _LH or lower. In this embodiment, frequency f _HL = frequency f _BH and frequency f _BL = frequency f _LH , but the frequencies may not be equal. What is necessary is just to determine suitably the value between each frequency value and each frequency by experiment. The HPF 81, BPF 82, and LPF 83 function as a band dividing unit that performs band division processing on the main acoustic signal and the sub acoustic signal.

  When the main sound signal and the sub sound signal are input to the signal processing device 106, first, the main sound signal and the sub sound signal are input to the HPF 81, the BPF 82, and the LPF 83. For the main acoustic signal, the HPF 81, the BPF 82, and the LPF 83 output a signal of only a frequency band corresponding to each filter to the time frequency conversion unit 10 and the multiplication unit 40. For the sub-acoustic signal, the HPF 81, the BPF 82, and the LPF 83 output a signal of only a frequency band corresponding to each filter to the time frequency conversion unit 10.

  After the HPF 81, the BPF 82, and the LPF 83 output the main sound signal and the sub sound signal to the time frequency conversion unit 10 and output the main sound signal to the multiplication unit 40, the high frequency band, the intermediate frequency band, and the low frequency band Since the same processing as in the first embodiment is performed, the description is omitted. The multiplication unit 40 outputs a main acoustic signal obtained by amplifying the signal level for each of the high frequency band, the intermediate frequency band, and the low frequency band to the adding unit 71.

  The adder 71 adds the main acoustic signal whose signal level is amplified for each frequency band and outputs the result as an output signal.

As described above, the signal processing device 106 performs band division processing on the main acoustic signal and the sub acoustic signal using the HPF 81, the BPF 82, and the LPF 83, and calculates a correlation value R _ms between the main acoustic signal and the sub acoustic signal for each frequency band. Then, the signal level of the main acoustic signal is amplified for each frequency band based on the correlation value R _ms and finally added. According to the signal processing device 106, the amplification control of the main acoustic signal can be finely performed for each frequency band in accordance with the correlation with the sub acoustic signal for each frequency band.

<Seventh embodiment>
The difference between the signal processing device 107 and the signal processing device 101 of this embodiment is that an HPF 81 and a BPF 82 are provided in the previous stage of the time-frequency conversion unit 10. In the signal processing device 107, the main acoustic signal is input to the time-frequency conversion unit 10 via the BPF 82 and also input to the multiplication unit 40, and the sub-acoustic signal is input to the time-frequency conversion unit 10 via the HPF 81.

In the present embodiment, since the Fourier transform and the correlation value R _ms are calculated for the acoustic signal band-divided by the HPF 81 and the BPF 82, the main acoustic signal and the sub acoustic signal in the sound finally emitted are emitted. The balance of each sound to be sounded is not lost, and the processing load of the device can be reduced as compared with the signal processing device 106.

  In the signal processing device 107, the BPF 82 is used for the main sound signal, and the HPF 81 is used for the sub sound signal. However, the signal processing device 107 is provided with HPF 81, BPF 82, and LPF 83, and the HPF 81, BPF 82, and LPF 83 are arranged according to the frequency band where the main acoustic signal and the sub acoustic signal input to the signal processing device 107 are concentrated. Different filters may be selected. Of course, an aspect in which any one of the HPF 81, the BPF 82, and the LPF 83 is used for only one of the main sound signal and the sub sound signal.

<Eighth Embodiment>
FIG. 9 is a block diagram of a signal processing apparatus 108 according to the eighth embodiment of the present invention. The signal processing device 108 is applied to, for example, a karaoke device. In FIG. 9, the same components as those in FIG. 1 are denoted by the same reference numerals. As is clear from comparison between FIG. 9 and FIG. 1, the difference between the signal processing device 108 and the signal processing device 101 is that the time-frequency conversion unit 10 receives a vocal sound signal and an accompaniment sound signal as sub-acoustic signals. This is a point where a composite signal (hereinafter referred to as a composite signal) of a vocal sound and an accompaniment sound signal is input to the multiplier 40 as a main sound signal.

In the signal processing device 108, the parameter calculation unit 30 calculates the amplification factor y based on the correlation value R _ms between the vocal sound and the accompaniment sound. The multiplier 40 performs amplification processing on the combined signal based on the amplification factor y.

Similar to the case where the vocal sound signal is amplified by separating the sound signal from the synthesized signal by directly amplifying the synthesized signal instead of the vocal sound signal as in this embodiment. The effect can be expected. That is, the output signal of this embodiment is obtained by separating the synthesized signal into a vocal sound signal and an accompaniment sound signal, and calculating a correlation value R _ms between the separated vocal sound signal and the accompaniment sound signal. Thus, the output signal is similar to an output signal that is amplified by a vocal sound signal and output. In the method using sound source separation, artificial noise may occur during sound source separation, and the sound quality after sound source separation often becomes a problem. In the present embodiment, even if the vocal sound signal and the accompaniment sound signal input to the signal processing device 108 are generated by subjecting the synthesized signal to sound source separation, they are subjected to amplification processing. Since the synthesized signal is not generated by sound source separation, there is an advantage that the balance between the vocal sound and the accompaniment sound is not lost, and the sound quality of the output signal does not easily depend on the performance of the sound source separation.

In the signal processing device 108, the synthesized signal is composed only of the acoustic signal input as the sub-acoustic signal. However, the present invention is not limited to this mode, and the synthesized signal is included in the acoustic signal input as the sub-acoustic signal. The aspect comprised from the at least 1 acoustic signal and the other acoustic signal may be sufficient. In this aspect, for example, the sub sound signal is a vocal sound sound signal and a drum sound sound signal, and the main sound signal is a composite signal of a drum sound and a guitar sound sound signal. Furthermore, when three or more types of acoustic signals are input as sub acoustic signals, the cross correlation detection unit 20 may select an acoustic signal for calculating the correlation value R _ms from the acoustic signals. In this aspect, for example, when a vocal sound, a drum sound, and a guitar sound acoustic signal are input as sub-acoustic signals, the cross-correlation detector 20 detects the vocal sound acoustic signal and the drum sound acoustic from the input acoustic signal. A signal is selected, and a correlation value R _ms of both signals is calculated. Further, this selection may be switched according to a user instruction. The main acoustic signal in this case may be a synthesized signal of vocal sound and drum sound, or may be a synthesized signal of drum sound and guitar sound. Of course, the main sound signal may be a composite signal of vocal sound, drum sound and guitar sound.

<Ninth Embodiment>
The signal processing device 109 of the present embodiment is also applied to, for example, a karaoke device. The difference between the signal processing device 109 and the signal processing device 101 is that a synthesized signal representing a time waveform of a synthesized sound of a vocal sound and an accompaniment sound is input to the time frequency conversion unit 10 and the multiplication unit 40 as a main acoustic signal, and the sub acoustic signal Is that the acoustic signal of the accompaniment sound is input to the time-frequency converter 10.

In the signal control device 109, the amplification factor y is calculated based on the correlation value R _ms between the accompaniment sound signal and the synthesized signal, and the synthesized signal is subjected to amplification processing based on the amplification factor y.

  In the present embodiment, if there is an input of an acoustic signal of an accompaniment sound and a composite signal without an input of an acoustic signal of a vocal sound, the composite signal is subjected to amplification processing without breaking the balance of the vocal sound and the accompaniment sound. be able to.

<Tenth Embodiment>
FIG. 10 is a block diagram of a signal processing apparatus 110 according to the tenth embodiment. The signal processing device 110 is also applied to, for example, a karaoke device. 10, the same components as those in FIG. 1 are denoted by the same reference numerals. As is apparent from a comparison between FIG. 10 and FIG. 1, the difference between the signal processing device 110 and the signal processing device 101 is that a plurality of types of sub-acoustic signals are input to the time frequency conversion unit 10 and the average calculation unit 90. Is a point provided. In the present embodiment, there are N types (N ≧ 2) of types of sub-acoustic signals input to the time-frequency conversion unit 10. Specific examples of the N types of sub sound signals include sound signals of accompaniment sounds for each accompaniment instrument such as drum sounds, guitar sounds, and keyboard sounds.

The time frequency conversion unit 10 outputs the Fourier transform data of the main sound signal and the N types of sub sound signals to the cross correlation detection unit 20. The cross correlation detection unit 20 detects the correlation between the main acoustic signal input from the time frequency conversion unit 10 and the N types of sub acoustic signals. More specifically, the cross-correlation detection unit 20 calculates a correlation value R _ms1 between the main acoustic signal and the sub acoustic signal 1 based on the equations (1) and (2), and the main acoustic signal and the sub acoustic signal. 2 correlation value R _ms2 is calculated. Similarly, the cross correlation detection unit 20 calculates up to the correlation value R _msN between the main acoustic signal and the sub acoustic signal N. The cross-correlation detection unit 20 outputs the calculated correlation values R _{ms1 to} R _msN to the average calculation unit 90. The average calculator 90 calculates the arithmetic average of the input correlation values R _{ms1 to} R _msN as the average correlation value R _ms . The average calculation unit 90 outputs the calculated average correlation value R _ms to the parameter calculation unit 30, and the multiplication unit 40 performs amplification processing on the main acoustic signal based on the amplification factor y calculated by the parameter calculation unit 30, Output as an output signal.

In the signal processing device 110, an average correlation value R _ms between all the sub-acoustic signals and the main acoustic signal is calculated, and the main acoustic signal is amplified at an amplification factor corresponding to the average correlation value R _ms . For this reason, the balance between the main sound signal and the plurality of sub sound signals in the sound finally emitted is not lost.

<Modification>
(1) The aspect which combined each said embodiment may be taken. For example, the seventh embodiment and the tenth embodiment may be combined. In this case, even if there are a plurality of input sub sound signals, the main sound signal is not lost in the balance between the main sound signal in the sound finally emitted and the sounds emitted by the plurality of sub sound signals. Can be amplified. Furthermore, in this case, there is only one sub acoustic signal, that is, the processing load can be reduced more than the processing load obtained by multiplying the processing load of the apparatus of the first embodiment by the number of sub acoustic signals.

(2) In each of the above embodiments, the main sound signal is subjected to amplification processing or reverberation processing, but the signal processing applied to the main sound signal is not limited to these processing, and other sound effects are also provided. May be applied to the main acoustic signal.

(3) In each of the embodiments described above, the time-frequency conversion unit 10 performs Fourier transform by dividing the two types of digital signals, the main sound signal and the sub-acoustic signal, that are received into frames of a certain time length, and performs Fourier transform. Although the calculated Fourier transform data is output to the cross-correlation detecting unit 20, an aspect in which the time-frequency converting unit 10 does not exist may be employed. In this aspect, the input main acoustic signal and sub-acoustic signal are directly input to the cross-correlation detection unit 20, and the cross-correlation detection unit 20 uses the formula (1) or the formula (6), and the main acoustic signal and the sub-acoustic signal. Based on the signal, the correlation value R _ms or the correlation value R _ms (τ) is calculated. In this case, τ is an index indicating each sample constituting the acoustic signal. Also according to this aspect, it is possible to compress the dynamic range of the acoustic signal and to apply the reverb effect to the acoustic signal while maintaining the balance of the entire sound emitted by the main acoustic signal and the sub acoustic signal.

(4) In the above embodiments, both the main sound signal and the sub sound signal are digital signals, but one (or both) may be analog signals. When an analog signal is input to the signal processing device of each embodiment, an A / D converter may be provided in the previous stage of the signal processing device.

(5) In the above embodiments, functions specific to the signal processing apparatus of the present invention are realized by a program (DSP microprogram). However, the program may be provided alone. For example, by adding the program of any of the above embodiments to the existing program of the karaoke apparatus, the acoustic signal of the vocal sound is maintained without breaking the balance between the vocal sound of the sound finally emitted and the accompaniment sound. Dynamic range compression and reverb effect can be applied.

(6) The amplification processing or the reverberation effect according to each of the above embodiments may be provided by an ASP (Application Service Provider) type communication service. For example, when the amplification processing of the first embodiment is provided in the ASP format, the signal processing device 101 is connected to a communication line. Then, the main acoustic signal and the sub acoustic signal are input to the signal processing device 101 via the communication line, and the signal processing device 101 outputs the main acoustic signal that has been subjected to amplification processing based on the correlation between the main acoustic signal and the sub acoustic signal. Output as a signal, and return the output signal via the communication line.

101, 102, 103, 104, 105, 106, 107, 108, 109, 110 ... signal processing device, 10 ... time frequency converter, 20, 21 ... cross-correlation detector, 30, 31 ... parameter calculation , 40, 41... Multiplying unit, 50... Weighting unit, 60... Reverberation signal generation unit, 70, 71 ...... Addition unit, 81 ...... HPF, 82 ...... BPF, 83 ...... LPF, 90 ...... Average calculator.

Claims (4)

A cross-correlation detector that calculates a correlation value between a plurality of signals among a main acoustic signal to be subjected to signal processing that gives an acoustic effect and one or a plurality of sub-acoustic signals;
A signal processing unit that determines the intensity of the acoustic effect according to the correlation value calculated in the cross-correlation detection unit;
A signal processing apparatus comprising:   The signal processing apparatus according to claim 1, further comprising a weighting unit that weights an audible range of at least one of the plurality of signals and applies the audible range to the cross-correlation detection unit.   The cross-correlation detection unit calculates a plurality of correlation values between one signal of the plurality of signals and one or more signals of the plurality of signals that are different in time from the signal, and the plurality of correlations 3. The signal processing apparatus according to claim 1, wherein one correlation value is obtained from the value. A band dividing unit that performs band division on at least one of the main acoustic signal and the one or more sub-acoustic signals;
The said cross correlation detection part calculates the said correlation value using at least 1 of the signal of the zone | band divided | segmented by the said band division part. The claim of any one of Claims 1-3 characterized by the above-mentioned. A signal processing device according to 1.

Images