JP4504892B2 - Echo canceling method, echo canceling apparatus, program, recording medium - Google Patents
Echo canceling method, echo canceling apparatus, program, recording medium Download PDFInfo
- Publication number
- JP4504892B2 JP4504892B2 JP2005251648A JP2005251648A JP4504892B2 JP 4504892 B2 JP4504892 B2 JP 4504892B2 JP 2005251648 A JP2005251648 A JP 2005251648A JP 2005251648 A JP2005251648 A JP 2005251648A JP 4504892 B2 JP4504892 B2 JP 4504892B2
- Authority
- JP
- Japan
- Prior art keywords
- signal
- power
- estimated value
- value
- echo
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Circuit For Audible Band Transducer (AREA)
- Cable Transmission Systems, Equalization Of Radio And Reduction Of Echo (AREA)
Abstract
Description
本発明は、ステレオ音声などを再生可能な2つのスピーカから一つ以上のマイクロホンへと回り込む反響を消去するための反響消去方法、反響消去装置、反響消去プログラム、記録媒体に関する。 The present invention relates to an echo canceling method, an echo canceling device, an echo canceling program, and a recording medium for canceling echo that circulates from two speakers capable of reproducing stereo sound and the like to one or more microphones.
ステレオ音声などを再生可能な2つのスピーカから少なくとも一つ以上のマイクロホンへ回り込む反響を消去する反響消去装置は、図2のように接続される。図2では、マイクロホンは一つの場合を挙げているが、複数のマイクロホンが存在する場合も各マイクロホン毎に同様な構成がとられる。従来、反響消去装置100内では、第一スピーカ1とマイクロホン3間の第一反響路4のインパルス応答に相当する長さLのベクトルH1を模擬する第一模擬特性Hm1(k)を保持する第一適応型模擬反響路101と、第二スピーカ2とマイクロホン3間の第二反響路5のインパルス応答に相当する長さLのベクトルH2を模擬する第二模擬特性Hm2(k)を保持する第二適応型模擬反響路102を有する。ここで、kは、所定間隔の離散時間を指すステップ数である。第一の再生信号x1(k)および第二の再生信号x2(k)は、それぞれ、第一反響路4および第二反響路5の各ベクトルH1とH2との畳込演算により第一反響信号d1(k)および第二反響信号d2(k)として、マイクロホン3において混合され収音される。
An echo canceling device that cancels the echo that circulates from at least one microphone to two speakers capable of reproducing stereo sound or the like is connected as shown in FIG. Although FIG. 2 shows a case where there is one microphone, the same configuration is adopted for each microphone even when there are a plurality of microphones. Conventionally, in the echo canceling apparatus 100, a first simulation characteristic Hm1 (k) that simulates a vector H1 of length L corresponding to the impulse response of the first echo path 4 between the
このとき、第一適応型模擬反響路101と第二適応型模擬反響路102とは、それぞれ、第一の再生信号x1(k)および第二の再生信号x2(k)を入力として、第一模擬特性Hm1(k)および第二模擬特性Hm2(k)との畳込演算により第一模擬反響信号dm1(k)および第二模擬反響信号dm2(k)を生成し、加算器103において加算した後、減算器104において、第一反響信号d1(k)および第二反響信号d2(k)を含むマイクロホン3の収音信号y(k)から減算することで、反響消去装置の出力信号でもある誤差信号e(k)を計算する。なお、加算器103と減算器104の機能を統合した誤差算出手段304で同様な処理を行ってもよい。
At this time, the first adaptive simulated
さて、第一適応型模擬反響路101および第二適応型模擬反響路102において、第一模擬特性Hm1(k)および第二模擬特性Hm2(k)は、それぞれ、
y(k)=d1(k)+d2(k)であり、d1(k)とd2(k)を個別に観測することは不可能である。
In the first adaptive simulated
y (k) = d1 (k) + d2 (k), and d1 (k) and d2 (k) cannot be observed individually.
実際には、図2の例では、誤差信号e(k)=y(k)−[dm1(k)+dm2(k)]に基づき推定値を与えている。すなわち、信号電力加重誤差配分手段105において、e1(k)の推定値として、
また、図3の例では、均等誤差配分手段205により、e1(k)の推定値として、
em1(k)=(1/2)e(k) (式5)
e2(k)の推定値として、
em2(k)=(1/2)e(k) (式6)
とe(k)を均等に配分し、(式1)および(式2)に適用する形で、第一模擬特性Hm1(k)および第二模擬特性Hm2(k)の特性更新を行っている。このような特性更新は、非特許文献1記載の「正規化法を用いた線形結合型多チャンネル適応フィルタ」の2チャンネルの場合の特性更新に相当する。
Further, in the example of FIG. 3, the equal error distribution means 205 uses the estimated value of e1 (k) as
em1 (k) = (1/2) e (k) (Formula 5)
As an estimate of e2 (k),
em2 (k) = (1/2) e (k) (Formula 6)
And e (k) are evenly distributed, and the characteristics of the first simulated characteristic Hm1 (k) and the second simulated characteristic Hm2 (k) are updated in a form that is applied to (Expression 1) and (Expression 2). . Such a characteristic update corresponds to a characteristic update in the case of two channels of “a linearly coupled multi-channel adaptive filter using a normalization method” described in Non-Patent
ここで、図2および図3に示した従来技術において、(式1)および(式2)で本来用いられるべきe1(k)、e2(k)に対して実際に適用される第一推定値および第二推定値em1(k)、em2(k)の精度について考える。まず、
e1(k)=ec(k)+ea(k) (式7)
e2(k)=ec(k)−ea(k) (式8)
とおく。ただし、
em1(k)=ec(k)+ema(k) (式9)
em2(k)=ec(k)−ema(k) (式10)
と表すことを考える。
Here, in the prior art shown in FIGS. 2 and 3, the first estimated value actually applied to e1 (k) and e2 (k) that should be originally used in (Expression 1) and (Expression 2). Consider the accuracy of the second estimated values em1 (k) and em2 (k). First,
e1 (k) = ec (k) + ea (k) (Formula 7)
e2 (k) = ec (k) −ea (k) (Formula 8)
far. However,
em1 (k) = ec (k) + ema (k) (Formula 9)
em2 (k) = ec (k) −ema (k) (Formula 10)
Think of the expression.
このとき、(式3)、(式4)に基づく図2の構成では、(式9)、(式10)において、
一方、(式5)、(式6)に基づく図3の構成では(式9)、(式10)において、
ema(k)=0 (式12)
と与えたことに相当する。(式12)では、ema(k)の平均値を零と仮定し、eu(k)の大きさ、すなわちαの値によらない反響路の模擬を実現している。
On the other hand, in the configuration of FIG. 3 based on (Equation 5) and (Equation 6), in (Equation 9) and (Equation 10),
ema (k) = 0 (Formula 12)
It is equivalent to giving. In (Expression 12), the average value of ema (k) is assumed to be zero, and the echo path is simulated regardless of the size of eu (k), that is, the value of α.
背景技術でのe1(k)、e2(k)に対する推定値em1(k)、em2(k)に関する考察より、図2の構成においては、αの大きさに対する、推定精度の依存性が強く、図3の構成においては、αの大きさによらず、固定的な誤差配分により、推定を実現している。図2の構成の推定精度は、αの値に依存して、図3の構成より優れる場合もあるが、劣る場合もある。
本発明の目的は、図2の構成の推定値em1(k)、em2(k)について、αの値に依存した精度劣化を抑え、反響路の模擬精度を高めることにより、反響消去性能の優れた反響消去方法及び装置を提案することにある。
From the consideration of the estimated values em1 (k) and em2 (k) for e1 (k) and e2 (k) in the background art, in the configuration of FIG. 2, the dependence of the estimation accuracy on the magnitude of α is strong. In the configuration of FIG. 3, estimation is realized by fixed error distribution regardless of the magnitude of α. The estimation accuracy of the configuration of FIG. 2 may be better than the configuration of FIG. 3 depending on the value of α, but may be inferior.
The object of the present invention is to improve the echo canceling performance by suppressing the accuracy deterioration depending on the value of α for the estimated values em1 (k) and em2 (k) of the configuration of FIG. 2 and increasing the simulation accuracy of the echo path. Another object is to propose an echo canceling method and apparatus.
本発明による反響消去装置は同一空間内に存在する第一の音響再生手段と音響収音手段との間の音響的な伝達経路である第一の反響路を介して第一の音響再生手段から再生する第一の再生信号が音響収音手段により収音される第一の反響信号と、第一の音響再生手段の同一の空間内に存在する第二の音響再生手段と音響収音手段との間の音響的な伝達経路である第二の反響路を介して、第二の音響再生手段から再生する第二の再生信号が音響収音手段により第一の反響信号と混合して収音される第二の反響信号とを、音響収音手段で収音する全ての収音信号の中から消去または低減する反響消去装置において、第一の反響路の模擬特性を有し、第一の再生信号を入力し、第一の反響信号を模擬する第一の反響模擬信号を出力する第一の適応型模擬反響路と、第二の反響路の模擬特性を有し、第二の再生信号を入力し、第二の反響信号を模擬する第二の反響模擬信号を出力する第二の適応型模擬反響路と、第一の反響模擬信号および第二の反響模擬信号を収音信号から差引き誤差信号を取得し、この誤差信号を出力する誤差算出手段と、及び誤差信号を均等割合で配分して得られる第一の推定値と第二の推定値を出力する均等誤差配分手段か、誤差信号を第一の再生信号の電力と第二の再生信号の電力との比に応じて配分して得られる第三の推定値と第四の推定値を出力する信号電力加重誤差配分手段との何れかを備えた反響消去装置は上記したように従来より存在するが、
本発明の特徴とする構成はこれら均等誤差配分手段と、信号電力加重誤差配分手段の双方を備える点と、均等誤差配分手段は誤差信号を均等割合で配分して得られる第一の推定値と第二の推定値とを出力する点と、信号電力加重誤差配分手段は第一の再生信号の電力と第二の再生信号の電力との比に応じて配分して得られる第三の推定値と第四の推定値を出力する点を特徴とし、更に第一の再生信号の電力と第二の再生信号の電力の大小関係に応じて異なる値を電力偏差として出力する電力偏差計算手段と、
電力偏差の値が、第一の再生信号の電力が第二の再生信号の電力より大きいことを示すほど、第三の推定値を第一の推定値よりも大きな割合で加重平均し、電力偏差の値が、第二の再生信号の電力が第一の再生信号の電力より大きいことを示すほど、第一の推定値を第三の推定値よりも大きな割合で加重平均し、第五の推定値として出力し、第一の適応型模擬反響路に与え、第一の適応型模擬反響路が有する模擬特性を更新させる第一の電力偏差加重平均手段と、電力偏差の値が、第一の再生信号の電力が第二の再生信号の電力より大きいことを示すほど、第二の推定値を第四の推定値よりも大きな割合で加重平均し、電力偏差の値が、第二の再生信号の電力が第一の再生信号の電力より大きいことを示すほど、第四の推定値を第二の推定値よりも大きな割合で加重平均し、第六の推定値として出力し、第二の適応型模擬反響路に与え、第二の適応型模擬反響路が有する模擬特性を更新させる第二の電力偏差加重平均手段とを備える構成とした点を特徴とする。
The echo canceling apparatus according to the present invention is connected to the first sound reproducing means via the first echo path which is an acoustic transmission path between the first sound reproducing means and the sound collecting means existing in the same space. A first reverberation signal in which a first reproduction signal to be reproduced is picked up by the sound pickup means; a second sound reproduction means and a sound pickup means that exist in the same space of the first sound reproduction means; The second reproduction signal reproduced from the second sound reproducing means is mixed with the first echo signal by the sound collecting means via the second echo path which is an acoustic transmission path between the two. In the echo canceling device for canceling or reducing the second echo signal to be collected from all the collected sound signals picked up by the sound pickup means, the first echo path has a simulation characteristic, A first adaptive type that inputs a playback signal and outputs a first echo simulation signal that simulates the first echo signal A second adaptive simulated echo that has simulated characteristics of a pseudo echo path and a second echo path, inputs a second reproduction signal, and outputs a second echo simulation signal that simulates the second echo signal. A path, a first echo simulation signal, and a second echo simulation signal are obtained by subtracting an error signal from the collected signal, and an error calculation means for outputting the error signal, and the error signal is distributed at an equal ratio. An equal error distribution means for outputting the first estimated value and the second estimated value obtained, or by distributing the error signal according to the ratio of the power of the first reproduction signal and the power of the second reproduction signal. As described above, there is a conventional echo canceller including any one of the third estimated value and the signal power weighted error distribution means for outputting the fourth estimated value.
The characteristic feature of the present invention is that both of the equal error distribution means and the signal power weighted error distribution means are provided, and the equal error distribution means has a first estimated value obtained by distributing the error signal at an equal ratio. The point of outputting the second estimated value and the third estimated value obtained by distributing the signal power weighted error distribution means according to the ratio of the power of the first reproduced signal and the power of the second reproduced signal And a fourth estimated value, and further, a power deviation calculating means for outputting a value different as a power deviation according to the magnitude relationship between the power of the first reproduced signal and the power of the second reproduced signal;
As the power deviation value indicates that the power of the first reproduced signal is greater than the power of the second reproduced signal, the third estimated value is weighted and averaged at a rate greater than the first estimated value. As the value of indicates that the power of the second reproduced signal is greater than the power of the first reproduced signal, the first estimated value is weighted average at a rate greater than the third estimated value, and the fifth estimated Output as a value, given to the first adaptive simulated echo path, the first power deviation weighted average means for updating the simulation characteristics of the first adaptive simulated echo path, the value of the power deviation is the first The second estimated value is weighted and averaged at a rate greater than the fourth estimated value, indicating that the power of the reproduced signal is greater than the power of the second reproduced signal, and the power deviation value is the second reproduced signal. The fourth estimate is the second estimate as the power of is greater than the power of the first playback signal. The second power deviation weight is weighted average with a larger ratio, output as a sixth estimated value, given to the second adaptive simulated echo path, and updating the simulated characteristics of the second adaptive simulated echo path It is characterized by having a configuration comprising an averaging means.
本発明による反響消去装置は更に、電力偏差計算手段において、電力偏差として、第一の再生信号の電力と第二の再生信号の電力の差を、第一の再生信号の電力と第二の再生信号の和で除した値を用いることを特徴とする。
本発明による反響消去装置は更に、第一の電力偏差加重平均手段において、第五の推定値を求める第一の推定値と第三の推定値の加重平均の値を、1から電力偏差を引いた値の二分の一と第一の推定値との積と、1に電力偏差を加えた値の二分の一と第三の推定値との積と、の和として得ることを特徴とする。
The echo canceling apparatus according to the present invention further includes, in the power deviation calculation means, the difference between the power of the first reproduction signal and the power of the second reproduction signal as the power deviation, and the power of the first reproduction signal and the second reproduction signal. A value obtained by dividing the sum of signals is used.
The echo canceling apparatus according to the present invention further subtracts the power deviation from 1 for the weighted average value of the first estimated value and the third estimated value for obtaining the fifth estimated value in the first power deviation weighted average means. The product of a half of the obtained value and the first estimated value and the product of the half of the value obtained by adding 1 to the power deviation and the third estimated value are obtained.
本発明による反響消去装置は更に、第二の電力偏差加重平均手段において、第六の推定値を求める第二の推定値と第四の推定値の加重平均の値を、1に電力偏差を加えた値の二分の一と第二の推定値との積と、1から電力偏差を引いた値の二分の一と第四の推定値との積と、の和として得ることを特徴とする。
更に具体的な説明をするならば、本発明では、背景技術で説明した図2の誤差配分と図3の誤差配分を、αの値に応じて、組み合わせることにより、前記課題を解決する。図2の構成において、ema(k)の推定精度が高くなるのは、eu(k)が十分小さい、すなわち、αが、1、または、−1に近いときである。さらに、(式3)、(式4)を、αを用いて表すと、
More specifically, in the present invention, the above problem is solved by combining the error distribution of FIG. 2 described in the background art and the error distribution of FIG. 3 according to the value of α. In the configuration of FIG. 2, the estimation accuracy of ema (k) is high when eu (k) is sufficiently small, that is, when α is close to 1 or −1. Furthermore, when (Equation 3) and (Equation 4) are expressed using α,
そこで、0以上1以下の値を取る変数γ1に対して、第一の再生信号ベクトルX1(k)の大きさが第二の再生信号ベクトルX2(k)の大きさより大きくなるほど、小さい値を与え、0以上1以下の値を取る変数γ2に対して、第一の再生信号ベクトルX1(k)の大きさが第二の再生信号ベクトルX2(k)の大きさよりも小さくなるほど、小さい値を与えるものとして、以下のように組合せて得られる第五推定値em5(k)、及び第六の推定値em6(k)を、本発明では用いる。
なお、本発明では、第一の再生信号の大きさと第二の再生信号の大きさの大小関係を表す量を電力偏差と呼び、例えば、前述のαを電力偏差として用いる。
In the present invention, an amount representing the magnitude relationship between the magnitude of the first reproduction signal and the magnitude of the second reproduction signal is called a power deviation, and for example, the aforementioned α is used as the power deviation.
本発明による反響消去装置は、(式1)、(式2)の中でe1(k)、e2(k)の代わりに適用する第五の推定値em5(k)と第六の推定値em6(k)の算出において、従来の(式3)、(式4)による推定値、および、従来の(式5)、(式6)による推定値それぞれの長所を活かすため、それらの貢献度を、eu(k)のパワー期待値と、e1(k)からeu(k)を除いた成分とパワー期待値、および、e2(k)から−eu(k)を除いた成分とのパワー期待値それぞれとの比に応じ、上記2種類の従来の推定値の貢献度を制御する手段を設けたため、従来の推定値よりも精度を高めることができる。このため、第一模擬特性Hm1(k)および第二模擬特性Hm2(k)の模擬精度を従来の二つの技術より高めることができる。 The echo canceling apparatus according to the present invention includes a fifth estimated value em5 (k) and a sixth estimated value em6 applied in place of e1 (k) and e2 (k) in (Expression 1) and (Expression 2). In the calculation of (k), in order to make use of the advantages of the conventional estimated values of (Equation 3) and (Equation 4) and the estimated values of the conventional (Equation 5) and (Equation 6), , The expected power value of eu (k), the expected power value of the component excluding eu (k) from e1 (k), and the expected power value of the component excluding −eu (k) from e2 (k) Since the means for controlling the contribution of the two types of conventional estimated values is provided in accordance with the ratio with each of them, the accuracy can be improved more than the conventional estimated values. For this reason, the simulation accuracy of the first simulation characteristic Hm1 (k) and the second simulation characteristic Hm2 (k) can be improved compared to the two conventional techniques.
本発明による反響消去装置はハードウェアによって構成することが可能であるが、最も簡素に実現するにはコンピュータに本発明で提案する反響消去プログラムをインストールし、コンピュータに本発明による反響消去装置として機能させる実施形態が最良の形態である。
コンピュータを本発明による反響消去装置として機能させる場合、コンピュータには第一の反響路の模擬特性を有し、第一の再生信号を入力し、第一の反響信号を模擬する第一の反響模擬信号を生成する第一の適応型模擬反響路と、
第二の反響路の模擬特性を有し、第二の再生信号を入力し、第二の反響信号を模擬する第二の反響模擬信号を生成する第二の適応型模擬反響路と、
第一の反響模擬信号および第二の模擬信号を収音信号から差引き誤差信号を得る誤差産出手段と
誤差信号を均等割合で配分して得られる第一の推定値と第二の推定値を出力する均等誤差配分手段と、
誤差信号を第一の再生信号の電力と第二の再生信号の電力との比に応じて配分して得られる第三の推定値と第四の推定値を出力する信号電力加重誤差配分手段と、
第一の再生信号の電力と第二の再生信号の電力の大小関係に応じて異なる値を電力偏差として出力する電力偏差計算手段と、
電力偏差の値が、第一の再生信号の電力が第二の再生信号の電力より大きいことを示すほど、第三の推定値を第一の推定値よりも大きな割合で加重平均し、電力偏差の値が、第二の再生信号の電力が第一の再生信号の電力より大きいことを示すほど、第一の推定値を第三の推定値よりも大きな割合で加重平均し、第五の推定値として出力し、第一の適応型模擬反響路に与え、第一の適応型模擬反響路が有する模擬特性を更新させる第一の電力偏差加重平均手段と、
電力偏差の値が、第一の再生信号の電力が第二の再生信号の電力より大きいことを示すほど、第二の推定値を第四の推定値よりも大きな割合で加重平均し、電力偏差の値が、第二の再生信号の電力が第一の再生信号の電力より大きいことを示すほど、第四の推定値を第二の推定値よりも大きな割合で加重平均し、第六の推定値として出力し、第二の適応型模擬反響路に与え、第二の適応型模擬反響路が有する模擬特性を更新させる第二の電力偏差加重平均手段とを構築し、反響消去装置として機能させる。
The echo canceling apparatus according to the present invention can be configured by hardware, but in order to realize the simplest, the echo canceling program proposed in the present invention is installed in a computer, and the computer functions as the echo canceling apparatus according to the present invention. The embodiment to be made is the best mode.
When the computer functions as the echo canceling apparatus according to the present invention, the computer has a first echo path simulation characteristic, the first reproduction signal is input, and the first echo simulation that simulates the first echo signal A first adaptive simulated echo path for generating a signal;
A second adaptive simulated reverberation path that has a second reverberation path simulation characteristic, inputs a second reproduction signal, and generates a second reverberation simulation signal that simulates the second reverberation signal;
An error generating means for subtracting the first echo simulation signal and the second simulation signal from the collected sound signal and the first estimate value and the second estimate value obtained by allocating the error signal in equal proportions Uniform error distribution means for outputting;
Signal power weighted error distribution means for outputting a third estimated value and a fourth estimated value obtained by allocating the error signal according to the ratio of the power of the first reproduced signal and the power of the second reproduced signal; ,
Power deviation calculation means for outputting a value different as a power deviation according to the magnitude relationship between the power of the first reproduction signal and the power of the second reproduction signal;
As the power deviation value indicates that the power of the first reproduced signal is greater than the power of the second reproduced signal, the third estimated value is weighted and averaged at a rate greater than the first estimated value. As the value of indicates that the power of the second reproduced signal is greater than the power of the first reproduced signal, the first estimated value is weighted average at a rate greater than the third estimated value, and the fifth estimated A first power deviation weighted averaging means for outputting to the first adaptive simulated echo path as a value and updating a simulation characteristic of the first adaptive simulated echo path;
As the value of the power deviation indicates that the power of the first reproduction signal is larger than the power of the second reproduction signal, the second estimated value is weighted and averaged at a rate larger than the fourth estimated value, and the power deviation As the value of indicates that the power of the second reproduced signal is greater than the power of the first reproduced signal, the fourth estimated value is weighted and averaged at a rate greater than the second estimated value, and the sixth estimated A second power deviation weighted averaging means that outputs the value as a value, gives it to the second adaptive simulated echo path, and updates the simulation characteristics of the second adaptive simulated echo path, and functions as an echo canceller .
図1に本発明による反響消去装置300の実施例を示す。この実施例において、第一スピーカとマイクロホン3間の第一反響路4のインパルス応答に相当する長さLのベクトルH1の第一模擬特性Hm1(k)を保持する第一適応型模擬反響路101と、第二スピーカ2とマイクロホン3間の第二反響路5のインパルス応答に相当する長さLのベクトルH2の第二模擬特性Hm2(k)を保持する第二適応型模擬反響路102と、第一適応型模擬反響路101と第二適応型模擬反響路102とにおいて、それぞれ、第一の再生信号x1(k)および第二の再生信号x2(k)を入力として第一模擬特性Hm1(k)および第二模擬特性Hm2(k)との畳込演算により生成される第一模擬反響信号dm1(k)および第二模擬反響信号dm2(k)を、第一反響路4および第二反響路5を経て生成される第一反響信号d1(k)および第二反響信号d2(k)が混合してマイクロホン3において得られる収音信号y(k)から差引き、誤差信号e(k)を出力する誤差算出手段304とから成る構成に加えて(式3)、(式4)に基づき、第一の再生信号x1(k)の電力と、第一の再生信号x1(k)の電力と第二の再生信号x2(k)の電力の和との比を誤差信号e(k)に乗ずることにより第三の推定値を出力するとともに、第二の再生信号x2(k)の電力と、第一の再生信号x1(k)の電力と第二の再生信号x2(k)の電力の和との比を誤差信号e(k)に乗ずることにより第四の推定値を出力する信号電力加重誤差配分手段105とを備えた図2の構成か又は(式5)、(式6)に基づき、誤差信号e(k)を二分の一に均等に分割し、第一の推定値と第二の推定値を出力する均等誤差配分手段205を備えた図3の構成は従来から知られている。
FIG. 1 shows an embodiment of an echo canceling apparatus 300 according to the present invention. In this embodiment, a first adaptive
本発明の特徴とする構成は図2で説明した信号電力加重誤差配分手段105と図3で説明した均等誤差配分手段205とを共に有する点と、電力偏差計算手段301と、第一電力偏差加重平均手段302と、第二電力偏差加重平均手段303とを設けた構成を特徴とするものである。
以下本発明の特徴とする部分の動作を説明する。
均等誤差配分手段205は図3で説明したのと同様に、(式5)、(式6)に基づき、誤差信号e(k)を二分の一に均等分割し、第一の推定値em1(k)=(1/2)・e(k)と第二推定値em2(k)=(1/2)・e(k)を出力する。
The feature of the present invention is that it has both the signal power weighted error distribution means 105 described in FIG. 2 and the equal error distribution means 205 described in FIG. 3, a power deviation calculation means 301, and a first power deviation weighting. The present invention is characterized in that an averaging means 302 and a second power deviation weighted averaging means 303 are provided.
The operation of the characteristic features of the present invention will be described below.
The equal
信号電力加重誤差配分手段105は(式3)、(式4)に基づき、第一の再生信号x1(k)の電力‖x1(k)‖2と、第一の再生信号x1(k)の電力‖x1(k)‖2と第二の再生信号x2(k)の電力‖x2(k)‖2の和との比‖x1(k)‖2/(‖x1(k)‖2+‖x2(k)‖2)を誤差信号e(k)に乗ずることにより第三推定値em3(k)を出力すると共に、第二の再生信号x2(k)との電力‖x2(k)‖2と、第一の再生信号x1(k)の電力‖x1(k)‖2との和との比‖x2(k)‖2/(‖x1(k)‖2+‖x2(k)‖2)を誤差信号e(k)に乗ずることにより第四推定値em4(k)を出力する。
つまり、第三推定値em3(k)と第四推定値em4(k)は
電力偏差計算手段301は第一の再生信号x1(k)の電力‖x1(k)‖2と第二の再生信号x2(k)の電力‖x2(k)‖2の差を、第一の再生信号x1(k)の電力と第二の再生信号x2(k)の電力の和で除した値を電力偏差αとして出力する。
Signal power weighted error distribution means 105 (equation 3), the basis of the equation (4), a power ‖X1 (k) ‖ 2 of the first reproduction signal x1 (k), the first reproduction signal x1 (k) power ‖x1 (k) ‖ 2 and the second ratio of the sum of the power ‖x2 (k) ‖ 2 of the reproduced signal x2 (k) ‖x1 (k) ‖ 2 / (‖x1 (k) ‖ 2 + ‖ x2 (k) ‖ 2 ) is multiplied by the error signal e (k) to output the third estimated value em3 (k) and the power of the second reproduction signal x2 (k) 再生 x2 (k) ‖ 2 When the ratio of the sum of the power ‖x1 (k) ‖ 2 of the first reproduction signal x1 (k) ‖x2 (k) ‖ 2 / (‖x1 (k) ‖ 2 + ‖x2 (k) ‖ 2 ) Is multiplied by the error signal e (k) to output a fourth estimated value em4 (k).
That is, the third estimated value em3 (k) and the fourth estimated value em4 (k) are
The difference in power ‖x2 (k) ‖ 2 power ‖x1 (k) ‖ 2 and the second reproduction signal x2 (k) of the first reproduction signal x1 is power deviation calculation means 301 (k), the first A value obtained by dividing the power of the reproduction signal x1 (k) by the sum of the power of the second reproduction signal x2 (k) is output as a power deviation α.
em5=(1/2)・(1−α)・em1(k)+(1/2)・(1+α)・em3(k)
として出力し、この第五推定値em5(k)を第一適応型模擬反響路101に与え、第一模擬特性Hm1(k)を特性更新させる。
em5 = (1/2) ・ (1−α) ・ em1 (k) + (1/2) ・ (1 + α) ・ em3 (k)
The fifth estimated value em5 (k) is given to the first adaptive
第二電力偏差加重平均手段303は(式14)に基づき、1に電力偏差αを加えた値の二分の一と第二推定値との積(1/2)・(1+α)・em2(k)と、1から電力偏差αを引いた値の二分の一と第四推定値em4(k)との積(1/2)・(1−α)・em4(k)と、の和を第6推定値em6(k)
em6=(1/2)・(1+α)・em2(k)+(1/2)・(1−α)・em4(k)
として出力し、この第六推定値em6(k)を第二適応型模擬反響路102に与え、第二模擬特性Hm2(k)を特性更新させる。
Based on (Equation 14), the second power deviation weighted average means 303 is a product (1/2) · (1 + α) · em2 (k) of a half of the value obtained by adding the power deviation α to 1 and the second estimated value. ) And the product of 1/2 of the value obtained by subtracting the power deviation α from 1 and the fourth estimated value em4 (k) (1/2) · (1−α) · em4 (k) 6 Estimated value em6 (k)
em6 = (1/2) ・ (1 + α) ・ em2 (k) + (1/2) ・ (1−α) ・ em4 (k)
And the sixth estimated value em6 (k) is given to the second adaptive
このように本発明によれば第三推定値em3(k)と第四推定値em4(k)の算出において、従来の(式3)、(式4)による推定値、および従来の(式5)、(式6)による推定値のそれぞれの長所を活かすため、従来の推定値よりも推定精度を高めることができる。この結果として第一模擬特性Hm1(k)および第二模擬特性の模擬精度を従来の二つの技術より高めることができ、複数チャネルの反響を高い精度で除去乃至は消去することができる。 Thus, according to the present invention, in the calculation of the third estimated value em3 (k) and the fourth estimated value em4 (k), the estimated values based on the conventional (Expression 3) and (Expression 4), and the conventional (Expression 5) ) And (6), the estimation accuracy can be improved as compared with the conventional estimation value. As a result, the simulation accuracy of the first simulation characteristic Hm1 (k) and the second simulation characteristic can be improved as compared with the conventional two techniques, and the echoes of a plurality of channels can be removed or eliminated with high accuracy.
図1の第一電力偏差加重平均手段302、および、第二電力偏差加重平均手段303において、(式13)および(式14)を適用するにあたり、(式16)、(式18)のSNR1、SNR2は、
図1の第一電力偏差加重平均手段302、および、第二電力偏差加重平均手段303において、(式13)および(式14)の代わりに
以上説明した本発明による反響消去装置はハードウェアによって構成することも可能であるが現実的ではない。本発明による反響消去装置を簡素に実現するにはコンピュータに本発明で提案する反響消去プログラムをインストールし、コンピュータに上述した反響消去装置として機能させる実施形態が最良の実施形態である。本発明による反響消去プログラムはコンピュータが解読可能なプログラム言語によって記述され、磁気ディスク或いはCD-ROM、半導体メモリのような記録媒体に記録される。コンピュータへのインストールはこれらの記録媒体から或いは通信回線を通じてインストールされる。コンピュータへインストールされた状態ではコンピュータに備えられたCPUまたはDSPで解読され、コンピュータを反響消去装置として機能させる。
In the first power deviation weighted average means 302 and the second power deviation weighted average means 303 in FIG. 1, instead of (Equation 13) and (Equation 14)
The echo canceling apparatus according to the present invention described above can be configured by hardware, but it is not realistic. In order to simply realize the echo canceling apparatus according to the present invention, an embodiment in which the echo canceling program proposed in the present invention is installed in a computer and the computer functions as the echo canceling apparatus described above is the best embodiment. The echo cancellation program according to the present invention is described in a computer-readable program language, and is recorded on a recording medium such as a magnetic disk, a CD-ROM, or a semiconductor memory. The computer is installed from these recording media or through a communication line. When installed in the computer, it is decoded by a CPU or DSP provided in the computer, and the computer is made to function as an echo canceller.
本発明による反響消去装置はハンズフリー通話、ハンズフリー音声認識などへの適用が可能である。 The echo canceling apparatus according to the present invention can be applied to hands-free calling and hands-free voice recognition.
1 第一スピーカ 105 信号電力加重誤差配分手段
2 第二スピーカ 205 均等誤差配分手段
3 マイクロホン 301 電力偏差計算手段
4 第一反響路 302 第一電力偏差加重平均手段
5 第二反響路 303 第二電力偏差加重平均手段
101 第一適応型模擬反響路
102 第二適応型模擬反響路
DESCRIPTION OF
Claims (10)
前記第一の反響路の模擬特性を有し、前記第一の再生信号を入力し、前記第一の反響信号を模擬する第一の反響模擬信号を生成する第一の適応型反響模擬ステップと、
前記第二の反響路の模擬特性を有し、前記第二の再生信号を入力し、前記第二の反響信号を模擬する第二の反響模擬信号を生成する第二の適応型反響模擬ステップと、
前記第一の反響模擬信号および前記第二の反響模擬信号を前記収音信号から差引き誤差信号を得る誤差算出ステップと、
前記誤差信号を均等割合で配分して得られる第一の推定値と第二の推定値を出力する均等誤差配分ステップと、
前記誤差信号を前記第一の再生信号の電力と前記第二の再生信号の電力との比に応じて配分して得られる第三の推定値と第四の推定値を出力する信号電力加重誤差配分ステップと、
前記第一の再生信号の電力と前記第二の再生信号の電力の大小関係に応じて異なる値を電力偏差として出力する電力偏差計算ステップと、
前記電力偏差の値が、前記第一の再生信号の電力が前記第二の再生信号の電力より大きいことを示すほど、前記第三の推定値を前記第一の推定値よりも大きな割合で加重平均し、前記電力偏差の値が、前記第二の再生信号の電力が前記第一の再生信号の電力より大きいことを示すほど、前記第一の推定値を前記第三の推定値よりも大きな割合で加重平均し、第五の推定値として出力し、前記第一の適応型反響模擬ステップに与え、前記第一の適応型反響模擬ステップが有する模擬特性を更新させる第一の電力偏差加重平均ステップと、
前記電力偏差の値が、前記第一の再生信号の電力が前記第二の再生信号の電力より大きいことを示すほど、前記第二の推定値を前記第四の推定値よりも大きな割合で加重平均し、前記電力偏差の値が、前記第二の再生信号の電力が前記第一の再生信号の電力よりも大きいことを示すほど、前記第四の推定値を前記第二の推定値よりも大きな割合で加重平均し、第六の推定値として出力し、前記第二の適応型反響模擬ステップに与え、前記第二の適応型反響模擬ステップが有する模擬特性を更新させる第二の電力偏差加重平均ステップと、
を含むことを特徴とする反響消去方法。 A first sound reproduced in the first sound reproduction step via a first echo path that is an acoustic transmission path between the first sound reproduction step and the sound pickup step executed in the same space. A first reverberation signal in which a reproduction signal is collected in the sound collection step, a second sound reproduction step executed in the same space of the first sound reproduction step, and the sound collection step. The second reproduction signal reproduced in the second sound reproduction step is mixed with the first echo signal in the sound collection step via the second echo path which is an acoustic transmission path between In the echo canceling method, the second echo signal to be collected is erased or reduced from all the collected sound signals collected in the acoustic sound collecting step.
A first adaptive reverberation simulation step having simulation characteristics of the first reverberation path, inputting the first reproduction signal, and generating a first reverberation simulation signal for simulating the first reverberation signal; ,
A second adaptive reverberation simulation step having simulation characteristics of the second reverberation path, inputting the second reproduction signal, and generating a second reverberation simulation signal for simulating the second reverberation signal; ,
An error calculating step of obtaining a subtraction error signal from the collected sound signal of the first echo simulation signal and the second echo simulation signal;
An equal error distribution step of outputting a first estimated value and a second estimated value obtained by distributing the error signal at an equal ratio;
A signal power weighting error that outputs a third estimated value and a fourth estimated value obtained by allocating the error signal according to the ratio of the power of the first reproduced signal and the power of the second reproduced signal. An allocation step;
A power deviation calculation step of outputting a value different as a power deviation according to the magnitude relationship between the power of the first reproduction signal and the power of the second reproduction signal;
The third estimated value is weighted at a rate greater than the first estimated value, as the power deviation value indicates that the power of the first reproduced signal is greater than the power of the second reproduced signal. On average, the first estimated value is larger than the third estimated value as the power deviation value indicates that the power of the second reproduced signal is greater than the power of the first reproduced signal. A first power deviation weighted average that is weighted averaged by a ratio, output as a fifth estimated value, given to the first adaptive echo simulation step, and updates a simulation characteristic of the first adaptive echo simulation step Steps,
The value of the power deviation indicates that the power of the first reproduction signal is greater than the power of the second reproduction signal, and the second estimation value is weighted at a rate larger than the fourth estimation value. On average, the value of the power deviation indicates that the power of the second reproduction signal is greater than the power of the first reproduction signal, so that the fourth estimated value is more than the second estimated value. Weighted average with a large proportion, output as a sixth estimated value, given to the second adaptive echo simulation step, the second power deviation weight to update the simulation characteristics of the second adaptive echo simulation step Average step and
The echo cancellation method characterized by including.
1に請求項2に基づく電力偏差を加えた値の二分の一と前記第二の推定値との積と、1から請求項2に基づく電力偏差を引いた値の二分の一と前記第四の推定値との積と、の和として得ることを特徴とする請求項1乃至3の何れかに記載の反響消去方法。 In the second power deviation weighted average step, a value of a weighted average of the second estimated value and the fourth estimated value for obtaining the sixth estimated value,
The product of one half of the value obtained by adding the power deviation based on claim 2 to the second estimated value and one half of the value obtained by subtracting the power deviation based on claim 2 from 1 and the fourth The echo canceling method according to claim 1, wherein the echo canceling method is obtained as a sum of a product of the estimated value and the estimated value.
前記第一の反響路の模擬特性を有し、前記第一の再生信号を入力し、前記第一の反響信号を模擬する第一の反響模擬信号を生成する第一の適応型模擬反響路と、
前記第二の反響路の模擬特性を有し、前記第二の再生信号を入力し、前記第二の反響信号を模擬する第二の反響模擬信号を生成する第二の適応型模擬反響路と、
前記第一の反響模擬信号および前記第二の反響模擬信号を前記収音信号から差引き誤差信号を得て、この誤差信号を出力する誤差算出手段と、
前記誤差信号を均等割合で配分して得られる第一の推定値と第二の推定値を出力する均等誤差配分手段と、
前記誤差値を前記第一の再生信号の電力と前記第二の再生信号の電力との比に応じて配分して得られる第三の推定値と第四の推定値を出力する信号電力加重誤差配分手段と、
前記第一の再生信号の電力と前記第二の再生信号の電力の大小関係に応じて異なる値を電力偏差として出力する電力偏差計算手段と、
前記電力偏差の値が、前記第一の再生信号の電力が前記第二の再生信号の電力より大きいことを示すほど、前記第三の推定値を前記第一の推定値よりも大きな割合で加重平均し、前記電力偏差の値が、前記第二の再生信号の電力が前記第一の再生信号の電力より大きいことを示すほど、前記第一の推定値を前記第三の推定値よりも大きな割合で加重平均し、第五の推定値として出力し、前記第一の適応型模擬反響路に与え、前記第一の適応型模擬反響路が有する模擬特性を更新させる第一の電力偏差加重平均手段と、
前記電力偏差の値が、前記第一の再生信号の電力が前記第二の再生信号の電力より大きいことを示すほど、前記第二の推定値を前記第四の推定値よりも大きな割合で加重平均し、前記電力偏差の値が、前記第二の再生信号の電力が前記第一の再生信号の電力より大きいことを示すほど、前記第四の推定値を前記第二の推定値よりも大きな割合で加重平均し、第六の推定値として出力し、前記第二の適応型模擬反響路に与え、前記第二の適応型模擬反響路が有する模擬特性を更新させる第二の電力偏差加重平均手段と、
を有することを特徴とする反響消去装置。 1st reproduction | regeneration reproduced | regenerated from said 1st sound reproduction | regeneration means via the 1st echo path which is an acoustic transmission path | route between the 1st sound reproduction | regeneration means and sound collection means which exist in the same space Between the first reverberation signal in which the signal is picked up by the sound pickup means, and between the second sound reproduction means and the sound pickup means existing in the same space of the first sound reproduction means. A second reproduction signal reproduced from the second sound reproducing means is mixed with the first echo signal by the acoustic sound collecting means via a second echo path that is an acoustic transmission path, and collected. In the echo canceling apparatus for canceling or reducing the second echo signal to be collected from all the collected sound signals collected by the acoustic sound collecting means,
A first adaptive simulated reverberation path having simulation characteristics of the first reverberation path, receiving the first reproduction signal and generating a first reverberation simulation signal for simulating the first reverberation signal; ,
A second adaptive simulated reverberation path having simulation characteristics of the second reverberation path, generating the second reverberation simulation signal for inputting the second reproduction signal and simulating the second reverberation signal; ,
An error calculating means for obtaining a subtraction error signal from the collected sound signal and outputting the error signal of the first echo simulation signal and the second echo simulation signal;
Equal error distribution means for outputting a first estimated value and a second estimated value obtained by distributing the error signal at an equal ratio;
A signal power weighting error that outputs a third estimated value and a fourth estimated value obtained by allocating the error value according to the ratio of the power of the first reproduced signal and the power of the second reproduced signal. Distribution means,
A power deviation calculating means for outputting a value different as a power deviation according to the magnitude relationship between the power of the first reproduction signal and the power of the second reproduction signal;
The third estimated value is weighted at a rate greater than the first estimated value, as the power deviation value indicates that the power of the first reproduced signal is greater than the power of the second reproduced signal. On average, the first estimated value is larger than the third estimated value as the power deviation value indicates that the power of the second reproduced signal is greater than the power of the first reproduced signal. A first power deviation weighted average that is weighted averaged as a ratio, output as a fifth estimated value, given to the first adaptive simulated echo path, and updates the simulated characteristics of the first adaptive simulated echo path Means,
The value of the power deviation indicates that the power of the first reproduction signal is greater than the power of the second reproduction signal, and the second estimation value is weighted at a rate larger than the fourth estimation value. On average, the fourth estimated value is larger than the second estimated value as the power deviation value indicates that the power of the second reproduced signal is larger than the power of the first reproduced signal. Weighted average with a ratio, output as a sixth estimated value, given to the second adaptive simulated echo path, and a second power deviation weighted average that updates the simulation characteristics of the second adaptive simulated echo path Means,
An echo canceling apparatus characterized by comprising:
1に請求項6に基づく電力偏差を加えた値の二分の一と前記第二の推定値との積と、1から請求項6に基づく電力偏差を引いた値の二分の一と前記第四の推定値との積と、の和として得ることを特徴とする請求項5乃至7の何れかに記載の反響消去装置。 In the second power deviation weighted average means, a weighted average value of the second estimated value and the fourth estimated value for obtaining the sixth estimated value,
The product of one half of the value obtained by adding the power deviation based on claim 6 to the second estimated value and one half of the value obtained by subtracting the power deviation based on 1 from 6 and the fourth The echo canceling apparatus according to claim 5, wherein the echo canceling apparatus is obtained as a sum of a product with an estimated value of
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2005251648A JP4504892B2 (en) | 2005-08-31 | 2005-08-31 | Echo canceling method, echo canceling apparatus, program, recording medium |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
JP2005251648A JP4504892B2 (en) | 2005-08-31 | 2005-08-31 | Echo canceling method, echo canceling apparatus, program, recording medium |
Publications (2)
Publication Number | Publication Date |
---|---|
JP2007067855A JP2007067855A (en) | 2007-03-15 |
JP4504892B2 true JP4504892B2 (en) | 2010-07-14 |
Family
ID=37929516
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
JP2005251648A Expired - Fee Related JP4504892B2 (en) | 2005-08-31 | 2005-08-31 | Echo canceling method, echo canceling apparatus, program, recording medium |
Country Status (1)
Country | Link |
---|---|
JP (1) | JP4504892B2 (en) |
Families Citing this family (1)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP5438629B2 (en) * | 2010-08-23 | 2014-03-12 | 日本電信電話株式会社 | Stereo echo canceling method, stereo echo canceling device, stereo echo canceling program |
Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2002261659A (en) * | 2001-02-27 | 2002-09-13 | Nippon Telegr & Teleph Corp <Ntt> | Multi-channel echo cancellation method, its apparatus, its program, and its storage medium |
JP2002261660A (en) * | 2001-02-27 | 2002-09-13 | Nippon Telegr & Teleph Corp <Ntt> | Multi-channel echo cancellation method, its apparatus, its program, and its storage medium |
-
2005
- 2005-08-31 JP JP2005251648A patent/JP4504892B2/en not_active Expired - Fee Related
Patent Citations (2)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2002261659A (en) * | 2001-02-27 | 2002-09-13 | Nippon Telegr & Teleph Corp <Ntt> | Multi-channel echo cancellation method, its apparatus, its program, and its storage medium |
JP2002261660A (en) * | 2001-02-27 | 2002-09-13 | Nippon Telegr & Teleph Corp <Ntt> | Multi-channel echo cancellation method, its apparatus, its program, and its storage medium |
Also Published As
Publication number | Publication date |
---|---|
JP2007067855A (en) | 2007-03-15 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3791565B1 (en) | Method and apparatus utilizing residual echo estimate information to derive secondary echo reduction parameters | |
JP4954334B2 (en) | Apparatus and method for calculating filter coefficients for echo suppression | |
JP5102371B2 (en) | Apparatus and method for calculating control information of echo suppression filter, and apparatus and method for calculating delay value | |
CN103137136B (en) | Sound processing device | |
JP2008517317A (en) | Audio data processing system, method, program element, and computer readable medium | |
JP5762479B2 (en) | Voice switch device, voice switch method, and program thereof | |
JP4504892B2 (en) | Echo canceling method, echo canceling apparatus, program, recording medium | |
JP4504891B2 (en) | Echo canceling method, echo canceling apparatus, program, recording medium | |
JP2003309493A (en) | Method, device and program for reducing echo | |
JP5937451B2 (en) | Echo canceling apparatus, echo canceling method and program | |
JP2003250193A (en) | Echo elimination method, device for executing the method, program and recording medium therefor | |
JP5438629B2 (en) | Stereo echo canceling method, stereo echo canceling device, stereo echo canceling program | |
JP5583181B2 (en) | Cascade connection type transmission system parameter estimation method, cascade connection type transmission system parameter estimation device, program | |
JP5774062B2 (en) | Echo canceling apparatus, echo canceling method, and program thereof | |
JP5826712B2 (en) | Multi-channel echo canceling apparatus, multi-channel echo canceling method, and program | |
JP2017191992A (en) | Echo suppressor, method therefor, program, and recording medium | |
JP2017191987A (en) | Echo suppressor, method thereof, program, and record medium | |
JP6790659B2 (en) | Sound processing equipment and sound processing method | |
JP4559989B2 (en) | Echo suppression method, echo suppression program, recording medium | |
JP6075783B2 (en) | Echo canceling apparatus, echo canceling method and program | |
JP5925149B2 (en) | Acoustic coupling amount estimating apparatus, echo canceling apparatus, method and program thereof | |
Cifani et al. | A PEM-AFROW based algorithm for acoustic feedback control in automotive speech reinforcement systems | |
CN113841198B (en) | Signal component estimation using coherence | |
JP2002261659A (en) | Multi-channel echo cancellation method, its apparatus, its program, and its storage medium | |
JP5325134B2 (en) | Echo canceling method, echo canceling apparatus, program thereof, and recording medium |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
A621 | Written request for application examination |
Free format text: JAPANESE INTERMEDIATE CODE: A621 Effective date: 20070810 |
|
TRDD | Decision of grant or rejection written | ||
A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 Effective date: 20100413 |
|
A01 | Written decision to grant a patent or to grant a registration (utility model) |
Free format text: JAPANESE INTERMEDIATE CODE: A01 |
|
A61 | First payment of annual fees (during grant procedure) |
Free format text: JAPANESE INTERMEDIATE CODE: A61 Effective date: 20100423 |
|
R150 | Certificate of patent or registration of utility model |
Ref document number: 4504892 Country of ref document: JP Free format text: JAPANESE INTERMEDIATE CODE: R150 Free format text: JAPANESE INTERMEDIATE CODE: R150 |
|
FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20130430 Year of fee payment: 3 |
|
FPAY | Renewal fee payment (event date is renewal date of database) |
Free format text: PAYMENT UNTIL: 20140430 Year of fee payment: 4 |
|
S531 | Written request for registration of change of domicile |
Free format text: JAPANESE INTERMEDIATE CODE: R313531 |
|
R350 | Written notification of registration of transfer |
Free format text: JAPANESE INTERMEDIATE CODE: R350 |
|
LAPS | Cancellation because of no payment of annual fees |