JP5950199B2 - Loudspeaker - Google Patents

Loudspeaker Download PDF

Info

Publication number
JP5950199B2
JP5950199B2 JP2012173385A JP2012173385A JP5950199B2 JP 5950199 B2 JP5950199 B2 JP 5950199B2 JP 2012173385 A JP2012173385 A JP 2012173385A JP 2012173385 A JP2012173385 A JP 2012173385A JP 5950199 B2 JP5950199 B2 JP 5950199B2
Authority
JP
Japan
Prior art keywords
loss
unit
signal
echo
feedback gain
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
JP2012173385A
Other languages
Japanese (ja)
Other versions
JP2014033372A (en
Inventor
哲平 鷲
哲平 鷲
恵一 ▲吉▼田
恵一 ▲吉▼田
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Panasonic Intellectual Property Management Co Ltd
Original Assignee
Panasonic Intellectual Property Management Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Panasonic Intellectual Property Management Co Ltd filed Critical Panasonic Intellectual Property Management Co Ltd
Priority to JP2012173385A priority Critical patent/JP5950199B2/en
Publication of JP2014033372A publication Critical patent/JP2014033372A/en
Application granted granted Critical
Publication of JP5950199B2 publication Critical patent/JP5950199B2/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Description

本発明は、住宅や事務所等で用いられるインターホンなどの拡声通話装置に関する。   The present invention relates to a loudspeaker device such as an interphone used in a house or office.

従来の拡声通話装置として、特許文献1に記載されているものがある。特許文献1記載の従来例は、不快なエコー及びハウリングの発生を防止するためにエコーキャンセラ並びに音声スイッチを備えている。そして、通話開始直後のエコーキャンセラが収束していない状態においては、音声スイッチが信号経路に挿入する損失の総量(総損失量)を十分に大きい初期値に固定する固定モードで動作することで不快なエコーやハウリングを抑制している。一方、エコーキャンセラが十分に収束した状態においては、音声スイッチが総損失量を随時更新する更新モードで動作することで双方向の同時通話を実現している。   A conventional loudspeaker device is described in Patent Document 1. The conventional example described in Patent Document 1 includes an echo canceller and a voice switch in order to prevent unpleasant echoes and howling. When the echo canceller immediately after the start of the call has not converged, it is uncomfortable to operate in a fixed mode in which the total amount of loss (total amount of loss) inserted into the signal path by the voice switch is fixed to a sufficiently large initial value. Echo and howling are suppressed. On the other hand, when the echo canceller is sufficiently converged, a two-way simultaneous call is realized by operating in an update mode in which the voice switch updates the total loss amount as needed.

さらに特許文献1記載の従来例では、エコーやハウリングを生じさせる帰還経路の利得(帰還利得)を周波数帯域別に求め、周波数帯域別に求めた帰還利得のうちで最大のものを帰還利得の推定値として選択している。故に参照信号(送話信号及び受話信号)の時間平均パワーから帰還利得を推定する場合に比べて帰還利得の推定精度の向上が図れる。   Furthermore, in the conventional example described in Patent Document 1, the gain (feedback gain) of the feedback path that causes echo and howling is obtained for each frequency band, and the largest one of the feedback gains obtained for each frequency band is used as the estimated value of the feedback gain. Selected. Therefore, the estimation accuracy of the feedback gain can be improved as compared with the case where the feedback gain is estimated from the time average power of the reference signal (transmitted signal and received signal).

特開2007−194740号公報JP 2007-194740 A

ところで、特許文献1記載の従来例では、周波数帯域別の帰還利得を求める際に、参照信号を離散フーリエ変換処理することにより周波数帯域別の信号レベルを求めている。しかしながら、離散フーリエ変換処理には三角関数や平方根の演算が必要であり、音声スイッチに用いられる、固定小数点演算を行う安価な信号処理回路(DSPなど)では処理量が大きいために処理時間も長くなってしまうという問題がある。一方、高速な演算処理が可能な信号処理回路を用いると、拡声通話装置の製造コストが上昇してしまうという問題がある。   By the way, in the conventional example described in Patent Document 1, when obtaining the feedback gain for each frequency band, the signal level for each frequency band is obtained by subjecting the reference signal to discrete Fourier transform processing. However, the discrete Fourier transform processing requires trigonometric functions and square root operations, and an inexpensive signal processing circuit (such as a DSP) that performs fixed-point arithmetic used for a voice switch has a large amount of processing, so the processing time is long. There is a problem of becoming. On the other hand, when a signal processing circuit capable of high-speed arithmetic processing is used, there is a problem that the manufacturing cost of the loudspeaker device increases.

本発明は、上記課題に鑑みて為されたものであり、製造コストの上昇を抑えつつ帰還利得の推定精度の向上を図ることを目的とする。   The present invention has been made in view of the above problems, and an object thereof is to improve the estimation accuracy of feedback gain while suppressing an increase in manufacturing cost.

本発明の拡声通話装置は、マイクロホン及びスピーカと、相手側の通話端末から送られてくる受話信号を前記スピーカに伝送する受話側信号経路並びに前記マイクロホンで集音された送話信号を伝送して前記相手側の通話端末へ送る送話側信号経路に損失を挿入することで通話状態を受話及び送話に切り換える音声スイッチと、前記マイクロホンとスピーカの音響結合によって生じる音響エコーを抑制するエコーキャンセラとを備えており、前記音声スイッチは、送話側の信号経路に損失を挿入する送話側損失挿入部と、受話側の信号経路に損失を挿入する受話側損失挿入部と、送話側及び受話側の前記各損失挿入部から挿入する損失量を制御する挿入損失量制御部とを具備し、前記挿入損失量制御部は、前記受話側損失挿入部の出力点から音響エコー経路を介して前記送話側損失挿入部の入力点へ帰還する経路の音響側帰還利得を推定するとともに、前記送話側損失挿入部の出力点から回線エコー経路を介して前記受話側損失挿入部の入力点へ帰還する経路の回線側帰還利得を推定する帰還利得推定部と、音響側及び回線側の各帰還利得の推定値に基づいて閉ループに挿入すべき損失量の総和を算出する総損失量算出部と、送話信号及び受話信号を監視して通話状態を推定し、この推定結果と前記総損失量算出部の算出値に応じて前記送話側損失挿入部及び前記受話側損失挿入部の各挿入損失量の配分を決定する挿入損失量分配処理部とを有する拡声通話装置において、前記帰還利得推定部は、音響側及び回線側の各帰還経路が固有にもつ信号伝達時間の差を補正する伝達時間差補正部と、前記伝達時間差補正部で補正された後の信号を複数の周波数帯域に弁別することにより周波数帯域別の信号レベルを求める周波数帯域別信号レベル算出部と、前記周波数帯域別信号レベル算出部で算出した信号レベルから周波数帯域別の帰還利得の推定値を演算する推定値演算部とを備え、前記周波数帯域別信号レベル算出部は、互いに異なる周波数帯域を通過帯域とする複数の帯域通過フィルタによって前記信号を複数の周波数帯域に弁別し、前記音響エコーを抑制する第1エコーキャンセラと、前記相手側の通話端末における音響結合又は前記回線エコー経路を介した前記送話信号の回り込みによって生じる回線エコーを抑制する第2エコーキャンセラとを備え、前記第1エコーキャンセラは、前記音響エコー経路のインパルス応答を適応的に同定して前記音響エコー経路への入力信号から前記音響エコーを推定する適応フィルタと、前記適応フィルタで推定された前記音響エコーを前記音響エコー経路からの出力信号より減算する減算器とを有し、前記第2エコーキャンセラは、前記回線エコー経路のインパルス応答を適応的に同定して前記回線エコー経路への入力信号から前記回線エコーを推定する適応フィルタと、前記適応フィルタで推定された前記回線エコーを前記回線エコー経路からの出力信号より減算する減算器とを有し、前記第1エコーキャンセラの前記適応フィルタ、並びに前記第2エコーキャンセラの前記適応フィルタは、前記挿入損失量分配処理部が決定する前記挿入損失量の配分に応じて、フィルタ係数を更新するタイミングを調整することを特徴とする。 The loudspeaker apparatus of the present invention transmits a microphone and a speaker, a reception side signal path for transmitting a reception signal transmitted from the other party's telephone terminal to the speaker, and a transmission signal collected by the microphone. A voice switch for switching a call state between reception and transmission by inserting a loss in a transmission side signal path to be sent to the call terminal on the other side, an echo canceller for suppressing acoustic echo caused by acoustic coupling of the microphone and the speaker, and The voice switch includes a transmission side loss insertion unit that inserts a loss into the signal path on the transmission side, a reception side loss insertion unit that inserts a loss into the signal path on the reception side, a transmission side, and An insertion loss amount control unit that controls a loss amount to be inserted from each loss insertion unit on the receiving side, and the insertion loss amount control unit is connected to an output point of the reception side loss insertion unit. With estimating the acoustic side feedback gain path for feeding back through the sound echo path to an input point of the transmitting end losses inserting portion, the receiving side through a line echo path from the output point of the transmitting end losses insertion portion A feedback gain estimator that estimates the line-side feedback gain of the path that returns to the input point of the loss insertion unit , and calculates the total amount of loss to be inserted into the closed loop based on the estimated values of the feedback gains on the acoustic and line sides A total loss amount calculation unit that monitors a transmission signal and a reception signal to estimate a call state, and the transmission side loss insertion unit and the reception side according to the estimation result and a calculated value of the total loss amount calculation unit In the loudspeaker apparatus having the insertion loss amount distribution processing unit for determining the distribution of each insertion loss amount of the side loss insertion unit, the feedback gain estimation unit transmits the signal inherently to each feedback path on the acoustic side and the line side Transmission time difference to compensate for time difference A signal level calculation unit for each frequency band that obtains a signal level for each frequency band by discriminating the signal corrected by the transmission time difference correction unit into a plurality of frequency bands, and a signal level calculation for each frequency band And an estimated value calculation unit for calculating an estimated value of feedback gain for each frequency band from the signal level calculated by the frequency unit, wherein the signal level calculation unit for each frequency band has a plurality of band passes having different frequency bands as pass bands. This is caused by the first echo canceller that discriminates the signal into a plurality of frequency bands by a filter and suppresses the acoustic echo, and acoustic coupling at the other party's call terminal or wraparound of the transmitted signal via the line echo path A second echo canceller for suppressing line echo, wherein the first echo canceller is an impulse of the acoustic echo path. An adaptive filter that adaptively identifies a response and estimates the acoustic echo from an input signal to the acoustic echo path, and a subtraction that subtracts the acoustic echo estimated by the adaptive filter from an output signal from the acoustic echo path The second echo canceller adaptively identifies an impulse response of the line echo path and estimates the line echo from an input signal to the line echo path, and the adaptive filter A subtractor that subtracts the estimated line echo from an output signal from the line echo path, and the adaptive filter of the first echo canceller and the adaptive filter of the second echo canceller include the insertion loss. depending on the allocation of the insertion loss amount distribution processing section determines, to adjust the timing for updating the filter coefficients And butterflies.

この拡声通話装置において、前記帰還利得推定部は、前記音響側帰還利得を推定する処理と、前記回線側帰還利得を推定する処理とを一定時間毎に交互に行うことが好ましい。   In this loudspeaker, it is preferable that the feedback gain estimation unit alternately performs the process of estimating the acoustic side feedback gain and the process of estimating the line side feedback gain at regular intervals.

この拡声通話装置において、前記周波数帯域別信号レベル算出部は、前記エコーキャンセラが帰還利得を推定する経路の途中で音声成分が印加されるダブルトーク状態を検出した場合、前記帯域通過フィルタのフィルタ処理を中止して前回のフィルタ処理で得られた信号レベルで代用することが好ましい。   In this loudspeaker apparatus, the signal level calculation unit for each frequency band detects a double talk state in which a voice component is applied in the middle of a path where the echo canceller estimates a feedback gain. It is preferable to substitute the signal level obtained by the previous filtering process by stopping the operation.

この拡声通話装置において、前記エコーキャンセラは、音響側及び回線側の各帰還経路が固有にもつ信号伝達時間の差を検出する伝達時間差検出手段を有し、前記伝達時間差補正部は、前記伝達時間差検出手段が検出する伝達時間差を用いて前記伝達時間差の補正処理を行うことが好ましい。   In this loudspeaker communication apparatus, the echo canceller includes a transmission time difference detection unit that detects a difference in signal transmission time inherent in each of the feedback paths on the acoustic side and the line side, and the transmission time difference correction unit includes the transmission time difference The transmission time difference is preferably corrected using the transmission time difference detected by the detecting means.

この拡声通話装置において、前記推定値演算部は、前記周波数帯域別信号レベル算出部で算出した信号レベルから周波数帯域別の帰還利得を推定する際、前記信号レベルが所定のしきい値以下となる前記周波数帯域の帰還利得の推定値を演算しないことが好ましい。   In this loudspeaker, when the estimated value calculation unit estimates a feedback gain for each frequency band from the signal level calculated by the signal level calculation unit for each frequency band, the signal level becomes a predetermined threshold value or less. It is preferable not to calculate an estimated value of the feedback gain in the frequency band.

この拡声通話装置において、前記総損失量算出部は、前記帰還利得推定部で推定される前記帰還利得の推定値に所定のマージンを加算して前記損失量の総和を算出することが好ましい。   In this loudspeaker apparatus, it is preferable that the total loss amount calculation unit calculates a total sum of the loss amounts by adding a predetermined margin to the estimated value of the feedback gain estimated by the feedback gain estimation unit.

この拡声通話装置において、前記総損失量算出部は、前記帰還利得の推定値が求められた周波数帯域毎に前記総損失量の総和を算出し、前記挿入損失量分配処理部は、前記送話側損失挿入部及び前記受話側損失挿入部の各挿入損失量の配分を周波数帯域毎に決定することが好ましい。   In the voice communication device, the total loss amount calculation unit calculates a total sum of the total loss amounts for each frequency band in which the estimated value of the feedback gain is obtained, and the insertion loss amount distribution processing unit It is preferable that the distribution of the insertion loss amounts of the side loss insertion unit and the reception side loss insertion unit is determined for each frequency band.

この拡声通話装置において、前記第1エコーキャンセラの前記適応フィルタは、前記挿入損失量分配処理部が前記送話側損失挿入部に対して相対的に高い挿入損失量を配分した場合に前記フィルタ係数を更新し、前記第2エコーキャンセラの前記適応フィルタは、前記挿入損失量分配処理部が前記受話側損失挿入部に対して相対的に高い挿入損失量を配分した場合に前記フィルタ係数を更新することが好ましい。   In this loudspeaker apparatus, the adaptive filter of the first echo canceller is configured such that when the insertion loss amount distribution processing unit distributes a relatively high insertion loss amount to the transmission side loss insertion unit, the filter coefficient And the adaptive filter of the second echo canceller updates the filter coefficient when the insertion loss amount distribution processing unit distributes a relatively high insertion loss amount to the reception-side loss insertion unit. It is preferable.

この拡声通話装置において、前記第1エコーキャンセラ及び前記第2エコーキャンセラの前記各適応フィルタは、前記フィルタ係数を更新している場合において、前記挿入損失量分配処理部が前記送話側及び前記受話側の各損失挿入部に対して前記挿入損失量を等しく配分する状態に変化してから一定期間が経過するまでは前記フィルタ係数の更新を継続することが好ましい。   In this loudspeaker, when each of the adaptive filters of the first echo canceller and the second echo canceller is updating the filter coefficient, the insertion loss amount distribution processing unit is configured to transmit the transmission side and the reception side. It is preferable to continue to update the filter coefficient until a certain period elapses after the insertion loss amount is equally distributed to the respective loss insertion portions.

本発明の拡声通話装置は、周波数帯域別信号レベル算出部が複数の帯域通過フィルタによって参照信号を複数の周波数帯域に弁別しているので、従来例と比較して、製造コストの上昇を抑えつつ帰還利得の推定精度の向上を図ることができるという効果がある。   In the loudspeaker according to the present invention, the signal level calculation unit for each frequency band discriminates the reference signal into a plurality of frequency bands by a plurality of band pass filters, and therefore feedback is performed while suppressing an increase in manufacturing cost compared to the conventional example. There is an effect that the estimation accuracy of the gain can be improved.

本発明に係る拡声通話装置の実施形態における挿入損失量制御部を示すブロック図である。It is a block diagram which shows the insertion loss amount control part in embodiment of the loudspeaker apparatus which concerns on this invention. 同上を示すブロック図である。It is a block diagram which shows the same as the above. 従来例における周波数帯域別信号レベル算出部の動作を説明するためのフローチャートである。It is a flowchart for demonstrating operation | movement of the signal level calculation part classified by frequency band in a prior art example.

以下、本発明に係る拡声通話装置の実施形態について図面を参照して詳細に説明する。   DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments of a loudspeaker device according to the present invention will be described in detail with reference to the drawings.

本実施形態は、図2に示すようにマイクロホン1、スピーカ2、2線−4線変換回路3、マイクロホンアンプG1、回線出力アンプG2、回線入力アンプG3、スピーカアンプG4、増幅器G5,G6、音声スイッチ10、第1及び第2のエコーキャンセラ30A,30Bなどを備える。なお、回線は2線式の伝送路からなる。   In the present embodiment, as shown in FIG. 2, a microphone 1, a speaker 2, a 2-wire to 4-wire conversion circuit 3, a microphone amplifier G1, a line output amplifier G2, a line input amplifier G3, a speaker amplifier G4, amplifiers G5 and G6, audio The switch 10 includes first and second echo cancellers 30A and 30B. The line consists of a two-wire transmission line.

第1エコーキャンセラ30Aは、適応フィルタ31Aと減算器32Aからなる従来周知の構成を有する。適応フィルタ31Aは、スピーカ2−マイクロホン1間の音響結合により形成される帰還経路(音響エコー経路)HACのインパルス応答を適応的に同定する。減算器32Aは、適応フィルタ31Aで推定された擬似エコー成分(音響エコー)をマイクロホンアンプG1の出力信号から減算することで音響エコーを抑制する。また、第2エコーキャンセラ30Bも適応フィルタ31Bと減算器32Bからなる従来周知の構成を有する。適応フィルタ31Bは、2線−4線変換回路3と伝送路との間のインピーダンスの不整合による反射および相手の通話端末におけるスピーカ−マイクロホン間の音響結合とにより形成される帰還経路(回線エコー経路)HLINのインパルス応答を適応的に同定する。減算器32Bは、適応フィルタ31Bで推定された擬似エコー成分(回線エコー)を受話信号から減算することで回線エコーを抑制する。 The first echo canceller 30A has a conventionally known configuration including an adaptive filter 31A and a subtracter 32A. Adaptive filter 31A identifies the impulse response of the feedback path (acoustic echo path) H AC formed by the acoustic coupling between the speaker 2 microphone 1 adaptively. The subtractor 32A suppresses the acoustic echo by subtracting the pseudo echo component (acoustic echo) estimated by the adaptive filter 31A from the output signal of the microphone amplifier G1. The second echo canceller 30B also has a conventionally known configuration including an adaptive filter 31B and a subtractor 32B. The adaptive filter 31B has a feedback path (line echo path) formed by reflection due to impedance mismatch between the 2-wire-to-wire conversion circuit 3 and the transmission path and acoustic coupling between the speaker and the microphone at the other party's call terminal. ) Adaptively identify the impulse response of H LIN . The subtractor 32B suppresses the line echo by subtracting the pseudo echo component (line echo) estimated by the adaptive filter 31B from the received signal.

音声スイッチ10は、送話側の信号経路に損失を挿入する送話側損失挿入部11と、受話側の信号経路に損失を挿入する受話側損失挿入部12と、送話側及び受話側の各損失挿入部11,12から挿入する損失量を制御する挿入損失量制御部13とを具備する。また挿入損失量制御部13は、帰還利得推定部14と、総損失量算出部15と、挿入損失量分配処理部16とを有する。帰還利得推定部14は、受話側損失挿入部12の出力点Routから音響エコー経路HACを介して送話側損失挿入部11の入力点Tinへ帰還する経路(以下、音響側帰還経路という)の音響側帰還利得αを推定する。さらに帰還利得推定部14は、送話側損失挿入部11の出力点Toutから回線エコー経路HLINを介して受話側損失挿入部12の入力点Rinへ帰還する経路(以下、回線側帰還経路という)の回線側帰還利得βを推定する。総損失量算出部15は、音響側及び回線側の各帰還利得α、βの推定値α'、β'に基づいて閉ループに挿入すべき損失量の総和(送話側損失挿入部11の挿入損失量と受話側損失挿入部12の挿入損失量の和)を算出する。挿入損失量分配処理部16は、送話信号及び受話信号を監視して通話状態を推定し、この推定結果と総損失量算出部15の算出値に応じて送話側損失挿入部11及び受話側損失挿入部12の各挿入損失量の配分を決定する。なお、本実施形態における第1及び第2のエコーキャンセラ30A,30B並びに音声スイッチ10は、DSP(Digital Signal Processor)のハードウエアをエコーキャンセラ用並びに音声スイッチ用のソフトウエア(プログラム)で制御することによって実現されている。従って、以下の説明における音声スイッチ10並びに第1及び第2のエコーキャンセラ30A,30Bの入出力信号(受話信号及び送話信号)は所定のサンプリング周期でサンプリングされ、且つA/D変換器により量子化されている。 The voice switch 10 includes a transmission side loss insertion unit 11 that inserts a loss into the signal path on the transmission side, a reception side loss insertion unit 12 that inserts a loss into the signal path on the reception side, and a transmission side and a reception side. And an insertion loss amount control unit 13 for controlling the loss amount inserted from each of the loss insertion units 11 and 12. The insertion loss amount control unit 13 includes a feedback gain estimation unit 14, a total loss calculation unit 15, and an insertion loss amount distribution processing unit 16. Feedback gain estimation unit 14, the path to be fed back from the output point Rout of the receiving-side loss insertion portion 12 into the input point Tin of the transmitting end losses insertion portion 11 via the acoustic echo path H AC (hereinafter referred to as acoustic-side feedback path) The acoustic side feedback gain α is estimated. Further, the feedback gain estimation unit 14 returns a path (hereinafter referred to as a line side feedback path) from the output point Tout of the transmission side loss insertion part 11 to the input point Rin of the reception side loss insertion part 12 via the line echo path HLIN. ) Is estimated. The total loss calculation unit 15 calculates the total loss amount to be inserted into the closed loop based on the estimated values α ′ and β ′ of the feedback gains α and β on the acoustic side and the line side (insertion of the transmission side loss insertion unit 11 The sum of the loss amount and the insertion loss amount of the reception-side loss insertion unit 12 is calculated. The insertion loss amount distribution processing unit 16 monitors the transmission signal and the reception signal to estimate the call state, and according to the estimation result and the calculated value of the total loss amount calculation unit 15, the transmission loss insertion unit 11 and the reception side The distribution of each insertion loss amount of the side loss insertion unit 12 is determined. The first and second echo cancellers 30A and 30B and the voice switch 10 in this embodiment control the DSP (Digital Signal Processor) hardware by software (program) for the echo canceller and voice switch. It is realized by. Therefore, the input / output signals (received signal and transmitted signal) of the voice switch 10 and the first and second echo cancellers 30A and 30B in the following description are sampled at a predetermined sampling period, and are quantized by the A / D converter. It has become.

総損失量算出部15は音響側帰還利得α及び回線側帰還利得βの各推定値α'、β'から所望の利得余裕MGを得るために必要な総損失量Ltを算出し、その値Ltを挿入損失量分配処理部16に出力する。   The total loss calculation unit 15 calculates a total loss Lt necessary to obtain a desired gain margin MG from the estimated values α ′ and β ′ of the acoustic feedback gain α and the line feedback gain β, and the value Lt Is output to the insertion loss amount distribution processing unit 16.

挿入損失量分配処理部16では、送話側損失挿入部11の入出力信号及び受話側損失挿入部12の入出力信号を監視し、これらの信号のパワーレベルの大小関係並びに音声信号の有無などの情報から通話状態(受話状態、送話状態等)を判定する。さらに挿入損失量分配処理部16は、判定した通話状態に応じた割合で総損失量Ltを送話側損失挿入部11と受話側損失挿入部12に分配するように各損失挿入部11,12の挿入損失量を調整する。   The insertion loss amount distribution processing unit 16 monitors the input / output signals of the transmission side loss insertion unit 11 and the input / output signals of the reception side loss insertion unit 12, and the magnitude relationship between the power levels of these signals and the presence or absence of a voice signal, etc. The call state (receiving state, transmitting state, etc.) is determined from the information. Furthermore, the insertion loss amount distribution processing unit 16 distributes the total loss amount Lt to the transmission side loss insertion unit 11 and the reception side loss insertion unit 12 at a rate corresponding to the determined call state. Adjust the amount of insertion loss.

ところで本実施形態における総損失量算出部15は、更新モードと固定モードの2つの動作モードを有している。更新モードとは、上述のように帰還利得推定部14が推定する各帰還利得α、βの推定値α'、β'に基づいて閉ループに挿入すべき損失量の総和を算出して適応更新する動作モードである。一方、固定モードとは、総損失量を所定の初期値に固定する動作モードである。そして、総損失量算出部15は、相手側の通話端末との通話開始から第1及び第2のエコーキャンセラ30A,30Bの適応フィルタ31A,31Bにおけるフィルタ係数が十分に収束するまでの期間には固定モードで動作する。そして、フィルタ係数が十分に収束した後の期間に、総損失量算出部15は更新モードで動作する。すなわち、総損失量算出部15では音響側帰還利得α及び回線側帰還利得βの推定値α'、β'がともに通話開始から所定時間(数百ミリ秒)以上継続して所定のしきい値ε(例えば、通話開始時における各推定値α'、β'に対して10〜15dB小さい値)を下回った時点でフィルタ係数が十分に収束したものとみなす。そして、総損失量算出部15は、上記時点以前には総損失量を初期値に固定する固定モードで動作し、上記時点以降には各推定値α'、β'に基づいて総損失量を適応更新する更新モードに動作モードを切り換える。なお、固定モードにおける総損失量の初期値は更新モードにおいて随時更新される総損失量よりも十分に大きな値に設定される。   By the way, the total loss amount calculation unit 15 in the present embodiment has two operation modes, an update mode and a fixed mode. In the update mode, as described above, the sum of the loss amounts to be inserted into the closed loop is calculated and adaptively updated based on the estimated values α ′ and β ′ of the feedback gains α and β estimated by the feedback gain estimation unit 14 as described above. It is an operation mode. On the other hand, the fixed mode is an operation mode in which the total loss amount is fixed to a predetermined initial value. Then, the total loss calculation unit 15 performs a period from the start of the call with the other party's call terminal until the filter coefficients in the adaptive filters 31A and 31B of the first and second echo cancellers 30A and 30B sufficiently converge. Operates in fixed mode. Then, during the period after the filter coefficients have sufficiently converged, the total loss calculation unit 15 operates in the update mode. That is, in the total loss amount calculation unit 15, the estimated values α ′ and β ′ of the acoustic side feedback gain α and the line side feedback gain β both continue for a predetermined time (several hundred milliseconds) from the start of the call for a predetermined threshold value. It is considered that the filter coefficient has sufficiently converged when it falls below ε (for example, a value that is 10 to 15 dB smaller than the estimated values α ′ and β ′ at the start of the call). The total loss amount calculation unit 15 operates in a fixed mode in which the total loss amount is fixed to the initial value before the time point, and after the time point, the total loss amount is calculated based on the estimated values α ′ and β ′. Switch the operation mode to the update mode for adaptive update. Note that the initial value of the total loss amount in the fixed mode is set to a value sufficiently larger than the total loss amount updated as needed in the update mode.

而して、通話開始直後のフィルタ係数が十分に収束していない状態においては、十分に大きな値に設定される初期値の総損失量が閉ループに挿入されるため、不快なエコーやハウリングの発生を抑制して安定した半二重通話を実現することができる。また、通話開始から時間が経過してフィルタ係数が十分に収束した状態においては、総損失量算出部15の動作モードが更新モードに切り換わり、閉ループに挿入される総損失量が初期値よりも十分に低い値に減少するため、双方向の同時通話が実現できる。   Thus, when the filter coefficient immediately after the start of the call is not sufficiently converged, the initial total loss amount set to a sufficiently large value is inserted into the closed loop, resulting in unpleasant echoes and howling. Stable half-duplex call can be realized by suppressing the above. In addition, when the time has elapsed since the start of the call and the filter coefficient has sufficiently converged, the operation mode of the total loss calculation unit 15 switches to the update mode, and the total loss inserted in the closed loop is less than the initial value. Since it decreases to a sufficiently low value, two-way simultaneous calls can be realized.

次に、帰還利得推定部14について、更に詳しく説明する。図1は、挿入損失量制御部13のうち、特に帰還利得推定部14の構成を詳細に示したブロック図である。帰還利得推定部14は、音響側帰還利得αを推定する音響側帰還利得推定部14Aと、回線側帰還利得βを推定する回線側帰還利得推定部14Bを有する。   Next, the feedback gain estimation unit 14 will be described in more detail. FIG. 1 is a block diagram showing in detail the configuration of the feedback gain estimation unit 14 in the insertion loss amount control unit 13. The feedback gain estimation unit 14 includes an acoustic side feedback gain estimation unit 14A that estimates the acoustic side feedback gain α, and a line side feedback gain estimation unit 14B that estimates the line side feedback gain β.

音響側帰還利得推定部14Aおよび回線側帰還利得推定部14Bは、ある時刻nにおいて、送話路上の送話側損失挿入部11の入力点Tinおよび出力点Tout、受話路上の受話側損失挿入部12の入力点Rinおよび出力点Routからそれぞれ送話信号並びに受話信号Tin(n)、Tout(n)、Rin(n)、Rout(n)を取り込み、音響側帰還利得α(n)および回線側帰還利得β(n)に対する推定値α'(n)、β'(n)を出力して、総損失量算出部15へ渡す。   The acoustic side feedback gain estimator 14A and the line side feedback gain estimator 14B are configured such that, at a certain time n, the input point Tin and the output point Tout of the transmission side loss insertion unit 11 on the transmission path, and the reception side loss insertion unit on the reception path The transmission signal and reception signal Tin (n), Tout (n), Rin (n), Rout (n) are taken in from the 12 input points Rin and output points Rout, respectively, and the acoustic feedback gain α (n) and line side Estimated values α ′ (n) and β ′ (n) for the feedback gain β (n) are output and passed to the total loss calculating unit 15.

音響側帰還利得推定部14Aは、受話側損失挿入部12の出力点Routから参照した参照信号(受話信号)を処理する受話側ブロックと、送話側損失挿入部11の入力点Tinから参照した参照信号(送話信号)を処理する送話側ブロックと、推定値演算部54Aとを有する。送話側ブロックは、参照信号記憶部52C、周波数帯域別信号レベル算出部53Cなどで構成される。参照信号記憶部52Cは、参照した参照信号を記憶し、周波数帯域別信号レベル算出部53Cは、互いに異なる周波数帯域を通過帯域とする複数の帯域通過フィルタ(図示せず)を有し、参照信号記憶部52Cから読み取った参照信号を複数の周波数帯域に弁別することで周波数帯域別の信号レベルを求める。   The acoustic-side feedback gain estimation unit 14A is referred to from the reception-side block that processes the reference signal (reception signal) referenced from the output point Rout of the reception-side loss insertion unit 12, and from the input point Tin of the transmission-side loss insertion unit 11. It has a transmission side block for processing a reference signal (transmission signal), and an estimated value calculation unit 54A. The transmission side block includes a reference signal storage unit 52C, a signal level calculation unit 53C for each frequency band, and the like. The reference signal storage unit 52C stores the referenced reference signal, and the signal level calculation unit 53C for each frequency band includes a plurality of bandpass filters (not shown) having different frequency bands as passbands. The signal level for each frequency band is obtained by discriminating the reference signal read from the storage unit 52C into a plurality of frequency bands.

また受話側ブロックは、伝達時間差補正部51A、参照信号記憶部52A、周波数帯域別信号レベル算出部53Aなどで構成される。伝達時間差補正部51Aは、音響側エコー経路HACを含めた音響側帰還経路が固有にもつ信号伝達時間の差を補正する。参照信号記憶部52Aは、信号伝達時間の差が補正された後の参照信号(受話信号)を記憶する。周波数帯域別信号レベル算出部53Aは、互いに異なる周波数帯域を通過帯域とする複数の帯域通過フィルタ(図示せず)を有し、参照信号記憶部52Aから読み取った参照信号を複数の周波数帯域に弁別することで周波数帯域別の信号レベルを求める。 The receiving side block includes a transmission time difference correction unit 51A, a reference signal storage unit 52A, a signal level calculation unit 53A for each frequency band, and the like. Transmission time difference correction unit 51A includes acoustic side feedback path including the acoustical side echo path H AC corrects the difference in signal transmission time with unique. The reference signal storage unit 52A stores the reference signal (received signal) after the signal transmission time difference is corrected. The signal level calculation unit 53A for each frequency band has a plurality of band pass filters (not shown) whose pass bands are different frequency bands, and discriminates the reference signal read from the reference signal storage unit 52A into a plurality of frequency bands. Thus, the signal level for each frequency band is obtained.

推定値演算部54Aは、周波数帯域別信号レベル算出部53Cで算出した周波数帯域別の信号レベルと、周波数帯域別信号レベル算出部53Aで算出した周波数帯域別の信号レベルとの比率から各周波数帯域毎の音響側帰還利得の推定値α’を演算する。そして、推定値演算部54Aで演算された各周波数帯域毎の音響側帰還利得の推定値α’が総損失量算出部15に入力される。   The estimated value calculation unit 54A calculates each frequency band from the ratio between the signal level for each frequency band calculated by the signal level calculation unit for each frequency band 53C and the signal level for each frequency band calculated by the signal level calculation unit for each frequency band 53A. The estimated value α ′ of the acoustic feedback gain for each is calculated. Then, the estimated value α ′ of the acoustic feedback gain for each frequency band calculated by the estimated value calculating unit 54A is input to the total loss calculating unit 15.

一方、回線側帰還利得推定部14Bは、送話側損失挿入部11の出力点Toutから参照した参照信号(送話信号)を処理する送話側ブロックと、受話側損失挿入部12の入力点Rinから参照した参照信号(受話信号)を処理する受話側ブロックと、推定値演算部54Bとを有する。受話側ブロックは、参照信号記憶部52D、周波数帯域別信号レベル算出部53Dなどで構成される。参照信号記憶部52Dは参照した参照信号を記憶する。周波数帯域別信号レベル算出部53Dは、互いに異なる周波数帯域を通過帯域とする複数の帯域通過フィルタ(図示せず)を有し、参照信号記憶部52Dから読み取った参照信号を複数の周波数帯域に弁別することで周波数帯域別の信号レベルを求める。   On the other hand, the line-side feedback gain estimator 14B includes a transmission side block that processes a reference signal (transmission signal) referenced from the output point Tout of the transmission side loss insertion unit 11, and an input point of the reception side loss insertion unit 12. A receiving side block that processes a reference signal (received signal) referenced from Rin, and an estimated value calculation unit 54B. The receiving side block includes a reference signal storage unit 52D, a signal level calculation unit 53D for each frequency band, and the like. The reference signal storage unit 52D stores the referenced reference signal. The signal level calculation unit 53D for each frequency band has a plurality of band pass filters (not shown) having different frequency bands as pass bands, and discriminates the reference signal read from the reference signal storage unit 52D into a plurality of frequency bands. Thus, the signal level for each frequency band is obtained.

また送話側ブロックは、伝達時間差補正部51B、参照信号記憶部52B、周波数帯域別信号レベル算出部53Bなどで構成される。伝達時間差補正部51Bは、回線側エコー経路HLINを含めた回線側帰還経路が固有にもつ信号伝達時間の差を補正する。参照信号記憶部52Bは、信号伝達時間の差が補正された後の参照信号(送話信号)を記憶する。周波数帯域別信号レベル算出部53Bは、互いに異なる周波数帯域を通過帯域とする複数の帯域通過フィルタ(図示せず)を有し、参照信号記憶部52Bから読み取った参照信号を複数の周波数帯域に弁別することで周波数帯域別の信号レベルを求める。 The transmission side block includes a transmission time difference correction unit 51B, a reference signal storage unit 52B, a signal level calculation unit 53B for each frequency band, and the like. The transmission time difference correction unit 51B corrects the difference in signal transmission time inherent in the line-side feedback path including the line-side echo path HLIN . The reference signal storage unit 52B stores the reference signal (transmission signal) after the difference in signal transmission time is corrected. The signal level calculation unit 53B for each frequency band has a plurality of band pass filters (not shown) having different frequency bands as pass bands, and discriminates the reference signal read from the reference signal storage unit 52B into a plurality of frequency bands. Thus, the signal level for each frequency band is obtained.

推定値演算部54Bは、周波数帯域別信号レベル算出部53Dで算出した周波数帯域別の信号レベルと、周波数帯域別信号レベル算出部53Cで算出した周波数帯域別の信号レベルとの比率から各周波数帯域毎の回線側帰還利得の推定値β’を演算する。そして、推定値演算部54Bで演算された各周波数帯域毎の回線側帰還利得の推定値β’が総損失量算出部15に入力される。なお、音声スイッチ10に含まれる上記各部はDSPのハードウェアを専用のプログラムで制御することによって実現されるものであり、音響側並びに回線側の各帰還利得推定部14A,14Bにおいて処理される信号は全てアナログの送話信号及び受話信号をサンプリングし且つ量子化したディジタルのデータとして扱われる。   The estimated value calculation unit 54B calculates each frequency band from the ratio between the signal level for each frequency band calculated by the signal level calculation unit 53D for each frequency band and the signal level for each frequency band calculated by the signal level calculation unit 53C for each frequency band. The estimated value β ′ of the line-side feedback gain for each is calculated. Then, the estimated value β ′ of the line-side feedback gain for each frequency band calculated by the estimated value calculating unit 54B is input to the total loss calculating unit 15. The above-described units included in the voice switch 10 are realized by controlling DSP hardware with a dedicated program, and are processed by the feedback gain estimation units 14A and 14B on the acoustic side and the line side. Are all treated as digital data obtained by sampling and quantizing analog transmission signals and reception signals.

ここで、推定値演算部54A,54Bでは、周波数帯域別信号レベル算出部53A〜53Dで算出した信号レベルから周波数帯域別の帰還利得の推定値を演算する際、参照信号の信号レベルが所定のしきい値以下となる周波数帯域の帰還利得の推定値を演算しないことが好ましい。すなわち、周囲騒音(背景ノイズ)などのノイズ成分に対して帰還利得の推定処理を行うと、実際の帰還利得とかけ離れた推定値が算出されてしまう可能性があるので、このようなノイズ成分に対して帰還利得の推定を行わないことで、帰還利得の推定精度の向上が図れる。   Here, in the estimated value calculation units 54A and 54B, when calculating the estimated value of the feedback gain for each frequency band from the signal level calculated by the signal level calculation units 53A to 53D for each frequency band, the signal level of the reference signal is a predetermined level. It is preferable not to calculate the estimated value of the feedback gain in the frequency band that is equal to or lower than the threshold value. In other words, if feedback gain estimation processing is performed on noise components such as ambient noise (background noise), an estimated value far from the actual feedback gain may be calculated. On the other hand, the estimation accuracy of the feedback gain can be improved by not estimating the feedback gain.

受話側損失挿入部12の出力点Routから出力された受話信号が音響エコー経路HACを含めた音響側帰還経路を経て送話側損失挿入部11の入力点Tinへ到達するまでにはその系固有の伝達時間が必要である。そのため伝達時間差補正部51Aでは、受話側損失挿入部12の出力点Routから発生させた単一パルスが送話側損失挿入部11の入力点Tinへ到達する時間を測定するなどして予め設定しておいたその系の伝達時間Dαだけ、受話側損失挿入部12の出力点Routからの参照信号を遅延させる。例えば、参照信号のサンプリング周波数を8[kHz]、測定した遅延時間が12[msec]の場合、8×12=96データ分の遅延処理用信号記憶部(参照信号記憶部52A〜52Dとは別のFIFO(First In First Out)型信号記憶部)を用意しておく。そして、時刻nにおいて、遅延処理用信号記憶部(図示せず)で最も古い12[msec]前のデータをDRout(n)として参照信号記憶部52Aに渡すとともに、受話側損失挿入部12の出力点Routから参照した参照信号Rout(n)を遅延処理用信号記憶部に新しく蓄積することで信号遅延を実現している。 Its system until receiving the signal output from the output point Rout of the receiving-side loss insertion portion 12 reaches the input point Tin of the transmitting end losses insertion portion 11 via the acoustic side feedback path including an acoustic echo path H AC Inherent transmission time is required. Therefore, the transmission time difference correction unit 51A is set in advance by measuring the time for the single pulse generated from the output point Rout of the receiving side loss insertion unit 12 to reach the input point Tin of the transmission side loss insertion unit 11. only the system transmission time D alpha of which had been delays the reference signal from the output point Rout of the receiving-side loss insertion portion 12. For example, when the sampling frequency of the reference signal is 8 [kHz] and the measured delay time is 12 [msec], a delay processing signal storage unit for 8 × 12 = 96 data (separate from the reference signal storage units 52A to 52D). FIFO (First In First Out) type signal storage unit) is prepared. At time n, the oldest 12 [msec] previous data in the delay processing signal storage unit (not shown) is passed to the reference signal storage unit 52A as DRout (n), and the output of the receiving side loss insertion unit 12 is output. The signal delay is realized by newly accumulating the reference signal Rout (n) referenced from the point Rout in the delay processing signal storage unit.

ただし、第1及び第2のエコーキャンセラ30A,30Bが伝達時間(伝達時間差)Dαを検出する手段を有している場合、伝達時間差補正部51A,51Bは、エコーキャンセラ30A,30Bで検出される伝達時間差Dαを用いて伝達時間差の補正処理を行うことが好ましい。なお、エコーキャンセラ30A,30Bで伝達時間差Dαを検出する方法としては、適応フィルタ31A,31Bにおけるフィルタ係数の値が最も大きくなるフィルタ次数が伝達時間差Dαと考えられるので、そのフィルタ次数から伝達時間差Dαを求める方法がある。 However, the first and second echo canceller 30A, if 30B has a means for detecting a transmission time (difference transmission time) D alpha, transmission time difference correction unit 51A, 51B is detected echo canceller 30A, at 30B that it is preferable to perform the correction processing of the transmission time difference by using a transfer time difference D alpha. Incidentally, an echo canceller 30A, as a method for detecting the transmission time difference D alpha in 30B, the adaptive filter 31A, since the filter order of the value of the filter coefficient is largest is considered to transmission time difference D alpha in 31B, transmitted from the filter order a method for determining the time difference D alpha.

ここで、特許文献1記載の従来例における周波数帯域別信号レベル算出部の信号レベルの算出方法を、一度に処理するデータ数(離散フーリエ変換処理の長さ)をNf8とし、図3のフローチャートを参照しながら説明する。周波数帯域別信号レベル算出部は、時系列に参照された複数(例えば、8個)の参照信号データDRout(n),DRout(n-1),…,DRout(n-7)を参照信号記憶部から読み込み(ステップ1)、読み込んだデータに対して離散フーリエ変換処理を行う(ステップ2)。この離散フーリエ変換処理においては4つに分けた周波数帯域[f0,f1,f2,f3]毎に実部と虚部を個別に演算し(ステップ3)、さらに実部と虚部の二乗和の平方根を演算することで各周波数帯域毎の信号レベル|FRout_f0(n)|,|FRout_f1(n)|,|FRout_f2(n)|,|FRout_f3(n)|を算出する(ステップ4)。すなわち、離散フーリエ変換は周波数成分について偶関数となるため、その長さNf=8の場合であれば半分の4成分を演算すれば十分であるから、参照信号のサンプリング周波数が8[kHz]でNf=8の場合、f0:0〜0.5[kHz]、f1:0.5〜1.5[kHz]、f2:1.5〜2.5[kHz]、f3:2.5〜3.5[kHz]の4つの周波数帯域成分を演算するようにしている。なお、図3のステップ2においては参照信号データDRout(n),DRout(n-1),…,DRout(n-7)の行列と周波数帯域毎の信号レベル|FRout_f0(n)|,|FRout_f1(n)|,|FRout_f2(n)|,|FRout_f3(n)|の行列との変換の関係式を4×8の行列Fで表している。そして、ステップ2の行列演算処理を行った後、[f0,f1,f2,f3]の帯域成分毎に、実部成分と虚部成分をそれぞれ2乗したもの(ステップ3)を加算し、さらに平方根演算処理を行う(ステップ4)ことで帯域成分毎の信号レベルの大きさ[|FRout_f0(n)|,|FRout_f1(n)|,|FRout_f2(n)|,|FRout_f3(n)|]を求めてエンベローブ部に出力する。 Here, in the signal level calculation method of the signal level calculation unit for each frequency band in the conventional example described in Patent Document 1, the number of data to be processed at one time (the length of the discrete Fourier transform process) is Nf8, and the flowchart of FIG. The description will be given with reference. The signal level calculation unit for each frequency band stores a plurality of (for example, eight) reference signal data DRout (n), DRout (n-1), ..., DRout (n-7) referenced in time series as reference signals. (Step 1), and a discrete Fourier transform process is performed on the read data (step 2). In this discrete Fourier transform process, the real part and the imaginary part are individually calculated for each of the four frequency bands [f0, f1, f2, f3] (step 3), and the square sum of the real part and the imaginary part is further calculated. Calculate the signal level for each frequency band | F Rout _f0 (n) |, | F Rout _f1 (n) |, | F Rout _f2 (n) |, | F Rout _f3 (n) | (Step 4). That is, since the discrete Fourier transform is an even function with respect to frequency components, if the length Nf = 8, it is sufficient to calculate half of the four components, so that the sampling frequency of the reference signal is 8 [kHz]. When Nf = 8, four frequency band components are calculated: f0: 0 to 0.5 [kHz], f1: 0.5 to 1.5 [kHz], f2: 1.5 to 2.5 [kHz], f3: 2.5 to 3.5 [kHz] I am doing so. 3, reference signal data DRout (n), DRout (n−1),..., DRout (n−7) matrix and signal level | F Rout — f0 (n) | | F Rout — f1 (n) |, | F Rout — f2 (n) |, | F Rout — f3 (n) | Then, after performing the matrix calculation process of step 2, for each band component of [f0, f1, f2, f3], add the square of the real part component and the imaginary part component (step 3), and By performing the square root processing (step 4), the magnitude of the signal level for each band component [| F Rout _f0 (n) |, | F Rout _f1 (n) |, | F Rout _f2 (n) |, | F Rout _f3 (n) |] the asking and outputs it to the envelope section.

ステップ4の平方根演算処理について、一般的に行われているニュートン・ラフソン(Newton-Rapson)法などのループ演算を繰り返す毎に値が真値に漸近していくアルゴリズムでは、2入力×4帯域=8個の各平方根演算においてループ演算を実行せねばならず、帰還利得推定部を実現しているDSPの演算処理の負担が大きい。   With respect to the square root calculation process in step 4, in an algorithm in which the value gradually approaches a true value every time a loop operation such as a Newton-Rapson method that is generally performed is repeated, 2 inputs × 4 bands = A loop operation must be executed in each of the eight square root operations, and the processing load of the DSP realizing the feedback gain estimation unit is heavy.

これに対して本実施形態における周波数帯域別信号レベル算出部53A〜53Dは、互いに異なる周波数帯域を通過帯域とする複数の帯域通過フィルタによって参照信号を複数の周波数帯域に弁別している。ここで、帯域通過フィルタは、2次のIIRフィルタを用いて実現でき、しかも、2次のIIRフィルタに必要な処理は積和演算のみであるから、従来例の上記離散フーリエ変換処理と比較して、DSPの演算処理負荷が低減可能であることは明らかである。   On the other hand, the signal level calculation units 53A to 53D for each frequency band in the present embodiment discriminate the reference signal into a plurality of frequency bands by using a plurality of band pass filters having different frequency bands as pass bands. Here, the band-pass filter can be realized using a second-order IIR filter, and the processing necessary for the second-order IIR filter is only a product-sum operation. It is clear that the DSP processing load can be reduced.

上述のように本実施形態では、周波数帯域別信号レベル算出部53A〜53Dが複数の帯域通過フィルタによって参照信号を複数の周波数帯域に弁別しているので、従来例と比較して製造コストの上昇を抑えつつ帰還利得の推定精度の向上を図ることができる。   As described above, in the present embodiment, the signal level calculation units 53A to 53D for each frequency band discriminate the reference signal into a plurality of frequency bands by a plurality of band pass filters, so that the manufacturing cost is increased compared to the conventional example. It is possible to improve the estimation accuracy of the feedback gain while suppressing it.

また相手側の通話端末と本機とで話者がほぼ同時に話すことにより音響エコー経路HACにおいて音声成分が印加される状態、いわゆるダブルトーク状態においては、定常的なノイズ信号が存在する場合と同様に帰還利得の推定精度が低下することになる。そこで本実施形態では、第1エコーキャンセラ30Aが有するダブルトーク検出機能によってダブルトーク状態を検出した場合、周波数帯域別信号レベル算出部53Aは、帯域通過フィルタのフィルタ処理を中止して前回のフィルタ処理で得られた信号レベルで代用する。したがって、上述のようにダブルトーク状態において帰還利得を推定することに起因した推定精度の低下を防ぐことができる。 The state the speaker at the other end of the call terminal and the unit audio component is applied in an acoustic echo path H AC by speaking almost simultaneously, in the so-called double-talk state, and if the stationary noise signal is present Similarly, the estimation accuracy of the feedback gain is lowered. Therefore, in this embodiment, when the double talk state is detected by the double talk detection function of the first echo canceller 30A, the signal level calculation unit 53A for each frequency band stops the filter processing of the band pass filter and performs the previous filter processing. Substitute the signal level obtained in step 1 above. Therefore, it is possible to prevent a decrease in estimation accuracy caused by estimating the feedback gain in the double talk state as described above.

ここで、帰還利得推定部14をDSPで構成する場合、参照信号をA/D変換する際のサンプリング時間毎に音響側及び回線側の各帰還利得推定値を求める割込処理を行うことはDSPの演算処理の負担がかなり大きくなってしまう。そこで、DSPでA/Dサンプリング時間毎に発生させている割り込み処理に対して、本実施形態では音響側帰還利得部14Aと回線側帰還利得部14Bを交互に動作させてDSPへの処理負荷を低減している。具体的には、伝送時間差補正部51及び参照信号記憶部52は常に動作させておき、周波数帯域別信号レベル算出部54を割込発生毎に排他的に動作/停止させる。   Here, when the feedback gain estimator 14 is configured by a DSP, interrupt processing for obtaining each feedback gain estimate value on the acoustic side and the line side is performed at each sampling time when the reference signal is A / D converted. The calculation processing burden is considerably increased. Therefore, in contrast to the interrupt processing generated at every A / D sampling time in the DSP, in this embodiment, the acoustic side feedback gain unit 14A and the line side feedback gain unit 14B are alternately operated to reduce the processing load on the DSP. Reduced. Specifically, the transmission time difference correction unit 51 and the reference signal storage unit 52 are always operated, and the signal level calculation unit 54 for each frequency band is operated / stopped exclusively every time an interrupt occurs.

ところで、総損失量算出部15は、帰還利得推定部14で推定される各周波数帯域毎の帰還利得の推定値のうちで最もレベルの高い周波数帯域の帰還利得推定値を選択し、選択した帰還利得推定値から損失量の総和を算出する。このとき、総損失量算出部15では、選択した帰還利得推定値に所定のマージンを加算して損失量の総和を算出することが好ましい。このようにすれば、帰還利得の推定値が実際の帰還利得よりも小さい値であった場合でも、ハウリングの発生を抑制することができる。   By the way, the total loss calculation unit 15 selects the feedback gain estimation value of the highest frequency band among the feedback gain estimation values for each frequency band estimated by the feedback gain estimation unit 14, and selects the selected feedback Calculate the total loss from the gain estimate. At this time, the total loss amount calculation unit 15 preferably calculates a total loss amount by adding a predetermined margin to the selected feedback gain estimation value. In this way, howling can be suppressed even when the estimated value of the feedback gain is smaller than the actual feedback gain.

なお、総損失量算出部15は、帰還利得の推定値が求められた各周波数帯域毎に総損失量の総和を算出してもよい。このとき、挿入損失量分配処理部16は、送話側損失挿入部11及び受話側損失挿入部12の各挿入損失量の配分を周波数帯域毎に決定する。このようにすれば、帰還利得の推定値が小さい周波数帯域に挿入する損失量を少なくできる。   Note that the total loss amount calculation unit 15 may calculate the total sum of the total loss amounts for each frequency band for which the estimated value of the feedback gain is obtained. At this time, the insertion loss amount distribution processing unit 16 determines the distribution of the insertion loss amounts of the transmission side loss insertion unit 11 and the reception side loss insertion unit 12 for each frequency band. In this way, it is possible to reduce the amount of loss inserted into the frequency band where the estimated feedback gain value is small.

ところで、第1エコーキャンセラ30Aは音響エコーを抑制するものであるから、音声スイッチ10が送話状態のときに動作する必要は無い。同様に、第2エコーキャンセラ30Bは回線エコーを抑制するものであるから、音声スイッチ10が受話状態のときに動作する必要は無い。したがって、第1エコーキャンセラ30Aの適応フィルタ31A、並びに第2エコーキャンセラ30Bの適応フィルタ31Bは、挿入損失量分配処理部16が決定する挿入損失量の配分(音声スイッチ10の受話状態、送話状態)に応じて、フィルタ係数を更新するタイミングを調整することが好ましい。つまり、挿入損失量分配処理部16が送話側損失挿入部11の挿入損失量の配分を多くしている場合(音声スイッチ10が受話状態の場合)、第1エコーキャンセラ30Aの適応フィルタ31Aがフィルタ係数を更新し、第2エコーキャンセラ30Bの適応フィルタ31Bはフィルタ係数を更新しなければよい。一方、挿入損失量分配処理部16が受話側損失挿入部12の挿入損失量の配分を多くしている場合(音声スイッチ10が送話状態の場合)、第1エコーキャンセラ30Aの適応フィルタ31Aがフィルタ係数を更新せず、第2エコーキャンセラ30Bの適応フィルタ31Bがフィルタ係数を更新すればよい。   Incidentally, since the first echo canceller 30A suppresses acoustic echoes, it is not necessary to operate when the voice switch 10 is in the transmission state. Similarly, since the second echo canceller 30B suppresses line echo, there is no need to operate when the voice switch 10 is in the receiving state. Therefore, the adaptive filter 31A of the first echo canceller 30A and the adaptive filter 31B of the second echo canceller 30B distribute the insertion loss amount determined by the insertion loss amount distribution processing unit 16 (the reception state and transmission state of the voice switch 10). ), It is preferable to adjust the timing for updating the filter coefficient. That is, when the insertion loss amount distribution processing unit 16 increases the distribution of the insertion loss amount of the transmission side loss insertion unit 11 (when the voice switch 10 is in the receiving state), the adaptive filter 31A of the first echo canceller 30A is The filter coefficient is updated, and the adaptive filter 31B of the second echo canceller 30B may not update the filter coefficient. On the other hand, when the insertion loss amount distribution processing unit 16 increases the distribution of the insertion loss amount of the reception side loss insertion unit 12 (when the voice switch 10 is in the transmission state), the adaptive filter 31A of the first echo canceller 30A is The adaptive filter 31B of the second echo canceller 30B may update the filter coefficient without updating the filter coefficient.

上述のように第1及び第2のエコーキャンセラ30A,30Bが音声スイッチ10の状態に連動してフィルタ係数の更新を行えば、音声スイッチ10の状態に関わらずにフィルタ係数を更新する場合と比較して、フィルタ係数の収束精度の向上やDSPの処理負荷の低減を図ることができる。   As described above, if the first and second echo cancellers 30A and 30B update the filter coefficient in conjunction with the state of the voice switch 10, the filter coefficient is updated regardless of the state of the voice switch 10. As a result, it is possible to improve the convergence accuracy of the filter coefficients and reduce the processing load of the DSP.

ところで、本実施形態の音声スイッチ10は、受話状態と送話状態と中立状態の3種類の状態に切り替えている。中立状態とは、受話状態又は送話状態の何れでもない状態であって、挿入損失量分配処理部15が送話側損失挿入部11と受話側損失挿入部12に同じ損失量を配分している状態をいう。そして、第1及び第2のエコーキャンセラ30A,30Bの各適応フィルタ31A,31Bは、フィルタ係数を更新している状態(受話状態又は送話状態)から中立状態に音声スイッチ10が切り替わった時点から一定期間が経過するまではフィルタ係数の更新を継続することが好ましい。このようにすれば、音声スイッチ10の状態が頻繁に切り替わる状況においても、適応フィルタ31A,31Bがフィルタ係数を更新し、第1及び第2のエコーキャンセラ30A,30Bが適切にエコーを抑制することができる。   By the way, the voice switch 10 of the present embodiment is switched to three types of states, that is, a reception state, a transmission state, and a neutral state. The neutral state is a state that is neither a reception state nor a transmission state, and the insertion loss amount distribution processing unit 15 distributes the same loss amount to the transmission side loss insertion unit 11 and the reception side loss insertion unit 12. The state that is. Then, each adaptive filter 31A, 31B of the first and second echo cancellers 30A, 30B starts from the time when the voice switch 10 is switched from the state in which the filter coefficient is updated (the reception state or the transmission state) to the neutral state. It is preferable to continue to update the filter coefficient until a certain period has elapsed. In this way, even when the state of the voice switch 10 is frequently switched, the adaptive filters 31A and 31B update the filter coefficients, and the first and second echo cancellers 30A and 30B appropriately suppress echoes. Can do.

13 挿入損失量制御部
14 帰還利得推定部
15 総損失量算出部
51A,51B 伝達時間差補正部
53A〜53D 周波数帯域別信号レベル算出部
13 Insertion loss control unit
14 Feedback gain estimator
15 Total loss calculator
51A, 51B Transmission time difference correction unit
53A to 53D Signal level calculator for each frequency band

Claims (9)

マイクロホン及びスピーカと、相手側の通話端末から送られてくる受話信号を前記スピーカに伝送する受話側信号経路並びに前記マイクロホンで集音された送話信号を伝送して前記相手側の通話端末へ送る送話側信号経路に損失を挿入することで通話状態を受話及び送話に切り換える音声スイッチと、前記マイクロホンとスピーカの音響結合によって生じる音響エコーを抑制するエコーキャンセラとを備えており、前記音声スイッチは、送話側の信号経路に損失を挿入する送話側損失挿入部と、受話側の信号経路に損失を挿入する受話側損失挿入部と、送話側及び受話側の前記各損失挿入部から挿入する損失量を制御する挿入損失量制御部とを具備し、前記挿入損失量制御部は、前記受話側損失挿入部の出力点から音響エコー経路を介して前記送話側損失挿入部の入力点へ帰還する経路の音響側帰還利得を推定するとともに、前記送話側損失挿入部の出力点から回線エコー経路を介して前記受話側損失挿入部の入力点へ帰還する経路の回線側帰還利得を推定する帰還利得推定部と、音響側及び回線側の各帰還利得の推定値に基づいて閉ループに挿入すべき損失量の総和を算出する総損失量算出部と、送話信号及び受話信号を監視して通話状態を推定し、この推定結果と前記総損失量算出部の算出値に応じて前記送話側損失挿入部及び前記受話側損失挿入部の各挿入損失量の配分を決定する挿入損失量分配処理部とを有する拡声通話装置において、
前記帰還利得推定部は、音響側及び回線側の各帰還経路が固有にもつ信号伝達時間の差を補正する伝達時間差補正部と、前記伝達時間差補正部で補正された後の信号を複数の周波数帯域に弁別することにより周波数帯域別の信号レベルを求める周波数帯域別信号レベル算出部と、前記周波数帯域別信号レベル算出部で算出した信号レベルから周波数帯域別の帰還利得の推定値を演算する推定値演算部とを備え、前記周波数帯域別信号レベル算出部は、互いに異なる周波数帯域を通過帯域とする複数の帯域通過フィルタによって前記信号を複数の周波数帯域に弁別し、
前記音響エコーを抑制する第1エコーキャンセラと、前記相手側の通話端末における音響結合又は前記回線エコー経路を介した前記送話信号の回り込みによって生じる回線エコーを抑制する第2エコーキャンセラとを備え、前記第1エコーキャンセラは、前記音響エコー経路のインパルス応答を適応的に同定して前記音響エコー経路への入力信号から前記音響エコーを推定する適応フィルタと、前記適応フィルタで推定された前記音響エコーを前記音響エコー経路からの出力信号より減算する減算器とを有し、前記第2エコーキャンセラは、前記回線エコー経路のインパルス応答を適応的に同定して前記回線エコー経路への入力信号から前記回線エコーを推定する適応フィルタと、前記適応フィルタで推定された前記回線エコーを前記回線エコー経路からの出力信号より減算する減算器とを有し、前記第1エコーキャンセラの前記適応フィルタ、並びに前記第2エコーキャンセラの前記適応フィルタは、前記挿入損失量分配処理部が決定する前記挿入損失量の配分に応じて、フィルタ係数を更新するタイミングを調整することを特徴とする拡声通話装置。
A microphone and a speaker, a reception side signal path for transmitting a reception signal sent from the other party's telephone terminal to the speaker, and a transmission signal collected by the microphone are transmitted to the other party's telephone terminal A voice switch for switching a call state to reception and transmission by inserting a loss in a signal path on the transmission side, and an echo canceller for suppressing an acoustic echo generated by acoustic coupling of the microphone and the speaker. A transmission side loss insertion unit that inserts a loss into the signal path on the transmission side, a reception side loss insertion unit that inserts a loss into the signal path on the reception side, and the loss insertion units on the transmission side and the reception side. An insertion loss amount control unit for controlling the loss amount inserted from the reception side loss insertion unit through the acoustic echo path from the output point of the reception side loss insertion unit. With estimating the acoustic side feedback gain of the path to return to the input point of the transmitter-side loss insertion portion, from the output point of the transmitting end losses insertion portion to the input point of the reception-side loss insertion portion via a line echo path A feedback gain estimator that estimates the line-side feedback gain of the feedback path, and a total loss amount calculator that calculates the total amount of loss to be inserted into the closed loop based on the estimated values of the feedback gains on the acoustic side and the line side; , The transmission signal and the reception signal are monitored to estimate the call state, and each insertion of the transmission side loss insertion unit and the reception side loss insertion unit according to the estimation result and the calculated value of the total loss amount calculation unit In a loudspeaker apparatus having an insertion loss amount distribution processing unit for determining a loss amount distribution,
The feedback gain estimation unit includes a transmission time difference correction unit that corrects a difference in signal transmission time inherent in each of the feedback paths on the acoustic side and the line side, and a signal corrected by the transmission time difference correction unit as a plurality of frequencies. A signal level calculation unit for each frequency band for obtaining a signal level for each frequency band by discriminating into bands, and an estimation for calculating an estimated value of the feedback gain for each frequency band from the signal level calculated by the signal level calculation unit for each frequency band A value calculation unit, and the signal level calculation unit for each frequency band discriminates the signal into a plurality of frequency bands by a plurality of band pass filters having different frequency bands as pass bands ,
A first echo canceller that suppresses the acoustic echo, and a second echo canceller that suppresses line echo caused by acoustic coupling at the counterpart telephone terminal or wraparound of the transmission signal via the line echo path, The first echo canceller adaptively identifies an impulse response of the acoustic echo path and estimates the acoustic echo from an input signal to the acoustic echo path, and the acoustic echo estimated by the adaptive filter And a subtractor that subtracts from the output signal from the acoustic echo path, and the second echo canceller adaptively identifies the impulse response of the line echo path and outputs the signal from the input signal to the line echo path. An adaptive filter for estimating line echo, and the line echo estimated by the adaptive filter A subtracter for subtracting from the output signal from the path, and the insertion filter determined by the insertion loss distribution processor determines the adaptive filter of the first echo canceller and the adaptive filter of the second echo canceller A loudspeaker apparatus that adjusts a timing for updating a filter coefficient in accordance with the distribution of the loss amount .
前記帰還利得推定部は、前記音響側帰還利得を推定する処理と、前記回線側帰還利得を推定する処理とを一定時間毎に交互に行うことを特徴とする請求項1記載の拡声通話装置。   2. The loudspeaker apparatus according to claim 1, wherein the feedback gain estimation unit alternately performs the process of estimating the acoustic-side feedback gain and the process of estimating the line-side feedback gain at regular intervals. 前記周波数帯域別信号レベル算出部は、前記エコーキャンセラが帰還利得を推定する経路の途中で音声成分が印加されるダブルトーク状態を検出した場合、前記帯域通過フィルタのフィルタ処理を中止して前回のフィルタ処理で得られた信号レベルで代用することを特徴とする請求項1又は2記載の拡声通話装置。   When the signal level calculation unit for each frequency band detects a double talk state in which a speech component is applied in the course of the path where the echo canceller estimates the feedback gain, the filter processing of the band pass filter is stopped and 3. The loudspeaker apparatus according to claim 1, wherein the signal level obtained by the filtering process is substituted. 前記エコーキャンセラは、音響側及び回線側の各帰還経路が固有にもつ信号伝達時間の差を検出する伝達時間差検出手段を有し、前記伝達時間差補正部は、前記伝達時間差検出手段が検出する伝達時間差を用いて前記伝達時間差の補正処理を行うことを特徴とする請求項1〜3の何れか1項に記載の拡声通話装置。   The echo canceller includes a transmission time difference detection unit that detects a difference in signal transmission time inherent in each of the feedback paths on the acoustic side and the line side, and the transmission time difference correction unit is a transmission detected by the transmission time difference detection unit. The loudspeaker apparatus according to any one of claims 1 to 3, wherein the transmission time difference is corrected using a time difference. 前記推定値演算部は、前記周波数帯域別信号レベル算出部で算出した信号レベルから周波数帯域別の帰還利得を推定する際、前記信号レベルが所定のしきい値以下となる前記周波数帯域の帰還利得の推定値を演算しないことを特徴とする請求項1〜4の何れか1項に記載の拡声通話装置。   The estimated value calculation unit, when estimating the feedback gain for each frequency band from the signal level calculated by the signal level calculation unit for each frequency band, the feedback gain of the frequency band for which the signal level is a predetermined threshold or less The loudspeaker apparatus according to any one of claims 1 to 4, wherein the estimated value is not calculated. 前記総損失量算出部は、前記帰還利得推定部で推定される前記帰還利得の推定値に所定のマージンを加算して前記損失量の総和を算出することを特徴とする請求項1〜5の何れか1項に記載の拡声通話装置。   6. The total loss amount calculation unit according to claim 1, wherein the total loss amount calculation unit calculates a total sum of the loss amounts by adding a predetermined margin to the estimated value of the feedback gain estimated by the feedback gain estimation unit. The loudspeaker apparatus according to any one of the preceding claims. 前記総損失量算出部は、前記帰還利得の推定値が求められた周波数帯域毎に前記総損失量の総和を算出し、前記挿入損失量分配処理部は、前記送話側損失挿入部及び前記受話側
損失挿入部の各挿入損失量の配分を周波数帯域毎に決定することを特徴とする請求項1〜6の何れか1項に記載の拡声通話装置。
The total loss amount calculation unit calculates a total sum of the total loss amounts for each frequency band for which the estimated value of the feedback gain is obtained, and the insertion loss amount distribution processing unit includes the transmission side loss insertion unit and the transmission side loss insertion unit The loudspeaker according to any one of claims 1 to 6, wherein the distribution of each insertion loss amount of the reception side loss insertion unit is determined for each frequency band.
前記第1エコーキャンセラの前記適応フィルタは、前記挿入損失量分配処理部が前記送話側損失挿入部に対して相対的に高い挿入損失量を配分した場合に前記フィルタ係数を更新し、前記第2エコーキャンセラの前記適応フィルタは、前記挿入損失量分配処理部が前記受話側損失挿入部に対して相対的に高い挿入損失量を配分した場合に前記フィルタ係数を更新することを特徴とする請求項1〜7の何れか1項に記載の拡声通話装置。 The adaptive filter of the first echo canceller updates the filter coefficient when the insertion loss amount distribution processing unit distributes a relatively high insertion loss amount to the transmission side loss insertion unit, and The adaptive filter of the 2-echo canceller updates the filter coefficient when the insertion loss amount distribution processing unit distributes a relatively high insertion loss amount to the reception-side loss insertion unit. Item 8. The voice communication device according to any one of Items 1 to 7. 前記第1エコーキャンセラ及び前記第2エコーキャンセラの前記各適応フィルタは、前記フィルタ係数を更新している場合において、前記挿入損失量分配処理部が前記送話側及び前記受話側の各損失挿入部に対して前記挿入損失量を等しく配分する状態に変化してから一定期間が経過するまでは前記フィルタ係数の更新を継続することを特徴とする請求項1〜8の何れか1項に記載の拡声通話装置 When each of the adaptive filters of the first echo canceller and the second echo canceller is updating the filter coefficient, the insertion loss amount distribution processing unit is configured to transmit each loss insertion unit on the transmitting side and the receiving side. the insertion loss equally from changing the allocation to state until a predetermined period has elapsed according to any one of claims 1 to 8, characterized in that to continue updating the filter coefficient with respect to Loudspeaker .
JP2012173385A 2012-08-03 2012-08-03 Loudspeaker Active JP5950199B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2012173385A JP5950199B2 (en) 2012-08-03 2012-08-03 Loudspeaker

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2012173385A JP5950199B2 (en) 2012-08-03 2012-08-03 Loudspeaker

Publications (2)

Publication Number Publication Date
JP2014033372A JP2014033372A (en) 2014-02-20
JP5950199B2 true JP5950199B2 (en) 2016-07-13

Family

ID=50282871

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2012173385A Active JP5950199B2 (en) 2012-08-03 2012-08-03 Loudspeaker

Country Status (1)

Country Link
JP (1) JP5950199B2 (en)

Family Cites Families (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0771026B2 (en) * 1985-03-04 1995-07-31 株式会社東芝 Echo signal canceller
JP3580168B2 (en) * 1999-03-18 2004-10-20 松下電工株式会社 Loudspeaker
JP4483134B2 (en) * 2001-06-11 2010-06-16 パナソニック電工株式会社 Loudspeaker call system
JP3864915B2 (en) * 2003-02-25 2007-01-10 松下電工株式会社 Loudspeaker calling system for apartment houses
JP4371002B2 (en) * 2004-08-05 2009-11-25 パナソニック電工株式会社 Loudspeaker
JP4483761B2 (en) * 2005-10-14 2010-06-16 パナソニック電工株式会社 Loudspeaker
JP2007267043A (en) * 2006-03-28 2007-10-11 Matsushita Electric Works Ltd Speech unit
JP4887181B2 (en) * 2007-02-28 2012-02-29 オンセミコンダクター・トレーディング・リミテッド Echo prevention device and program
JP2009017029A (en) * 2007-07-02 2009-01-22 Sony Corp Device and method for processing sound signal
JP4900184B2 (en) * 2007-10-16 2012-03-21 パナソニック電工株式会社 Loudspeaker
JP5086769B2 (en) * 2007-10-23 2012-11-28 パナソニック株式会社 Loudspeaker
JP5227123B2 (en) * 2008-09-18 2013-07-03 パナソニック株式会社 Intercom device

Also Published As

Publication number Publication date
JP2014033372A (en) 2014-02-20

Similar Documents

Publication Publication Date Title
EP0508847B1 (en) An echo canceller
US8160239B2 (en) Echo canceller and speech processing apparatus
JP5086769B2 (en) Loudspeaker
WO2009070073A1 (en) A method and arrangement for echo cancellation of voice signals
US8077858B2 (en) Echo canceler and echo canceling method
JP2008263336A (en) Echo canceler and residual echo suppressing method thereof
JP4483761B2 (en) Loudspeaker
TWI399968B (en) Speakerphone apparatus
JP4075835B2 (en) Loudspeaker
JP5950199B2 (en) Loudspeaker
JP4650163B2 (en) Loudspeaker
JP4725330B2 (en) Loudspeaker
JP3220979B2 (en) Voice switch
JP4371002B2 (en) Loudspeaker
JP4900185B2 (en) Loudspeaker
JP3580168B2 (en) Loudspeaker
JP2007124162A (en) Loudspeaker call device
JP3244416B2 (en) Echo canceller
JP2013225747A (en) Communication device
JP3941580B2 (en) Loudspeaker
JP4900184B2 (en) Loudspeaker
JP4047867B2 (en) Echo cancel circuit
JP3404236B2 (en) Loudspeaker
JP2000165300A (en) Method and system for identifying unknown system using sub band adaptive filter
JP2007124163A (en) Call apparatus

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20150311

A711 Notification of change in applicant

Free format text: JAPANESE INTERMEDIATE CODE: A711

Effective date: 20150312

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20160209

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20160324

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20160426

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20160526

R151 Written notification of patent or utility model registration

Ref document number: 5950199

Country of ref document: JP

Free format text: JAPANESE INTERMEDIATE CODE: R151