JP2004072178A - Apparatus for determining permissible packet loss rate and program therefor - Google Patents

Apparatus for determining permissible packet loss rate and program therefor Download PDF

Info

Publication number
JP2004072178A
JP2004072178A JP2002224952A JP2002224952A JP2004072178A JP 2004072178 A JP2004072178 A JP 2004072178A JP 2002224952 A JP2002224952 A JP 2002224952A JP 2002224952 A JP2002224952 A JP 2002224952A JP 2004072178 A JP2004072178 A JP 2004072178A
Authority
JP
Japan
Prior art keywords
packet loss
quality
loss rate
video
permissible
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Granted
Application number
JP2002224952A
Other languages
Japanese (ja)
Other versions
JP3860093B2 (en
Inventor
Takaaki Kurita
栗田 孝昭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nippon Telegraph and Telephone Corp
Original Assignee
Nippon Telegraph and Telephone Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nippon Telegraph and Telephone Corp filed Critical Nippon Telegraph and Telephone Corp
Priority to JP2002224952A priority Critical patent/JP3860093B2/en
Publication of JP2004072178A publication Critical patent/JP2004072178A/en
Application granted granted Critical
Publication of JP3860093B2 publication Critical patent/JP3860093B2/en
Anticipated expiration legal-status Critical
Expired - Lifetime legal-status Critical Current

Links

Images

Landscapes

  • Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
  • Data Exchanges In Wide-Area Networks (AREA)

Abstract

<P>PROBLEM TO BE SOLVED: To provide an apparatus for determining a permissible packet loss rate that provides a deteriorated video image permissible rate A, and a packet loss rate AL for a long time viewing with respect to the permissible rate A without the need for a subject to actually evaluate a deteriorated video image for a long period of time. <P>SOLUTION: In the apparatus, deteriorated quality Q of a video image for a unit time (10 seconds) with respect to various packet loss rates L is subjectively evaluated to obtain a relational expression Q=C0+C1×ln(L) in advance, a deterioration occurrence time interval is obtained at which the deteriorated video image with each quality per unit time caused by viewing for long time is evaluated to be permissible with respect to various permissible rate A (percentage of approving subjects to all the subjects) to obtain a relative equation of T=D0+D1×Q in advance, the Q is then obtained (10) from the received rate L, and the T is obtained (20) from the Q and the received rate A, and AL=(L/T) ×U is calculated (30). <P>COPYRIGHT: (C)2004,JPO

Description

【0001】
【発明の属する技術分野】
この発明は、映像信号及び/又は音声や音楽などの音響信号を伝送するサービス・装置の開発時の品質設計において、長時間映像及び/又は音響を視聴する場合の劣化映像及び/又は音響の許容率に対し許容されるパケット損失率を決定する装置及びそのプログラムに関する。
【0002】
【従来の技術】
従来、映像信号、音声信号、音楽信号、映像信号と音声信号又は音楽信号(以下映像及び/又は音響信号と記す)を伝送するシステムでは、伝送路や伝送装置・端末における符号誤りやパケット到着時間揺らぎによってパケット損失が生じ、映像及び/又は音響品質が劣化する場合がある。従来、許容される映像及び/又は音響品質となるパケット損失率を求める手段として、例えば映像信号にパケット損失を生じさせた10秒間の劣化映像を、被験者が主観的に評価する方法がある。また例えばITU−R BT.500−10勧告に基づいて映像の主観品質評価を行なう方法がある。この方法によれば、少なくとも4種類以上の映像シーンを用いて各種損失率の劣化映像を、最小15名以上の被験者が評価することが勧告されており、評価条件数が膨大となる。そのため、上記勧告で定められた10秒間を単位時間とする映像提示にならざるを得ず、テレビジョンやインターネット配信の数分から数時間の映像及び/又は音響に時間離散的にパケット損失が生じた場合の劣化映像及び/又は音響を、被験者に提示して品質評価を行なうことは著しく困難である。さらに、数分から数時間の長時間の劣化映像及び/又は音響の品質評価は、被験者に多大の労力をかけるため、著しく困難である。上記従来技術の許容パケット損失率決定方法は、10秒という単位時間に生じる劣化映像及び/又は音響を評価しており、数分から数時間に劣化が時間離散的に生じた場合に視聴者が許容する劣化発生頻度の要因を考慮していないため、長時間視聴時の劣化映像及び/又は音響の許容率及び許容されるパケット損失率を決定することができないという問題点がある。
【0003】
【発明が解決しようとする課題】
この発明の目的は、被験者が長時間の劣化映像及び/又は音響を実際に評価せずに、長時間視聴時の劣化映像及び/又は音響に対する許容率と対応する許容されるパケット損失率を決定する装置及びそのプログラムを提供することにある。
【0004】
【課題を解決するための手段】
この発明の一面によれば、上記目的を達するために、単位時間長の劣化映像及び/又は音響品質とパケット損失率の関係、及び単位時間長の劣化映像及び/又は音響品質と許容される劣化発生時間間隔の関係を用い、単位時間のパケット損失率と長時間視聴時の劣化映像及び/又は音響に対する許容率を与えることで長時間視聴時に許容されるパケット損失率を決定する。
この発明の他面によれば単位時間のパケット損失率と劣化区間の発生時間間隔を与えることで長時間視聴時の劣化映像及び/又は音響に対する許容率と許容されるパケット損失率を決定する。
【0005】
【発明の実施の形態】
以下にこの発明の実施形態を、実施例により説明する。またこの発明は、映像信号のみならず、音声信号、音楽信号、更に映像信号と音声信号又は/及び音楽信号の組み合せを長時間配信するシステムの設計に必要とする劣化許容率とパケット損失率の決定に適用することができるが、以下の説明では映像信号を代表して説明する。
実施例1
この発明の実施例1は、単位時間のパケット損失率Lと、視聴者の長時間視聴時の劣化映像に対する許容率Aを指定することにより、許容されるパケット損失率ALを決定する。
【0006】
この発明の実施例1の機能構成を図1に示す。
劣化品質推定部10は、端子11からの単位時間のパケット損失率Lを入力することにより、単位時間の劣化映像の品質Qを出力する。劣化発生時間間隔推定部20は劣化品質推定部10よりの単位時間の劣化映像の品質Qと、端子12からの劣化映像に対する許容率Aを入力することにより、視聴者が許容できる劣化映像区間の発生時間間隔Tを出力する。許容パケット損失率算出部30は、端子11からの単位時間のパケット損失率Lと、劣化発生時間間隔推定部20よりの劣化映像区間の発生時間間隔Tを入力することにより、許容されるパケット損失率ALを端子31に出力する。
【0007】
劣化品質推定部10は、単位時間内にパケット損失を発生させた場合のパケット損失率Lと劣化映像の品質Qの関係式F1を、主観評価と統計的手法により予め求めておき、入力された単位時間パケット損失率Lを関係式F1へ代入することで、単位時間の劣化映像の品質Qを推定する。関係式F1の導出例として、主観評価より求めたパケット損失率と劣化映像の品質の関係を図2に示す。この図2に示したデータは次のようにして求めたものである。実験網の両端にVTR及び映像パケット化装置を設置し、DV映像を伝送する系を構成し、パケット損失は網内に設置した品質エミュレータを用いランダム損失(損失個数を変えた5段階)を発生させ、評価用映像には10秒間の標準動画像を8種類用い、映像品質の評価法は二重刺激連続品質尺度(DSCQS)法を用い、各劣化映像の指示・評価においては、映像品質評価に加え、許容できる劣化発生頻度を求めるため、10秒間の劣化映像区間が数十分に1回発生したと想定した時許容できるか否かを問い、その回答を求めた。また許容劣化発生頻度は有料/無料視聴により異なる可能性があるため両ケースを評価した。被験者は一般人24名である。図2中の評価値群41,42,43,44,45はそれぞれ10秒当りの損失個数が1個、3個、5個、10個の場合における24人の評価値の各シーンごとの平均値であり、損失個数が1個、3個、5個については同じ損失率の映像を2パターン(10秒中における損失個所が異なる各2つのパターン)評価した。同じ損失率でも評価映像の相違により品質劣化の程度が大きく異なる。
【0008】
ITU−R BT.800−2 勧告の標記法に従い、評価映像シーン中で最低値の品質値を最悪品質とし、75%のシーン数(評価値群41,42,43は各12個、評価値群44,45は各6個)が満たす品質値を75%品質とすると、これらは損失率の増加に伴い比例するという統計的性質を利用して一定の傾きの直線46,45を表わす一意の関係式で映像品質を、最悪品質あるいは75%品質として表すことができる。
関係式F1の例1:映像品質Qを最悪品質とした場合、
Q=C01+C11×ln(L)、C01,C11は定数
関係式F1の例2:映像品質Qを75%品質とした場合、
Q=C02+C12×ln(L)、C02,C12は定数。
この実施例では、C01=126.67,C11=9.06,
C02=105.46,C12=8.31である。
例えばパケット損失率Lが10−4(1E−4)の場合、劣化映像品質Qは30となる。
【0009】
以上より劣化品質推定部10は、映像信号にパケット損失を生じさせた映像音声の主観品質評価と統計的性質により関数Q=F1(L)として予め定式化し、このQ=F1(L)に単位時間のパケット損失率Lを代入して単位時間の劣化映像の品質Qを演算して求める構成とされている。つまり定数項としてC01又はC02が、1次係数としてC11又はC12がレジスタに格納され、入力されたパケット損失率Lの対数値ln(L)を求めて、これを変数として一次関数F1が演算される。
劣化発生時間間隔推定部20は、予め主観評価実験により求められた、単位時間の劣化映像の品質Qと、許容される劣化発生時間間隔Tと、劣化許容率Aとの三つの相互に関連する主観品質値を蓄える。つまり先に述べた実験では、10秒間の劣化映像区間が数十分間、例えば1時間に1回発生したと想定した時、その長時間映像が許容できるか否かを、各劣化品質と各種劣化発生時間間隔について被験者に問い合せ、その回答から各種許容率Aについて、劣化映像品質Qと許容できる劣化発生時間間隔Tとの関係を求める。
劣化発生時間間隔推定部20は許容率Aと劣化品質推定部10から出力された劣化品質Qを入力し、劣化発生時間間隔Tを出力する。このため、先に求めた、種々の品質値Qの単位時間の劣化映像がT時間に1回発生したとして許容できるか否かの主観評価結果から、関係式T=F2(Q,A)を得る。劣化映像に対する許容率は、許容できるとした視聴者(被験者)の全被験者に対する人数比(%)としている。図3は、この実験結果より求めた、劣化映像に対する許容率Aが90%の場合における、有料視聴と無料視聴の2種類の場合の、劣化映像品質Qと許容できる劣化発生時間間隔Tとの関係を示している。
【0010】
関係式F2の例1:有料視聴の場合
T(分)=D01+D11×Q
関係式F2の例2:無料視聴の場合
T(分)=D02+D12×Q
許容率90%のこの実施例では、D01=−7.33,D11=4.37、
D02=−13.22,D12=3.78
であり、直線48と49にそれぞれ対応する。有料視聴は無料視聴に比べ、許容できる劣化発生頻度が低い、これは視聴者の料金支払い意識が反映されているためと考えられる。この例において例えば劣化映像品質Qを30とすると、無料視聴時は、劣化発生時間間隔Tはほぼ100分となる。
【0011】
つまり劣化発生時間間隔推定部20には例えば、関数格納部21が設けられ、関数格納部21には各許容率Aについて、関数F2の定数(D01又はD02)と係数(D11又はD12)が格納され、入力された許容率Aについて、定数と係数が関数格納部21から読み出され、これら定数と係数の一次関数F2に、入力された劣化品質Qが変数として代入され、演算部22で劣化発生時間間隔Tが演算され、これが出力される。なお、関数格納部21には有料視聴と無料視聴とを区別して、各許容率に対する定数と係数を格納しておいて、有料視聴と無料視聴とで区別して劣化時間間隔Tを出力するようにしてもよい。
許容パケット損失率算出部30は、端子11よりの単位時間におけるパケット損失率L及び劣化発生時間間隔推定部20よりの劣化区間の発生時間間隔Tを用い、許容パケット損失率ALを下記関係式から計算する。
関係式F4:AL=L/(U/T)=L×T/U
【0012】
単位時間長Uは、劣化品質推定部10で用いる関係式F1及び劣化発生時間間隔推定部20で用いる関係式F2とF3を導出する際の単位時間長を用いる。この実施例では、単位時間長Uは10秒である。損失率Lは、単位時間U(=10秒)での送出パケット数NPUで、単位時間U内でのパケット損失個数NPLを割った値NPL/NPUである。

Figure 2004072178
例えば、劣化発生時間間隔推定部20より算出された劣化発生時間間隔Tが30分(=1800秒)の場合の、許容パケット損失率ALは、下記で計算する。この実施例では、許容パケット損失率ALは、ネットワークのシステム設計において用いられることが多い、時間単位や日単位の長時間で算出する。
ALを1時間単位(=3600秒)の観測時間で求めた場合、
Figure 2004072178
上記例にあるようにALの算出は、関係式F4で一意に求めることが可能であり、長時間視聴時の許容できるパケット損失率ALを求めていることになる。
この図1に示した構成によれば、単位時間パケット損失率Lと、ユーザにより要求される劣化許容率Aを入力して、Lから劣化品質Qを劣化品質推定部10により求め、劣化発生時間間隔推定部20で入力された許容率Aと対応した関数を取り出し、これにQを代入することにより、劣化発生時間間隔Tを求め、このTとLから、許容率Aの場合にシステムに要求される長時間パケット損失率ALが求められる。
【0013】
実施例2
この発明の実施例2は、単位時間のパケット損失率Lと単位時間の劣化映像の発生時間間隔Tを指定することにより、劣化映像に対する許容率Aと許容されるパケット損失率ALを決定する。
この実施例2の機能構成を図4に示し、図1と対応する部分に同一参照番号を付けてある。単位時間の劣化品質推定部10は、端子11よりの単位時間のパケット損失率Lを入力することにより、単位時間の劣化映像の品質Qを出力するもので、実施例1のそれと同様に構成される。損失率算出部30は、端子11よりの単位時間のパケット損失率Lと、端子32よりの劣化映像区間の発生時間間隔Tを入力することにより、許容されるパケット損失率ALを出力するもので、その構成も実施例1のそれと同様に構成される。許容率推定部50は端子32よりの劣化発生時間間隔Tと劣化品質推定部10よりの単位時間の劣化映像の品質Qとを入力して劣化映像に対する許容率Aを端子51に出力する。
【0014】
許容率推定部50は予め主観評価と統計的手法により求められた、単位時間の劣化映像の品質Qと、許容される劣化発生時間間隔Tと劣化映像に対する許容率Aとの三つの相互の関連を示す主観品質値を蓄える。許容率Aの導出例を示す。例えば実施例1で述べた主観評価実験における、10秒間の劣化映像区間が数十分に1回発生したと想定した時のこれが許容できるか否かの回答に基づき、図5に示すように各種劣化映像品質Qについて、許容劣化発生時間間隔Tと、劣化映像に対する許容率Aとの関係をプロットし、種々の品質値Qの単位時間の劣化映像がT時間に1回発生したとして許容できるか否かの主観評価からこれらの関係式T=F3(Q,T)を得る。劣化映像に対する許容率は、許容できると評価した視聴者の、全被試験者に対する人数比(%)としている。図5は、劣化映像の映像品質Qが28の場合における、有料視聴と無料視聴の2種類の場合の、許容率Aと劣化発生時間間隔Tの関係を示している。これら主観評価値からAとTの関係を表わす曲線52(有料視聴時)と曲線53(無料視聴時)の各式F3を求めることができる。
関係式F3の例1:有料視聴の場合
A=D01+D11×T+D21×T×T
関係式F3の例2:無料視聴の場合
A=D02+D12×T+D22×T×T
この実施例では、
D01=19.3,D11=0.755,D21=−1.40E−3
D02=18.8,D12=1.42,D22=−6.40E−3
である。
【0015】
関係式F3は、予め種々の映像品質Qに対し定める。Q=28の場合、許容できる劣化発生時間間隔Tを約60分とすると、有料視聴の場合、曲線52の式より劣化映像に対する許容率Aは約57%となる。このLとTから約57%の人は劣化映像を許容し、その時の長時間パケット損失率ALは(L/T)×Uにより求まる。
許容率推定部50は、関数格納部54に、各品質Qごとに、関係式F3の定数(D01又はD02)、1次係数(D11又はD12)、2次係数(D21又はD22)が格納され入力されたQに応じて関数格納部54から、定数、1次係数、2次係数が取り出され、演算部55でこれらによりなる2次関数F3にTを代入して、その関数値が演算されてその結果Aが出力される。
【0016】
なお関数格納部54には有料視聴に対するもの、無料視聴に対するものの両者を格納しておいてもよい。実施例1又は2により求めた各種劣化映像に対する許容率Aと、その場合の長時間パケット損失率ALとの関係を例えば図6に示すように作っておくとよい。図6では、実線56,58は無料視聴を想定したもの、破線57,59は有料視聴を想定したものであり、最悪品質に対する関係式を線56,57で、75%品質に対するものを表わす。このグラフからシステム設計時に、ユーザの許容率Aが予測されると、それと対応するパケット損失率ALを推定することができる。
【0017】
図1、図4に示した装置はコンピュータに機能させることもできる。この場合は、例えば図7に示すように各部はバス60により接続され、入力部61に単位時間パケット損失率Lと、許容率Aが入力設定されると、ワークメモリとして動作する記憶部62に必要に応じて格納され、劣化品質関数格納部63に格納されている定数C01又はC02と係数(C11又はC12)が取り出され、関数F1にLを代入した演算がなされて、劣化品質Qが求められ、更に入力されたAにより劣化発生時間間隔関数格納部64から、対応する定数と係数が取り出され、関数F2にQを代入した演算がなされて、劣化発生時間間隔Tが計算され、更にTとLから(L/T)×Uが計算され、その結果ALが出力部67により表示され、記録され又は他の処理装置へ出力される。これらの処理がCPU(マイクロプロセッサ)66によりプログラムメモリ65に格納されている許容パケット損失率決定プログラムを実行することにより行われる。
【0018】
許容パケット損失率決定プログラムはメモリ65に予め、CD−ROM、磁気ディスクなどの記録媒体からインストールし、又は通信回線を介してダウンロードしておく。実施例2の場合は入力部61から、単位時間パケット損失率Lと劣化発生時間間隔Tが入力され、図7中に括弧書きで示すように図4中に示した関数格納部54と同様構成の許容率関数格納部64が用いられ、又は実施例2の説明で行った処理が実行されることになる。
【0019】
【発明の効果】
上述のようにこの発明によれば、被験者が長時間の劣化映像及び/又は音響を実際に評価することなく、劣化映像及び/又は音響に対する許容率及びこれと対応した長時間視聴時に許容されるパケット損失率が得られる。
【図面の簡単な説明】
【図1】この発明の実施例1の機能構成例を示す図。
【図2】実施例において、劣化品質推定部10で用いる関係式F1の導出を説明するためのグラフ。
【図3】実施例において、劣化発生時間間隔推定部20で用いる関係式F2の導出例を説明するためのグラフ。
【図4】この発明の実施例2の機能構成例を示す図。
【図5】実施例2において、許容率推定部50で用いる関係式F3の導出例を説明するためのグラフ。
【図6】求めた長時間視聴に許容されるパケット損失率と、劣化映像に対する許容率との関係例を示すグラフ。
【図7】この発明装置をコンピュータに機能させるための構成例を示す図。[0001]
TECHNICAL FIELD OF THE INVENTION
According to the present invention, in the quality design at the time of development of a service / device for transmitting a video signal and / or an audio signal such as audio or music, a deterioration video and / or audio allowance when viewing a video and / or audio for a long time is considered. The present invention relates to an apparatus and a program for determining an allowable packet loss rate for a rate.
[0002]
[Prior art]
2. Description of the Related Art Conventionally, in a system for transmitting a video signal, an audio signal, a music signal, a video signal and an audio signal or a music signal (hereinafter, referred to as a video and / or audio signal), a code error or a packet arrival time in a transmission path, a transmission device, or a terminal. The fluctuations may cause packet loss and degrade video and / or audio quality. 2. Description of the Related Art Conventionally, as a means for obtaining a packet loss rate that achieves acceptable video and / or audio quality, there is a method in which a subject subjectively evaluates, for example, a 10-second deteriorated video that causes packet loss in a video signal. Also, for example, ITU-R BT. There is a method of performing subjective quality evaluation of video based on the 500-10 recommendation. According to this method, it is recommended that at least 15 or more subjects evaluate a deteriorated video having various loss rates using at least four or more types of video scenes, and the number of evaluation conditions becomes enormous. For this reason, video presentation must be performed with a unit time of 10 seconds as defined in the above recommendation, and packet loss occurs discretely in video and / or audio for several minutes to several hours of television or Internet distribution. It is extremely difficult to present the deteriorated video and / or sound in such a case to a subject to perform quality evaluation. In addition, long-term degraded video and / or sound quality evaluations of minutes to hours are extremely difficult because they require a great deal of effort on the subject. The above-described prior art allowable packet loss rate determination method evaluates the deteriorated video and / or sound that occurs in a unit time of 10 seconds, and allows the viewer to accept the deterioration when the deterioration occurs discretely from several minutes to several hours. Since the factor of the frequency of occurrence of deterioration is not taken into consideration, there is a problem that it is not possible to determine the allowable rate of the deteriorated video and / or sound and the allowable packet loss rate during long viewing.
[0003]
[Problems to be solved by the invention]
SUMMARY OF THE INVENTION It is an object of the present invention to determine a permissible packet loss rate corresponding to a permissible rate of degraded video and / or audio during long viewing without a subject actually evaluating long-term degraded video and / or audio. And a program therefor.
[0004]
[Means for Solving the Problems]
According to one aspect of the present invention, in order to achieve the above object, a relationship between a deteriorated video and / or sound quality of a unit time length and a packet loss rate, and a deteriorated video and / or sound quality of a unit time length and an allowable deterioration. The packet loss rate permissible during long viewing is determined by giving the packet loss rate per unit time and the permissible rate for degraded video and / or audio during long viewing using the relationship between the occurrence time intervals.
According to another aspect of the present invention, the permissible rate and permissible packet loss rate for deteriorated video and / or audio during long viewing are determined by giving the packet loss rate per unit time and the occurrence time interval of the degraded section.
[0005]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to examples. In addition, the present invention provides a system which distributes not only a video signal but also an audio signal, a music signal, and a combination of a video signal and an audio signal and / or a music signal for a long time. Although it can be applied to the determination, in the following description, the video signal will be described as a representative.
Example 1
In the first embodiment of the present invention, an allowable packet loss rate AL is determined by designating a packet loss rate L per unit time and an allowable rate A for a deteriorated image when a viewer views for a long time.
[0006]
FIG. 1 shows a functional configuration of the first embodiment of the present invention.
The degradation quality estimation unit 10 outputs the quality Q of the degradation video per unit time by inputting the packet loss rate L per unit time from the terminal 11. The degradation occurrence time interval estimation unit 20 inputs the quality Q of the degradation video per unit time from the degradation quality estimation unit 10 and the allowable ratio A for the degradation video from the terminal 12, and thereby the degradation video section that the viewer can tolerate. Output the occurrence time interval T. The permissible packet loss rate calculation unit 30 inputs the packet loss rate L per unit time from the terminal 11 and the occurrence time interval T of the deteriorated video section from the deterioration occurrence time interval estimation unit 20, thereby allowing the permissible packet loss. The rate AL is output to the terminal 31.
[0007]
The deterioration quality estimating unit 10 previously obtains a relational expression F1 between the packet loss rate L and the quality Q of the deteriorated video when a packet loss occurs within a unit time by a subjective evaluation and a statistical method, and is input. By substituting the unit time packet loss rate L into the relational expression F1, the quality Q of the deteriorated video per unit time is estimated. As an example of deriving the relational expression F1, FIG. 2 shows the relationship between the packet loss rate obtained from the subjective evaluation and the quality of the deteriorated video. The data shown in FIG. 2 is obtained as follows. VTRs and video packetizers are installed at both ends of the experimental network, and a system for transmitting DV video is constructed. Packet loss generates random losses (5 stages with different numbers of losses) using a quality emulator installed in the network Eight kinds of standard moving images of 10 seconds are used for the evaluation video, and the image quality is evaluated using the double stimulus continuous quality scale (DSCQS) method. In addition, in order to obtain an allowable deterioration occurrence frequency, it was asked whether or not it was allowable when it was assumed that the deterioration video section of 10 seconds occurred once several tens of minutes, and the answer was obtained. In addition, both cases were evaluated because the allowable deterioration occurrence frequency may differ depending on paid / free viewing. The subjects were 24 ordinary people. The evaluation value groups 41, 42, 43, 44, and 45 in FIG. 2 are the averages of the evaluation values of 24 scenes for each scene when the number of losses per 10 seconds is 1, 3, 5, and 10, respectively. With respect to the loss numbers of 1, 3, and 5, two images having the same loss rate were evaluated (two patterns having different loss locations in 10 seconds). Even at the same loss rate, the degree of quality deterioration greatly differs due to the difference in the evaluation video.
[0008]
ITU-R BT. According to the notation method of the 800-2 recommendation, the lowest quality value in the evaluation video scene is regarded as the worst quality, and the number of scenes of 75% (the evaluation value groups 41, 42, and 43 are 12 each, and the evaluation value groups 44 and 45 are the Assuming that the quality values satisfied by each of the six (6) are 75% quality, the video quality is represented by a unique relational expression expressing straight lines 46 and 45 having a constant slope by utilizing the statistical property that these are proportional to the increase of the loss rate. Can be expressed as worst quality or 75% quality.
Example 1 of relational expression F1: When the video quality Q is the worst quality,
Q = C01 + C11 × ln (L), where C01 and C11 are constant relational expressions F1: Example 2: When the video quality Q is 75% quality,
Q = C02 + C12 × ln (L), where C02 and C12 are constants.
In this embodiment, C01 = 126.67, C11 = 9.06,
C02 = 105.46 and C12 = 8.31.
For example, when the packet loss rate L is 10 −4 (1E−4), the degraded video quality Q is 30.
[0009]
As described above, the deterioration quality estimating unit 10 formulates a function Q = F1 (L) in advance based on the subjective quality evaluation and statistical properties of the video and audio that caused the packet loss in the video signal, and the unit is expressed as Q = F1 (L). The configuration is such that the quality Q of the deteriorated video per unit time is calculated by substituting the packet loss rate L per unit time. That is, C01 or C02 is stored in a register as a constant term, and C11 or C12 is stored in a register as a primary coefficient. A logarithmic value ln (L) of the input packet loss rate L is obtained, and a linear function F1 is calculated using this as a variable. You.
The deterioration occurrence time interval estimating unit 20 relates three interrelationships of the quality Q of the deterioration image per unit time, the allowable deterioration occurrence time interval T, and the deterioration allowable rate A, which are obtained in advance by a subjective evaluation experiment. Store subjective quality values. In other words, in the experiment described above, when it is assumed that a 10-second deteriorated video section occurs for several tens of minutes, for example, once an hour, it is determined whether or not the long-time video is permissible. The subject is inquired about the deterioration occurrence time interval, and from the answer, the relationship between the deterioration image quality Q and the allowable deterioration occurrence time interval T is obtained for various allowable rates A.
The deterioration occurrence time interval estimating unit 20 receives the allowable rate A and the deterioration quality Q output from the deterioration quality estimating unit 10, and outputs a deterioration occurrence time interval T. For this reason, the relational expression T = F2 (Q, A) is obtained from the subjective evaluation result obtained as to whether or not the previously determined deteriorated image of various quality values Q per unit time occurs once per T time. obtain. The permissible rate for the deteriorated video is defined as the ratio (%) of the number of viewers (subjects) to all the test subjects that is considered to be permissible. FIG. 3 shows the relationship between the degraded video quality Q and the allowable degradation occurrence time interval T in the two cases of pay viewing and free viewing when the permissible rate A for the degraded video obtained from the experimental results is 90%. Shows the relationship.
[0010]
Example of relational expression F2 1: In the case of pay viewing, T (minute) = D01 + D11 × Q
Example 2 of relational expression F2: For free viewing T (minutes) = D02 + D12 × Q
In this embodiment with a 90% tolerance, D01 = -7.33, D11 = 4.37,
D02 = -13.22, D12 = 3.78
And correspond to the straight lines 48 and 49, respectively. Paid viewing has a lower permissible deterioration frequency than free viewing, which is considered to reflect the viewer's awareness of paying for the fee. In this example, assuming that the degraded video quality Q is 30, for example, during free viewing, the degradation occurrence time interval T is approximately 100 minutes.
[0011]
That is, the deterioration occurrence time interval estimating unit 20 is provided with, for example, a function storage unit 21, and the function storage unit 21 stores a constant (D01 or D02) and a coefficient (D11 or D12) of the function F2 for each allowable rate A. Then, for the input allowable rate A, constants and coefficients are read from the function storage unit 21, and the input deterioration quality Q is substituted as a variable into a linear function F2 of these constants and coefficients, and the deterioration An occurrence time interval T is calculated and output. It should be noted that the function storage unit 21 stores constants and coefficients for each permissible rate for distinguishing between pay viewing and free viewing, and outputs the deterioration time interval T separately for pay viewing and free viewing. You may.
The allowable packet loss rate calculation unit 30 uses the packet loss rate L per unit time from the terminal 11 and the occurrence time interval T of the degradation section from the degradation occurrence time interval estimation unit 20 to calculate the allowable packet loss rate AL from the following relational expression. calculate.
Relational expression F4: AL = L / (U / T) = L × T / U
[0012]
As the unit time length U, a unit time length for deriving the relational expression F1 used in the deterioration quality estimation unit 10 and the relational expressions F2 and F3 used in the deterioration occurrence time interval estimation unit 20 is used. In this embodiment, the unit time length U is 10 seconds. The loss rate L is a value N PL / N PU obtained by dividing the number N PU of transmitted packets in the unit time U (= 10 seconds) by the number N PL of lost packets in the unit time U.
Figure 2004072178
For example, when the deterioration occurrence time interval T calculated by the deterioration occurrence time interval estimation unit 20 is 30 minutes (= 1800 seconds), the allowable packet loss rate AL is calculated as follows. In this embodiment, the allowable packet loss rate AL is calculated over a long period of time, such as hours or days, often used in network system design.
When AL is obtained in observation time of one hour unit (= 3600 seconds),
Figure 2004072178
As in the above example, the calculation of AL can be uniquely obtained by the relational expression F4, which means that the allowable packet loss rate AL during long viewing is obtained.
According to the configuration shown in FIG. 1, a unit time packet loss rate L and a degradation tolerance rate A required by a user are input, and a degradation quality Q is obtained from L by the degradation quality estimating unit 10, and the degradation occurrence time is calculated. A function corresponding to the allowable rate A input by the interval estimating unit 20 is extracted, and Q is substituted into the function to determine a deterioration occurrence time interval T. From the T and L, a request is issued to the system in the case of the allowable rate A. The long-term packet loss rate AL is calculated.
[0013]
Example 2
In the second embodiment of the present invention, the permissible rate A and the permissible packet loss rate AL for the deteriorated video are determined by specifying the packet loss rate L per unit time and the generation time interval T of the deteriorated video per unit time.
FIG. 4 shows a functional configuration of the second embodiment, and portions corresponding to those in FIG. 1 are denoted by the same reference numerals. The unit-time deterioration quality estimating unit 10 outputs the unit-time deteriorated video quality Q by inputting the unit-time packet loss rate L from the terminal 11, and has the same configuration as that of the first embodiment. You. The loss rate calculation unit 30 outputs an allowable packet loss rate AL by inputting a packet loss rate L per unit time from the terminal 11 and a generation time interval T of a deteriorated video section from the terminal 32. The configuration is the same as that of the first embodiment. The allowable rate estimating unit 50 receives the deterioration occurrence time interval T from the terminal 32 and the quality Q of the deteriorated image per unit time from the deterioration quality estimating unit 10 and outputs the allowable rate A for the deteriorated image to the terminal 51.
[0014]
The permissible rate estimating unit 50 has three mutual relations between the quality Q of the degraded video per unit time, the permissible degradation occurrence time interval T, and the permissible rate A for the degraded video, which are obtained in advance by a subjective evaluation and a statistical method. Is stored. An example of deriving the allowable ratio A will be described. For example, based on the subjective evaluation experiment described in the first embodiment, based on the answer as to whether or not this is permissible when it is assumed that the degraded video section of 10 seconds occurs once several ten minutes, as shown in FIG. For the deteriorated video quality Q, the relationship between the allowable deterioration occurrence time interval T and the permissible rate A for the deteriorated video is plotted, and it is acceptable if the deteriorated video of various quality values Q in the unit time occurs once in the T time. These relational expressions T = F3 (Q, T) are obtained from the subjective evaluation of whether or not to do so. The permissible rate for the deteriorated video is the ratio (%) of the number of viewers who are evaluated as permissible to all the test subjects. FIG. 5 shows the relationship between the allowable ratio A and the degradation occurrence time interval T in the case of two types of viewing, pay viewing and free viewing, when the video quality Q of the deteriorated video is 28. From these subjective evaluation values, the formulas F3 of the curve 52 (at the time of pay viewing) and the curve 53 (at the time of free viewing) representing the relationship between A and T can be obtained.
Example of relational expression F3 1: In the case of paid viewing A = D01 + D11 × T + D21 × T × T
Example 2 of relational expression F3: For free viewing A = D02 + D12 × T + D22 × T × T
In this example,
D01 = 19.3, D11 = 0.755, D21 = -1.40E-3
D02 = 18.8, D12 = 1.42, D22 = -6.40E-3
It is.
[0015]
The relational expression F3 is determined in advance for various video qualities Q. In the case of Q = 28, assuming that the permissible deterioration occurrence time interval T is about 60 minutes, the permissible rate A for the deteriorated video is about 57% from the equation of the curve 52 for pay viewing. Approximately 57% of the persons from L and T allow a deteriorated video, and the long-term packet loss rate AL at that time is obtained by (L / T) × U.
The allowance estimating unit 50 stores the constant (D01 or D02), the first order coefficient (D11 or D12), and the second order coefficient (D21 or D22) of the relational expression F3 for each quality Q in the function storage unit 54. Constants, first-order coefficients, and second-order coefficients are extracted from the function storage unit 54 according to the input Q, and T is substituted into the quadratic function F3 formed by the arithmetic unit 55 to calculate the function value. As a result, A is output.
[0016]
Note that the function storage unit 54 may store both for paid viewing and free viewing. The relationship between the permissible ratio A for various types of deteriorated video obtained in the first or second embodiment and the long-term packet loss ratio AL in that case may be made, for example, as shown in FIG. In FIG. 6, the solid lines 56 and 58 are for free viewing, and the broken lines 57 and 59 are for pay viewing. The relational expressions for the worst quality are shown by lines 56 and 57, and for 75% quality. When the user's allowable rate A is predicted from the graph at the time of system design, the corresponding packet loss rate AL can be estimated.
[0017]
The devices shown in FIGS. 1 and 4 can be caused to function by a computer. In this case, for example, as shown in FIG. 7, each unit is connected by a bus 60, and when a unit time packet loss rate L and an allowable rate A are input to an input unit 61, the storage unit 62 that operates as a work memory is input to the storage unit 62. The constant C01 or C02 and the coefficient (C11 or C12) stored as necessary and stored in the deterioration quality function storage unit 63 are extracted, and an operation in which L is substituted for the function F1 is performed to obtain the deterioration quality Q. Then, the corresponding constants and coefficients are extracted from the deterioration occurrence time interval function storage unit 64 by the input A, and the operation is performed by substituting Q for the function F2, and the deterioration occurrence time interval T is calculated. Then, (L / T) × U is calculated from L and L. As a result, the AL is displayed on the output unit 67, recorded, or output to another processing device. These processes are performed by the CPU (microprocessor) 66 executing the allowable packet loss rate determination program stored in the program memory 65.
[0018]
The allowable packet loss rate determination program is installed in the memory 65 in advance from a recording medium such as a CD-ROM or a magnetic disk, or is downloaded via a communication line. In the case of the second embodiment, the unit time packet loss rate L and the deterioration occurrence time interval T are input from the input unit 61, and have the same configuration as the function storage unit 54 shown in FIG. 4 as shown in parentheses in FIG. Or the processing described in the second embodiment is executed.
[0019]
【The invention's effect】
As described above, according to the present invention, the test subject does not actually evaluate the degraded video and / or sound for a long time, and the permissible rate for the degraded video and / or the sound and the permissible rate corresponding to the permissible long time viewing are allowed. The packet loss rate is obtained.
[Brief description of the drawings]
FIG. 1 is a diagram showing an example of a functional configuration according to a first embodiment of the present invention.
FIG. 2 is a graph for explaining the derivation of a relational expression F1 used in the deterioration quality estimation unit 10 in the embodiment.
FIG. 3 is a graph for explaining an example of deriving a relational expression F2 used in a deterioration occurrence time interval estimating unit 20 in the embodiment.
FIG. 4 is a diagram showing an example of a functional configuration according to a second embodiment of the present invention.
FIG. 5 is a graph for explaining an example of deriving a relational expression F3 used in an allowable ratio estimating unit 50 in the second embodiment.
FIG. 6 is a graph showing an example of the relationship between the obtained packet loss rate for long-time viewing and the allowable rate for degraded video.
FIG. 7 is a diagram showing an example of a configuration for causing a computer to function according to the present invention.

Claims (6)

単位時間のパケット損失率から単位時間長の劣化映像及び/又は音響の品質を推定する劣化品質推定手段と、
上記推定された単位時間長の劣化映像及び/又は音響の品質と劣化映像及び/又は音響に対する許容率とから許容される劣化区間の発生時間間隔を推定する劣化発生時間間隔推定手段と、
上記単位時間のパケット損失率と上記推定された許容される劣化区間の発生時間間隔から劣化映像及び/又は音響に対して許容されるパケット損失率を決定する損失率算出手段と、
を具備する許容パケット損失率決定装置。
Deterioration quality estimation means for estimating the quality of the deteriorated video and / or sound of the unit time length from the packet loss rate of the unit time;
Degradation occurrence time interval estimating means for estimating an occurrence time interval of a degradation section allowed from the estimated quality of the degradation video and / or sound of the unit time length and an allowable rate for the degradation video and / or sound;
Loss rate calculating means for determining a packet loss rate allowed for the deteriorated video and / or sound from the packet loss rate per unit time and the estimated time interval of occurrence of the allowable deterioration section,
A permissible packet loss rate determining apparatus comprising:
単位時間のパケット損失率から単位時間長の劣化映像及び/又は音響の品質を推定する劣化品質推定手段と、
上記推定された単位時間長の劣化映像及び/又は音響の品質と劣化区間の発生時間間隔とから劣化映像及び/又は音響に対する許容率を推定する許容率推定手段と、
上記単位時間のパケット損失率と上記劣化区間の発生時間間隔から劣化映像及び/又は音響に対して許容されるパケット損失率を決定する損失率算出手段と、を具備する許容パケット損失率決定装置。
Deterioration quality estimation means for estimating the quality of the deteriorated video and / or sound of the unit time length from the packet loss rate of the unit time;
A permissible rate estimating means for estimating a permissible rate for the degraded video and / or sound from the estimated quality of the degraded video and / or sound of the unit time length and the occurrence time interval of the degraded section;
A permissible packet loss rate determination device comprising: a permissible packet loss rate determining unit that determines a permissible packet loss rate for degraded video and / or audio from the per unit time packet loss rate and the occurrence time interval of the degraded section.
請求項1又は2に記載の装置において、
上記劣化品質推定手段は、映像及び/又は音響信号にパケット損失を生じさせた映像及び/又は音響の品質を主観評価する試験により求めた、単位時間のパケット損失率を変数とする関数に、単位時間のパケット損失率を関数へ代入して単位時間の劣化映像及び/又は音響の品質を演算する手段であることを特徴とする許容パケット損失率決定装置。
The device according to claim 1 or 2,
The deterioration quality estimating means includes a unit having a unit of a packet loss rate as a variable, which is obtained by a test for subjectively evaluating the quality of video and / or audio in which packet loss has occurred in a video and / or audio signal. A permissible packet loss rate determining apparatus, which is a means for calculating the quality of a deteriorated video and / or sound per unit time by substituting a time packet loss rate into a function.
請求項1に記載の装置において、
上記劣化発生時間間隔推定手段は、単位時間長の劣化映像及び/又は音響を視聴した場合に視聴者が許容される劣化映像及び/又は音響区間の発生時間間隔を、主観評価する試験により求めた劣化映像及び/又は音響に対する許容率をパラメータとし、劣化品質を変数とする関数を格納する関数格納部と、上記推定された上記劣化映像及び/又は音響に許容率と対応する関数を格納部が取り出して、上記単位時間長の劣化品質を代入して許容される劣化区間の発生時間間隔を演算する手段とを備えることを特徴とする許容パケット損失率決定装置。
The apparatus according to claim 1,
The deterioration occurrence time interval estimating means obtains, by a subjective evaluation test, a generation time interval of a deterioration image and / or sound section which is permissible to a viewer when viewing a deterioration image and / or sound having a unit time length. A function storage unit that stores a function that uses the deterioration rate as a parameter with the allowable ratio for the deteriorated image and / or sound as a parameter, and stores a function corresponding to the estimated ratio for the estimated deteriorated image and / or sound. Means for calculating a permissible degradation interval occurrence time interval by taking out the degradation quality of the unit time length and substituting the pertinent degradation quality for the unit time length.
請求項2に記載の装置において、
上記許容率推定手段は、単位時間長の劣化映像及び/又は音響を視聴した場合に視聴者が劣化映像及び/又は音響に対する許容率を、主観評価する試験により求めた劣化品質をパラメータとし、劣化発生時間間隔を変数とする関数を格納する関数格納部と、上記推定された劣化品質に応じて取り出された関数に劣化区間の発生時間間隔を代入して演算して、劣化映像及び/又は音響に対する許容率を演算する手段とを備えることを特徴とする許容パケット損失率決定装置。
The apparatus according to claim 2,
The permissible rate estimating means uses, as a parameter, a deterioration quality obtained by a test for subjectively evaluating a permissible rate for the deteriorated video and / or sound when the viewer views a deteriorated video and / or sound having a unit time length, and A function storage unit for storing a function using the occurrence time interval as a variable, and substituting the occurrence time interval of the deterioration section into a function extracted in accordance with the estimated deterioration quality to calculate the deterioration image and / or sound. Calculating means for calculating an allowable packet loss rate.
請求項1〜5の何れかに記載した許容パケット損失率決定装置としてコンピュータを機能させるためのプログラム。A program for causing a computer to function as the allowable packet loss rate determining device according to claim 1.
JP2002224952A 2002-08-01 2002-08-01 Allowable packet loss rate determining apparatus and program thereof Expired - Lifetime JP3860093B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2002224952A JP3860093B2 (en) 2002-08-01 2002-08-01 Allowable packet loss rate determining apparatus and program thereof

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
JP2002224952A JP3860093B2 (en) 2002-08-01 2002-08-01 Allowable packet loss rate determining apparatus and program thereof

Publications (2)

Publication Number Publication Date
JP2004072178A true JP2004072178A (en) 2004-03-04
JP3860093B2 JP3860093B2 (en) 2006-12-20

Family

ID=32012771

Family Applications (1)

Application Number Title Priority Date Filing Date
JP2002224952A Expired - Lifetime JP3860093B2 (en) 2002-08-01 2002-08-01 Allowable packet loss rate determining apparatus and program thereof

Country Status (1)

Country Link
JP (1) JP3860093B2 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005311906A (en) * 2004-04-23 2005-11-04 Ntt Communications Kk Apparatus for acquiring basic data for voice quality calculation and program therefor
JP2007043642A (en) * 2005-03-04 2007-02-15 Nippon Telegr & Teleph Corp <Ntt> Video quality evaluation apparatus, method and program
JP2007194893A (en) * 2006-01-19 2007-08-02 Nippon Telegr & Teleph Corp <Ntt> Image quality evaluation apparatus and method
CN105262950A (en) * 2015-10-15 2016-01-20 广州市百果园网络科技有限公司 Method and device for generating video information

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2005311906A (en) * 2004-04-23 2005-11-04 Ntt Communications Kk Apparatus for acquiring basic data for voice quality calculation and program therefor
JP4536415B2 (en) * 2004-04-23 2010-09-01 エヌ・ティ・ティ・コミュニケーションズ株式会社 Basic data device, program, voice quality calculation method, and voice quality calculation system
JP2007043642A (en) * 2005-03-04 2007-02-15 Nippon Telegr & Teleph Corp <Ntt> Video quality evaluation apparatus, method and program
JP4514155B2 (en) * 2005-03-04 2010-07-28 日本電信電話株式会社 Video quality evaluation apparatus, method and program
JP2007194893A (en) * 2006-01-19 2007-08-02 Nippon Telegr & Teleph Corp <Ntt> Image quality evaluation apparatus and method
JP4490374B2 (en) * 2006-01-19 2010-06-23 日本電信電話株式会社 Video quality evaluation apparatus and method
CN105262950A (en) * 2015-10-15 2016-01-20 广州市百果园网络科技有限公司 Method and device for generating video information
CN105262950B (en) * 2015-10-15 2018-08-17 广州市百果园网络科技有限公司 A kind of method and apparatus generating video information

Also Published As

Publication number Publication date
JP3860093B2 (en) 2006-12-20

Similar Documents

Publication Publication Date Title
EP1924101B1 (en) Video communication quality estimation device, method, and program
KR100935650B1 (en) Video Quality Estimation Apparatus, Method, and Computer-Readable Recording Medium for Recording Program
US7965203B2 (en) Video quality estimation apparatus, method, and program
EP2018068B1 (en) Video quality estimating device, method, and program
US8749639B2 (en) Method and apparatus for obtaining video quality parameter, and electronic device
US11463742B2 (en) Quality estimation of adaptive multimedia streaming
Pinson et al. An objective method for combining multiple subjective data sets
CN103379358B (en) Method and device for assessing multimedia quality
CN103297814A (en) Television viewing rate assessment method and system based on internet protocol television (IPTV)
JP4490374B2 (en) Video quality evaluation apparatus and method
Staelens et al. No-reference bitstream-based visual quality impairment detection for high definition H. 264/AVC encoded video sequences
CN112887795B (en) Video playing method, device, equipment and medium
JP2004072178A (en) Apparatus for determining permissible packet loss rate and program therefor
JP2019197996A (en) Engagement estimation device, engagement estimation method, and program
JP4802183B2 (en) Multimedia total quality estimation method and apparatus
WO2020170869A1 (en) Engagement estimation device, engagement estimation method and program
WO2019194034A1 (en) Viewing behaviour estimation device, viewing behaviour estimation method, and program
JP5149248B2 (en) Network quality control threshold value calculation apparatus and method
JP7255704B2 (en) Engagement estimation device, engagement estimation method and program
JP2008301027A (en) Image quality estimating method and device
Brunnström et al. Video expert assessment of high quality video for Video Assistant Referee (VAR): A comparative study

Legal Events

Date Code Title Description
A621 Written request for application examination

Free format text: JAPANESE INTERMEDIATE CODE: A621

Effective date: 20040726

A977 Report on retrieval

Free format text: JAPANESE INTERMEDIATE CODE: A971007

Effective date: 20060602

A131 Notification of reasons for refusal

Free format text: JAPANESE INTERMEDIATE CODE: A131

Effective date: 20060613

A521 Written amendment

Free format text: JAPANESE INTERMEDIATE CODE: A523

Effective date: 20060811

RD03 Notification of appointment of power of attorney

Free format text: JAPANESE INTERMEDIATE CODE: A7423

Effective date: 20060811

TRDD Decision of grant or rejection written
A01 Written decision to grant a patent or to grant a registration (utility model)

Free format text: JAPANESE INTERMEDIATE CODE: A01

Effective date: 20060905

A61 First payment of annual fees (during grant procedure)

Free format text: JAPANESE INTERMEDIATE CODE: A61

Effective date: 20060920

R150 Certificate of patent or registration of utility model

Free format text: JAPANESE INTERMEDIATE CODE: R150

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20090929

Year of fee payment: 3

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20100929

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20100929

Year of fee payment: 4

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20110929

Year of fee payment: 5

FPAY Renewal fee payment (event date is renewal date of database)

Free format text: PAYMENT UNTIL: 20120929

Year of fee payment: 6