JP2004215201A - Information processing apparatus and information processing method, data communication system, recording medium, and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To insert an I frame into data to be transmitted as needed in order to prevent error propagation. <P>SOLUTION: In step S71, it is judged whether an RR packet or an APP packet is received or not. When it is judged that the RR packet or the APP packet is received, in step S72, it is judged whether or not a refresh instruction is contained in the received RR packet or APP packet. When it is judged that the refresh instruction is contained, in step S72, encoding is controlled to generate the I frame. The I frame is then transmitted by an RTP packet, processing is returned to step S71 and succeeding processing is repeated. The present invention can be applied to a data transmitting apparatus. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

【０１４３】
ここで、括弧［］内は、その時刻が、どの時計を参照しているかを示すものである（以下、同様）。
【０１４４】
データ送信装置１０１およびデータ受信装置１０２のＮＴＰによる時刻が同期しており、ＲＴＣＰのＳＲパケットにＮＴＰ情報が正しく入っている場合、次の式（３）が成立する。
【０１４５】

【０１４６】
そして、ＳＲパケット内のタイムスタンプとＮＴＰのマッピング情報より、式（４）に示されるように、ＲＴＰパケットの送信時刻とタイムスタンプを関連付けるＮＴＰからＲＴＰタイムスタンプへの写像関数ｆ１を求め、式（３）のＲＴＰタイムスタンプと置き換えた式（５）により、ＲＴＰパケットの遅延を求めることが可能となる。
【０１４７】
ＲＴＰタイムスタンプ＝ｆ１［ＮＴＰ］・・・（４）

【０１４８】
これに対して、例えば、ＳＲパケットのＮＴＰが明らかにずれていたり、ＳＲパケットを受信することができなかったなどの理由により、ＲＴＰのタイムスタンプからＮＴＰへのマッピングができず、式（４）が得られない場合、データ受信装置１０２内部の時計で遅延を計測する必要がある。この場合、実際には、あるＲＴＰパケットの到着時刻を基点として、相対的な遅延を測定することにより、ＲＴＰパケットの遅延が求められる。
【０１４９】
例えば、第１のＲＴＰパケットのタイムスタンプをＴＳ１、第１のＲＴＰパケットの受信時刻（単位は、ｍｓ）をＴＲ１とする。第１のＲＴＰパケットのタイムスタンプも、第１のＲＴＰパケットの受信時刻も、ＮＴＰとは異なる基準で計測される時刻である。ここで、第１のＲＴＰパケットのタイムスタンプと第１のＲＴＰパケットの受信時刻との時刻の単位をあわせるために、受信時刻にＲＴＰのリファレンスクロックを乗じることにする。ＲＴＰ Ｐａｙｌｏａｄ ｆｏｒｍａｔ ｆｏｒ ＭＰＥＧ４ Ｅｌｅｍｅｎｔａｒｙ Ｓｔｒｅａｍ（ＲＦＣ３０１６）によるデータ通信においては、リファレンスクロックとして、９０ＫＨｚのクロックが用いられている。したがって、受信時刻の単位がｍｓであった場合、タイムスタンプのリファレンスクロックに変換するためには、次の式（６）を用いればよい。
【０１５０】
受信時刻［リファレンスクロック］＝ＴＲ１（ｍｓ）×９０・・・（６）
【０１５１】
ここで、第１のＲＴＰパケット乃至第ｎのＲＴＰパケットを、ＲＴＰ１乃至ＲＴＰｎとし、それぞれのタイムスタンプをＴＳ１乃至ＴＳｎとした場合、ネットワーク中の輻輳やパケットロスがなかったとすると、次の式（７）は、ほぼ定数となる。
【０１５２】

【０１５３】
式（７）で示されるＴＳｎ−（ＴＲｎ×９０）が、データ受信装置１０２が内部に有する時計を用いて、ＲＴＰのリファレンスクロックで計測した場合の、データ送信装置１０１とデータ受信装置１０２との時間差である。ただし、ジッタ、および、送受信間のクロックずれは、式（７）においては無視されている。
【０１５４】
しかしながら、この時間差が、絶対時刻において、どの程度の時間差に相当するのかはわからない。つまり、式（７）の値が０だったとしても、パケットの遅延の大きさを求めることができない。したがって、データ演算部１３２は、ある時刻におけるパケットの遅延を含むデータ送信装置１０１とデータ受信装置１０２との時間差を、初期端末間時間差とし、これを基に、式（８）によって、相対的な遅延を測定する。
【０１５５】

【０１５６】
ただし、ＴＲはＲＴＰパケットの受信時刻（単位はｍｓ）、ＴＳはＲＴＰのタイムスタンプである。
【０１５７】
ところで、上述したように、式（７）において、ジッタ、および、送受信間のクロックずれは無視され、式（７）から求められる送受信間タイムスタンプの差は一定であると仮定されているが、実際には、式（７）に示される端末間の時間差とパケットの遅延との和は、データ送信装置１０１とデータ受信装置１０２との基本クロックのずれにより、ゆっくりと単調減少するか、もしくは、単調増加するかのいずれか一方の傾向を示す。例えば、図１４に示されるように、受信時刻に対して、遅延がゆっくりと減少していく状況が発生する。
【０１５８】
このような場合、式（８）を用いて算出された初期時間差を、初期の値のまま使い続けると、得られる相対遅延も、単調減少、もしくは単調増加してしまう。そこで、式（８）を用いて算出される初期時間差は、定期的に更新する必要がある。
【０１５９】
また、データ演算部１３２は、更に、パケットロス数、受信パケット数、および受信データ量（受信バイト数）の供給を受けるので、タイマ１３３を参照し、例えば、５秒や１０秒などの所定の時間における、ＲＴＰパケットの相対遅延の総和、パケットロス数、受信パケット数、および受信データ量（受信バイト数）の総和を求める。
【０１６０】
次に、相対遅延増加率演算部１３４は、データ演算部１３２から供給された、所定時間内の受信パケット数およびＲＴＰパケットの相対遅延の総和を基に、次の式（９）を用いて、所定の期間（時刻ｔ１から時刻ｔ２の間）のＲＴＰパケットの平均相対遅延を演算する。
【０１６１】
【数１】

【０１６２】
更に、相対遅延増加率演算部１３４は、式（９）を用いて算出した、ＲＴＰパケットの平均相対遅延と、１つ前の区間の平均相対遅延とを比較し、平均相対遅延増加率Ｄを算出する。
【０１６３】
そして、パケットロス率演算部１３５は、データ演算部１３２から供給された情報から、パケットロス率Ｌを求める。
【０１６４】
一般的に、ＲＴＰパケットのパケットロスは、アクセスラインのビット誤りによるフレームの破棄、または、ボトルネックリンクにおけるパケットの破棄により発生する。アクセスラインのビット誤りに関しては、誤り訂正や、リンクレイヤーでの再送（特に無線リンクの場合）などにより、ある程度防ぐことが可能ではあるが、実際の無線リンクにおいては、電波の状況により、ある程度のパケットロスが発生してしまう。また、ボトルネックリンクにおけるパケットの破棄が発生した場合、ボトルネックリンクのＱｕｅｕｅがあふれていることによりパケットロスが発生するので、例えば、ＦＩＦＯ（Ｆｉｒｓｔ Ｉｎ Ｆｉｒｓｔ Ｏｕｔ）型のＱｕｅｕｅを用いている状況では、あふれた分だけパケットロスが発生することになる。
【０１６５】
つまり、アクセスラインのビット誤りによるパケットロスは、ある程度の確率で発生することが想定されるので、送信側のレートを下ることにより、パケットロス率が低くなるとは限らない。しかしながら、ボトルネックリンクにおけるパケットの破棄が発生した場合は、送信側のレートを下げない限り、パケットロスが発生しつづけることになる。
【０１６６】
ここで、パケットロス率演算部１３５は、上述した図６のパケットロス率演算部８２と同様にして、式（１）を用いて、パケットロス率Ｌを演算する。
【０１６７】
状態遷移制御部１３６は、相対遅延増加率演算部１３４およびパケットロス率演算部１３５から供給された、ＲＴＰパケットの平均相対遅延増加率Ｄ、および、パケットロス率Ｌを基に、受信レートの制御状態を遷移するか否かを判断する。
【０１６８】
状態遷移制御部１３６は、式（１０）を用いて、平均相対遅延増加率と所定の閾値Ｈ１とを定期的に比較し、平均相対遅延増加率が閾値Ｈ１より大きい場合、受信レートを下げるように制御状態を遷移する。
【０１６９】
平均相対遅延増加率Ｄ＞閾値Ｈ１ ・・・（１０）
【０１７０】
更に、状態遷移制御部１３６は、式（１１）を用いて、所定の短い時間（例えば、数１００ｍｓｅｃ乃至１ｓｅｃの、平均相対遅延増加率を求める場合よりも充分短い時間）内のパケットロス率と、所定の閾値Ｈ２とを定期的に比較し、パケットロス率が閾値Ｈ２より大きい場合、受信レートを下げるように制御状態を遷移する。
【０１７１】
パケットロス率Ｌ＞閾値Ｈ２・・・（１１）
【０１７２】
データ送信装置１０１とデータ受信装置１０２とのデータ伝送経路の伝送帯域がある程度の大きさを有するときは、ネットワーク中のＱｕｅｕｅの長さが相対的に短くなるため、データ伝送経路の状態が不安定になることにより、パケットロス率が増加しやすくなるので、式（１０）と式（１１）とにおいて、式（１１）が成立しやすい。それに対して、データ送信装置１０１とデータ受信装置１０２とのデータ伝送経路の伝送帯域がある比較的小さいとき、ネットワーク中のＱｕｅｕｅの長さが相対的に長くなるため、データ伝送経路の状態が不安定になることにより、平均相対遅延増加率Ｄが増加しやすくなるので、式（１０）と式（１１）とにおいて、式（１０）が成立しやすい。
【０１７３】
このように、式（１０）と式（１１）との２つの評価式により、受信レートを下げる必要があるか否かを判断するようにすることにより、データ伝送帯域が大きな場合においても、小さな場合においても、データ伝送路の状態に最適なデータ受信レートの制御が可能となる。
【０１７４】
また、状態遷移制御部１３６は、式（１０）および式（１１）による比較結果を基に、データ伝送状態が安定している場合、受信レートを一定に保ったり、受信レートを上げるように状態を遷移する場合がある。
【０１７５】
状態遷移制御部１３６による状態遷移図を図１５に示す。
【０１７６】
状態には、データ受信レートを上げるように制御するＵＰ状態、データ受信レートを下げるように制御するＤｏｗｎ状態、および、データ受信レートを変更しないように制御するＨｏｌｄの３状態がある。
【０１７７】
制御のはじめにおいて、状態はＨｏｌｄ状態である。状態遷移制御部１３６は、Ｈｏｌｄ状態を一定時間Ｔだけ保っている間の、式（１０）および式（１１）による比較結果を基に、平均相対遅延増加率Ｄ≦閾値Ｈ１かつパケットロス率Ｌ≦閾値Ｈ２である場合、状態をＵＰ状態に遷移し、平均相対遅延増加率Ｄ＞閾値Ｈ１、または、パケットロス率Ｌ＞閾値Ｈ２である場合、状態をＤｏｗｎ状態に遷移する。
【０１７８】
ＵＰ状態において、状態遷移制御部１３６は、式（１０）および式（１１）による比較結果を基に、平均相対遅延増加率Ｄ≦閾値Ｈ１かつパケットロス率Ｌ≦閾値Ｈ２である場合、状態をＵＰ状態のまま保持し、平均相対遅延増加率Ｄ＞閾値Ｈ１、または、パケットロス率Ｌ＞閾値Ｈ２である場合、状態をＤｏｗｎ状態に遷移する。
【０１７９】
また、Ｄｏｗｎ状態において、状態遷移制御部１３６は、式（１０）および式（１１）による比較結果を基に、平均相対遅延増加率Ｄ≦閾値Ｈ１かつパケットロス率Ｌ≦閾値Ｈ２である場合、状態をＨｏｌｄ状態に遷移し、平均相対遅延増加率Ｄ＞閾値Ｈ１、または、パケットロス率Ｌ＞閾値Ｈ２である場合、状態をＤｏｗｎ状態のまま保持する。
【０１８０】
状態遷移制御部１３６は、現在の状態を示す状態遷信号を、受信レート設定部１３７に出力する。
【０１８１】
受信レート設定部１３７は、状態遷移制御部から供給された状態遷移信号に基づいて、受信レートを設定し、レート制御命令を生成する。
【０１８２】
状態がＤｏｗｎ状態である場合、受信レート設定部１３７は、次の式（１２）により、再設定レートＲを算出する。
【０１８３】
再設定レートＲ＝受信レート×Ｃ１・・・（１２）
【０１８４】
ここで、式（１２）のＣ１は、予め定められている１以下の定数か、もしくは、別途、平均相対遅延増加率の逆数を演算することなどにより求められた１以下の値である。
【０１８５】
そして、状態がＵＰ状態である場合、受信レート設定部１３７は、次の式（１３）により、再設定レートＲを算出する。
【０１８６】
再設定レートＲ＝受信レート×Ｃ２・・・（１３）
【０１８７】
ただし、Ｃ２は、予め定められている１以上の定数か、パケットロス率Ｌの逆数を求めることなどより算出される１以上の値である。
【０１８８】
なお、受信レートの上限に関しては、予め、ＨＴＴＰやパケットペアなどを用いてボトルネックリンクのリンクスピード（パケットロスや、遅延の増加が発生しないネットワーク状況におけるデータ伝送スピード）を測定しておき、これを上限としてもよいし、上限を設けないようにしてもよい。
【０１８９】
このようにして、図１５の状態遷移図に示される状態に基づいて、データ受信レートが制御されることにより、データ受信レートは、そのときのデータ伝送路の条件により、最適に制御される。図１６は、図１５の状態遷移図に示される状態とデータ受信レートの関係を示す図である。
【０１９０】
データ伝送路の状態は、その時刻によって変動する場合がある。本発明を適応したデータ受信装置１０２によるデータ受信レート制御によると、データ受信レートは一定の値に収束するように制御されるのではなく、データ伝送路の状況が良好で、遅延やパケットロスが発生しない状態においては、状態がＵＰ状態となり、データ転送レートは増加するように制御される。
【０１９１】
このようにして、輻輳予測部１２１は、必要に応じて、レート制御命令を生成する。輻輳予測部１２１は、生成したレート制御命令を、レート制御命令送信部１２２に出力する。
【０１９２】
レート制御命令送信部１２２は、レート制御命令を、ＲＴＣＰのＡＰＰパケットとして、データ送信装置１０１に送信する。
【０１９３】
データ送信装置１０１のデータ受信部４３は、レート制御命令を受信し、エンコード制御部１１１および送信レート制御部４５に供給する。
【０１９４】
エンコード制御部１１１は、データ受信部４３からレート制御命令の供給を受け、そのレート制御命令が、レートを下げることを命令している場合、パケットロスなどにより、データ受信側においてエラーが発生する可能性があるため、エンコーダ４１のエンコードを制御して、Ｉフレームを生成させる。
【０１９５】
送信レート制御部４５は、データ受信部４３から供給されたレート制御命令を基に、送信レート制御信号を生成し、データ生成部４１に供給する。
【０１９６】
データ生成部４１は、送信レート制御部４５の制御に基づいて、データの生成量を必要に応じて変更する。したがって、データ送信部４２により送信されるデータのデータレートが制御される。また、データ生成部４１は、エンコード制御部１１１の制御に基づいて、必要に応じて、通常のエンコード順にかかわらず、Ｉフレームを生成する。このようにして、データ生成部４１で生成されたデータは、データ送信部４２に供給され、パケットに分割されて、送信される。
【０１９７】
このようにして、データ送信装置１０１は、ネットワークの状態に応じた、最適な送信レートで、パケットデータを送信し、かつ、必要に応じて、送信するデータにＩフレームを挿入することができる。ここでは、データ受信装置１０２は、レート制御命令を、ＲＴＣＰのＡＰＰパケットとして、データ送信装置１０１に送信するものとして説明したが、レート制御命令を、ＲＴＣＰのＲＲパケットを用いて、データ送信装置１０１に送信することができるようにしても良いことは言うまでもない。
【０１９８】
次に、図１７のフローチャートを参照して、データ受信装置１０２で実行される統計情報生成処理について説明する。
【０１９９】
ステップＳ１１１において、データ受信部５１は、ネットワーク３３を介して、データ送信装置１０１から送信されたパケットが受信されたか否かを判断する。ステップＳ１１１において、パケットが受信されていないと判断された場合、パケットが受信されたと判断されるまで、ステップＳ１１１の処理が繰り返される。
【０２００】
ステップＳ１１１において、パケットが受信されたと判断された場合、データ受信部５１は、受信したＲＴＰパケットの受信時刻、タイムスタンプ、パケットサイズ、シーケンス番号、および、受信したＲＴＣＰのＳＲパケットのＮＴＰ情報など、データの遅延やパケットロス率を求めるために必要な情報を輻輳予測部１２１に供給し、ステップＳ１１２において、輻輳予測部１２１のデータ演算部１３２は、受信したＲＴＰパケットの受信時刻を内部のメモリに記録する。
【０２０１】
輻輳予測部１２１のデータ演算部１３２は、ステップＳ１１３において、上述したように、データ送信装置１０１からデータ受信装置１０２へのデータ伝送における片方向遅延時間、パケットロス数、受信パケット数、および、受信バイト数を計算し、ステップＳ１１４において、片方向遅延時間、パケットロス数、受信パケット数、および、受信バイト数のそれぞれの総和を更新する。ステップＳ１１４の処理の終了後、処理は、ステップＳ１１１に戻り、それ以降の処理が繰り返される。
【０２０２】
このような処理により、輻輳予測部１２１のデータ演算部１３２において、統計情報が生成される。
【０２０３】
次に、図１８のフローチャートを参照して、図１７を用いて説明した統計情報生成処理において生成された統計情報を基に実行されるレート制御処理について説明する。
【０２０４】
ステップＳ１３１において、状態遷移制御部１３６は、制御の状態を、Ｈｏｌｄ状態にセットする。
【０２０５】
ステップＳ１３２において、パケットロス率演算部１３５は、データ演算部１３２から、統計情報のうち、パケットロス数および受信パケット数を取得し、式（１）を用いて、パケットロス率Ｌを算出する。
【０２０６】
ステップＳ１３３において、相対遅延増加率演算部１３４は、データ演算部１３２から、統計情報のうち、ＲＴＰパケットの相対遅延の所定時間内の総和を取得し、式（９）を用いてＲＴＰパケットの平均相対遅延を算出し、ＲＴＰパケットの平均相対遅延と、１つ前の区間の平均相対遅延とを比較して、平均相対遅延増加率Ｄを算出する。
【０２０７】
ステップＳ１３４において、状態遷移制御部１３６は、同一状態で一定時間が経過したか否かを判断する。ステップＳ１３４において、同一状態で一定時間が経過していないと判断された場合、処理は、ステップＳ１３２に戻り、それ以降の処理が繰り返される。
【０２０８】
ステップＳ１３４において、同一状態で一定時間が経過したと判断された場合、ステップＳ１３５において、状態遷移制御部１３６は、ステップＳ１３２において算出されたパケットロス率ＬおよびステップＳ１３３において算出された平均相対遅延増加率Ｄを基に、図１５を用いて説明したように、必要に応じて、制御状態を遷移し、受信レート設定部１３７に、状態遷移信号を供給する。
【０２０９】
ステップＳ１３６において、受信レート設定部１３７は、状態遷移制御部１３６から供給される状態遷移信号に基づいて、受信ビットレートを変更するか否かを判断する。状態遷移信号がＨｏｌｄ状態を示し、ステップＳ１３６において、受信ビットレートを変更しないと判断された場合、処理は、ステップＳ１３２に戻り、それ以降の処理が繰り返される。
【０２１０】
状態遷移信号がＵＰ状態、または、Ｄｏｗｎ状態を示し、ステップＳ１３６において、受信ビットレートを変更すると判断された場合、ステップＳ１３７において、受信レート設定部１３７は、上述した式（１２）または式（１３）を用いて、ビットレートを演算し、レート制御命令を生成して、レート制御命令送信部１２２に供給する。レート制御命令送信部１２２は、ネットワーク３３を介して、レート制御命令をデータ送信装置１０１に送信する。
【０２１１】
ステップＳ１３７の処理の終了後、処理は、ステップＳ１３２に戻り、それ以降の処理が繰り返される。
【０２１２】
このような処理により、データ受信装置１０２において、データ伝送路（ネットワーク３３）の状態に応じたレート制御命令が生成されて、データ送信装置１０１に送信される。
【０２１３】
次に、図１９のフローチャートを参照して、レート制御命令を受信したデータ送信装置１０１が実行するＩフレーム挿入処理２について説明する。
【０２１４】
ステップＳ１５１において、データ送信装置１０１のデータ受信部４３は、ネットワーク３３を介して、データ受信装置１０２から、ＲＲパケット、または、ＡＰＰパケットを受信したか否かを判断する。ステップＳ１５１において、ＲＲパケット、または、ＡＰＰパケットを受信していないと判断された場合、ＲＲパケット、または、ＡＰＰパケットを受信したと判断されるまで、ステップＳ１５１の処理が繰り返される。
【０２１５】
ステップＳ１５１において、ＲＲパケット、または、ＡＰＰパケットを受信したと判断された場合、ステップＳ１５２において、データ受信部４３は、受信したＲＲパケット、または、ＡＰＰパケットに、レート制御命令が含まれているか否かを判断する。ステップＳ１５２において、受信したＲＲパケット、または、ＡＰＰパケットに、レート制御命令が含まれていないと判断された場合、処理は、ステップＳ１５１に戻り、それ以降の処理が繰り返される。
【０２１６】
ステップＳ１５２において、受信したＲＲパケット、または、ＡＰＰパケットに、レート制御命令が含まれていると判断された場合、データ受信部４３は、レート制御命令を、送信レート制御部４５およびエンコード制御部１１１に供給するので、ステップＳ１５３において、送信レート制御部４５は、レート制御命令に基づいて、エンコーダ４１を制御して、生成されるデータ量を減少または増加させることにより、データ送信部４２から送信されるデータの送信ビットレートを変更する。
【０２１７】
エンコード制御部１１１は、ステップＳ１５４において、エンコーダ４１から、通常のエンコード順による次のＩフレームの送信時刻を取得し、ステップＳ１５５において、レート制御命令がレートを下げることを指令しているか否か、および、通常のエンコード順によるＩフレームの送信時刻までの時間が、例えば、１秒などの、所定の時間より長いか否かを基に、Ｉフレーム挿入処理を実行するか否かを判断する。
【０２１８】
エンコード制御部１１１は、通常のエンコード順によるＩフレームの送信時刻までの時間が、例えば、１秒などの、所定の時間より長く、かつ、レート制御命令がレートを下げることを指令している場合、Ｉフレーム挿入処理を実行すると判断する。ステップＳ１５５において、Ｉフレーム挿入処理を実行しないと判断された場合、処理は、ステップＳ１５１に戻り、それ以降の処理が繰り返される。
【０２１９】
ステップＳ１５５において、Ｉフレーム挿入処理を実行すると判断された場合、ステップＳ１５６において、エンコード制御部１１１は、エンコーダ４１を制御して、通常のエンコード順にかかわらず、Ｉフレームを生成させ、データ送信部４２に供給させて、送信するＲＴＰパケットに、Ｉフレームを挿入させる。ステップＳ１５６の処理の終了後、処理は、ステップＳ１５１に戻り、それ以降の処理が繰り返される。
【０２２０】
このような処理により、レート制御命令を受信したデータ送信装置１０１は、送信されるデータのレートを制御するとともに、通常のエンコード順において、次のＩフレームの送信までの時間が、所定の時間より長く、かつ、レート制御命令がレートを下げることを指令している場合、Ｉフレームを生成して、データ受信装置１０２に送信するようにしたので、データ受信装置１０２において、パケットロスなどによるデータエラーの伝播が長時間続くような状況の発生を防止することが可能となる。
【０２２１】
また、データ受信装置１０２によるレート制御命令の生成方法は、以上において説明した方法以外の方法であっても良いことは言うまでもない。データ送信装置１０１は、データ受信装置１０２が送信したレート制御命令の種類によらず、これを受信して、必要に応じて、送信するデータのレートを制御するとともに、Ｉフレームを生成して、送信することができる。
【０２２２】
第１の実施の形態では、データ受信装置３２において、リフレッシュ命令が生成されて、リフレッシュ命令を受信したデータ送信装置が、通常のエンコード順にかかわらず、必要に応じてＩフレームを生成して、データ受信装置３２に送信した。また、第２の実施の形態では、データ受信装置１０２において、レート制御命令が生成されて、レート制御命令を受信したデータ送信装置が、データ伝送レートを制御するのに加えて、通常のエンコード順にかかわらず、必要に応じてＩフレームを生成して、データ受信装置１０２に送信した。
【０２２３】
これに対して、データ受信側の装置でデータ送信側の装置に対する命令を生成するのではなく、データ受信側の装置が生成して送信するＲＲパケットを基に、データ送信側の装置が、Ｉフレームの挿入の必要があるか否かを判断して、必要に応じて、通常のエンコード順にかかわらず、Ｉフレームを生成して、送信するようにしても良い。
【０２２４】
図２０は、第３の実施の形態におけるデータ送信装置１６１およびデータ受信装置１６２の構成を示すブロック図である。
【０２２５】
なお、図２における場合と対応する部分には同一の符号を付してあり、その説明は適宜省略する。
【０２２６】
すなわち、図２０のデータ送信装置１６１は、エンコード制御部４４に代わって、ＲＲパケットに含まれている情報を基に、通常のエンコードの順番にかかわらず、Ｉフレームを生成して送信するか否かを判断し、必要に応じて、Ｉフレーム生成のための制御信号をエンコーダ４１に出力するエンコード制御部１７１が備えられている以外は、図２のデータ送信装置３１と同様の構成を有するものであり、図２０のデータ受信装置１６２は、リフレッシュ命令生成部５３およびリフレッシュ命令送信部５５が省略されている以外は、図２のデータ受信装置３２と同様の構成を有するものである。
【０２２７】
次に、データ送信装置１６１およびデータ受信装置１６２の動作について説明する。
【０２２８】
データ送信装置１６１のエンコーダ４１は、図示しないデータ入力部から、例えば、音声、画像、映像、テキストデータ、または、これらの混合したデータの入力を受けて、例えば、ＭＰＥＧ２やＭＰＥＧ４などの方法でエンコードし、データ送信部４２に供給する。エンコーダ４１は、送信レート制御部４５の制御に基づいて、生成するデータの量を決定する。生成されるデータの量を制御することにより、送信レートが制御される。
【０２２９】
データ送信部４２は、エンコーダ４１から供給されたデータを時間単位でパケットに分割し、所定の時間ごとにＳＲパケットを付加して、ネットワーク３３を介して、データ受信装置１６２に送信する。
【０２３０】
データ受信装置１６２のデータ受信部５１は、ネットワーク３３を介して、データ送信装置３１から、ＲＴＰパケットおよびＲＴＣＰ ＳＲパケットを受信する。データ受信部５１は、データ処理部５２に、例えば、映像、音声、テキストなどのデータを供給する。
【０２３１】
データ送信部５４は、ＲＴＣＰ ＲＲパケットを、ネットワーク３３を介して、データ送信装置１６１に送信する。
【０２３２】
データ送信装置１６１のデータ受信部４３は、ＲＲパケットを受信し、受信したＲＲパケットに含まれる、タイムスタンプの情報や、パケットロス、ジッタなどの情報を、エンコード制御部１７１に供給する。
【０２３３】
エンコード制御部１７１は、データ受信部４３から供給された情報を基に、パケットロス率やジッタなどの値と、所定の閾値とを比較し、その比較結果に基づいて、通常のフレーム順にかかわらず、次にエンコードして生成されるフレームをＩフレームとするか否かを判断し、必要に応じて、次にエンコードして生成されるフレームがＩフレームとなるようにエンコードを制御するための制御信号を生成し、エンコーダ４１に供給する。また、エンコード制御部１７１は、パケットロス率やジッタなどの値と、所定の閾値との比較結果を、送信レート制御部４５に供給する。
【０２３４】
送信レート制御部４５は、エンコード制御部１７１から供給された情報を基に、送信レートを変更する必要があるか否かを判断し、送信レートの変更が必要である場合、データ生成量を制御するための制御信号を生成し、エンコーダ４１に供給する。
【０２３５】
エンコーダ４１は、エンコード制御部４４から供給された制御信号に基づいて、必要に応じて、Ｉフレームを挿入するようにしながら、供給されるデータのエンコードを行い、それ以外の場合は、予め定められたエンコード方式に基づいて、エンコードを行い、エンコードされた符号化データを、データ送信部４２に供給する。
【０２３６】
データ送信部４２は、供給された符号化データを、時間ごとに分割してＲＴＰパケットを生成し、ＲＴＰパケットを、ネットワーク３３を介して、データ受信装置１６２に送信する。また、データ送信部４２は、例えば、５秒などの所定の時間ごとに、上述したＳＲパケットを生成し、ネットワーク３３を介して、データ受信装置１６２に送信する。
【０２３７】
このようにして、データ送信装置１６１は、データ受信装置１６２において、パケットロスによるエラーが発生しにくい、または、発生しても、エラー伝播時間が短くなるように、通常のエンコード処理により生成されるフレームの順序にかかわらず、Ｉフレームを適宜生成して、データ受信装置３２に送信することができる。
【０２３８】
次に、図２１のフローチャートを参照して、ＲＲパケットに含まれる情報を基に、データ送信装置１６１が実行する、Ｉフレーム挿入処理３について説明する。
【０２３９】
ステップＳ１８１において、データ送信装置１６１のデータ受信部４３は、データ受信装置１６２から送信されるＲＲパケットを受信したか否かを判断する。ステップＳ１８１において、ＲＲパケットを受信していないと判断された場合、受信したと判断されるまで、ステップＳ１８１の処理が繰り返される。
【０２４０】
ステップＳ１８１において、ＲＲパケットを受信したと判断された場合、データ受信部４３は、ＲＲパケットに記載されている情報を、エンコード制御部１７１に供給するので、ステップＳ１８２において、エンコード制御部１７１は、供給されたデータを基に、例えば、上述した式（１）を用いて、パケットロス率を演算し、パケットロス率が、所定の閾値以上であるか否かを判断する。ステップＳ１８２において、パケットロス率が、所定の閾値以上であると判断された場合、処理は、ステップＳ１８４に進む。
【０２４１】
ステップＳ１８２において、パケットロス率が、所定の閾値以下であると判断された場合、ステップＳ１８３において、エンコード制御部１７１は、ジッタが、所定の閾値以上であるか否かを判断する。ステップＳ１８３において、ジッタが所定の閾値以下であると判断された場合、処理は、ステップＳ１８１に戻り、それ以降の処理が繰り返される。
【０２４２】
ステップＳ１８２において、パケットロス率が所定の閾値以上であると判断された場合、または、ステップＳ１８３において、ジッタが所定の閾値以上であると判断された場合、ステップＳ１８４において、エンコード制御部１７１は、送信レート制御部４５に、パケットロス率が所定の閾値以上である、または、ジッタが所定の閾値以上であることを通知する。送信レート制御部４５は、エンコーダ４１による処理を制御して、生成されるデータ量を減少させることにより、データ送信部４２から送信されるデータの送信ビットレートを下げる。
【０２４３】
なお、ここでは、エンコード制御部１７１が、送信レート制御部４５に、パケットロス率が所定の閾値以上である、または、ジッタが所定の閾値以上であることを通知するものとして説明したが、送信レート制御部４５が、ステップＳ１８２およびステップＳ１８３と同様の処理を実行するようにして、その結果、必要に応じて、送信レートを下げることができるようにしても良いことはもちろんである。
【０２４４】
ステップＳ１８５において、エンコード制御部１７１は、データ送信部４２から送信される次のＩフレームの送信時刻を取得する。
【０２４５】
ステップＳ１８６において、エンコード制御部１７１は、パケットロス率、ジッタ、および、次のＩフレームの送信時刻までの時間を基に、Ｉフレーム挿入処理を実行するか否かを判断する。エンコード制御部１７１は、パケットロス率またはジッタのうちのいずれか一方が所定の閾値以上であり、かつ、次のＩフレームの送信時刻までの時間が、例えば、１秒などの、所定の時間よりも長かった場合に、Ｉフレーム挿入処理を実行する。ステップＳ１８６において、Ｉフレーム挿入処理を実行しないと判断された場合、処理は、ステップＳ１８１に戻り、それ以降の処理が繰り返される。
【０２４６】
また、エンコード制御部１７１は、例えば、パケットロス率およびジッタが、いずれも所定の閾値以上であり、かつ、次のＩフレームの送信時刻までの時間が、例えば、１秒などの、所定の時間よりも長かった場合に、Ｉフレーム挿入処理を実行するものとしても良い。
【０２４７】
ステップＳ１８６において、Ｉフレーム挿入処理を実行すると判断された場合、ステップＳ１８７において、エンコード制御部１７１は、エンコーダ４１を制御して、フレーム内符号化を実行させて、データ送信部４２から送信させるデータにＩフレームを挿入して、処理は、ステップＳ１８１に戻り、それ以降の処理が繰り返される。
【０２４８】
このような処理により、データ送信装置１６１は、データ受信装置１６２において、パケットロスによるエラーが発生しにくい、または、発生しても、エラー伝播時間が短くなるように、通常のエンコード処理により生成されるフレームの順序にかかわらず、Ｉフレームを適宜生成して、データ受信装置１６２に送信することができる。
【０２４９】
また、ここでは、データ受信装置１６２がデータ送信装置１６１に送信するＲＴＣＰ ＲＲパケットに記載されている情報を基に、データ送信装置１６１において、送信されるデータにＩフレームを挿入するか否かが判断されるものとして説明したが、データ受信装置１６２が、データ送信装置１６１に対して、ＲＴＣＰ ＲＲパケット以外のデータ（例えば、ＲＴＣＰ ＡＰＰパケットなど）にＩフレームを挿入するか否かを判断するために必要な情報を記載して送信するようにしてもよい。
【０２５０】
以上においては、パケットデータを送信するデータ送信装置３１、データ送信装置１０１、または、データ送信装置１６１と、パケットデータを受信するデータ受信装置３２、データ受信装置１０２、または、データ受信装置１６２における場合について説明したが、パケットデータの送受信が可能なデータ送受信装置間で、パケットデータを送受信する場合においても、本発明は適用可能である。
【０２５１】
上述した一連の処理は、ソフトウェアにより実行することもできる。そのソフトウェアは、そのソフトウェアを構成するプログラムが、専用のハードウェアに組み込まれているコンピュータ、または、各種のプログラムをインストールすることで、各種の機能を実行することが可能な、例えば汎用のパーソナルコンピュータなどに、記録媒体からインストールされる。
【０２５２】
この記録媒体は、図２２に示すように、コンピュータとは別に、ユーザにプログラムを提供するために配布される、プログラムが記録されている磁気ディスク２２１（フレキシブルディスクを含む）、光ディスク２２２（ＣＤ−ＲＯＭ（Ｃｏｍｐａｃｔ Ｄｉｓｋ−Ｒｅａｄ Ｏｎｌｙ Ｍｅｍｏｒｙ），ＤＶＤ（Ｄｉｇｉｔａｌ Ｖｅｒｓａｔｉｌｅ Ｄｉｓｋ）を含む）、光磁気ディスク２２３（ＭＤ（Ｍｉｎｉ−Ｄｉｓｋ）（商標）を含む）、もしくは半導体メモリ２２４などよりなるパッケージメディアなどにより構成される。
【０２５３】
図２２は、上記処理を実行するパーソナルコンピュータ２０１の構成例を表している。パーソナルコンピュータ２０１のＣＰＵ（Ｃｅｎｔｒａｌ Ｐｒｏｃｅｓｓｉｎｇ Ｕｎｉｔ）２１１は、ＲＯＭ（Ｒｅａｄ Ｏｎｌｙ Ｍｅｍｏｒｙ）２１２に記憶されているプログラム、またはＨＤＤ（Ｈａｒｄ Ｄｉｓｃ Ｄｒｉｖｅ）２１８から、ＲＡＭ（Ｒａｎｄｏｍ Ａｃｃｅｓｓ Ｍｅｍｏｒｙ）２１３にロードされたプログラムにしたがって各種の処理を実行する。ＲＡＭ２１３にはまた、ＣＰＵ２１１が各種の処理を実行する上において必要なデータなども適宜記憶される。
【０２５４】
ＣＰＵ２１１、ＲＯＭ２１２、およびＲＡＭ２１３は、内部バス２１４を介して相互に接続されている。この内部バス２１４にはまた、入出力インタフェース２１５も接続されている。
【０２５５】
入出力インタフェース２１５には、キーボード、マウスなどよりなる入力部２１６、ＣＲＴ（Ｃａｔｈｏｄｅ Ｒａｙ Ｔｕｂｅ）、およびＬＣＤ（Ｌｉｑｕｉｄ Ｃｒｙｓｔａｌ Ｄｉｓｐｌａｙ）などよりなるディスプレイ、並びにスピーカなどよりなる出力部２１７、ハードディスクにより構成されるＨＤＤ２１８、モデム、ターミナルアダプタなどより構成されるネットワークインターフェース２２０が接続されている。ネットワークインターフェース２２０は、例えば、インターネットなどのネットワークを介しての通信処理を行う。
【０２５６】
入出力インタフェース２１５にはまた、必要に応じてドライブ２１９が接続され、磁気ディスク２２１、光ディスク２２２、光磁気ディスク２２３、または、半導体メモリ２２４などが適宜装着され、それらから読み出されたコンピュータプログラムが、必要に応じてＨＤＤ２１８にインストールされる。
【０２５７】
また、本明細書において、記録媒体に記録されるプログラムを記述するステップは、記載された順序に沿って時系列的に行われる処理はもちろん、必ずしも時系列的に処理されなくとも、並列的または個別に実行される処理をも含むものである。
【０２５８】
なお、本明細書において、システムとは、複数の装置により構成される装置全体を表すものである。
【０２５９】
【発明の効果】
このように、本発明によれば、符号化フレームを送信することができる。特に、データ通信状況に対応して、フレーム内符号化により生成されたイントラフレームを送信することができる。
【０２６０】
また、他の本発明によれば、データを受信することができる他、データ通信状況を基に、フレーム内符号化により生成されるイントラフレームの送信を指令する情報を生成し、データ送信元に送信することができる。
【図面の簡単な説明】
【図１】Ｉフレーム間隔について説明するための図である。
【図２】本発明を適用した第１のデータ通信システムのデータ送信装置およびデータ受信装置の構成を示すブロック図である。
【図３】ＳＲパケットについて説明する図である。
【図４】ＲＲパケットについて説明する図である。
【図５】ＡＰＰパケットについて説明する図である。
【図６】図２のリフレッシュ命令生成部の構成を示すブロック図である。
【図７】リフレッシュ命令発行処理１について説明するフローチャートである。
【図８】リフレッシュ命令発行処理１について説明するフローチャートである。
【図９】リフレッシュ命令発行処理２について説明するフローチャートである。
【図１０】リフレッシュ命令発行処理２について説明するフローチャートである。
【図１１】Ｉフレーム挿入処理１について説明するフローチャートである。
【図１２】本発明を適用した第２のデータ通信システムのデータ送信装置およびデータ受信装置の構成を示すブロック図である。
【図１３】図１２の輻輳予測部の構成を示すブロック図である。
【図１４】受信時刻による、遅延の単純減少について説明する図である。
【図１５】状態遷移について説明する図である。
【図１６】状態遷移とデータ送信レートについて説明する図である。
【図１７】統計情報生成処理について説明するフローチャートである。
【図１８】レート制御処理について説明するフローチャートである。
【図１９】Ｉフレーム挿入処理２について説明するフローチャートである。
【図２０】本発明を適用した第３のデータ通信システムのデータ送信装置およびデータ受信装置の構成を示すブロック図である。
【図２１】Ｉフレーム挿入処理３について説明するフローチャートである。
【図２２】パーソナルコンピュータの構成を示すブロック図である。
【符号の説明】
３１ データ送信装置， ３２ データ受信装置， ３３ ネットワーク， ４１ エンコーダ， ４２ データ送信部， ４３ データ受信部， ４４ エンコード制御部， ４５ 送信レート制御部， ５１ データ受信部， ５２ データ処理部， ５３ リフレッシュ命令生成部， ５４ データ送信部， ５５ リフレッシュ命令送信部， ８１ データ取得部， ８２ パケットロス率演算部， ８４ 片方向遅延演算部， ８６ Ｉフレーム間隔測定部， ８９ リフレッシュ命令発行部， ８９ Ｉフレーム受信時刻予測部， １０１ データ送信装置， １０２ データ受信装置， １１１ エンコード制御部， １２１ 輻輳予測部， １２２ レート制御命令送信部， １３２ データ演算部，１３４ 相対遅延増加率演算部， １３５ パケットロス率演算部， １３６状態遷移制御部， １３７ 受信レート設定部， １６１ データ送信装置，１６２ データ受信装置， １７１ エンコード制御部[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an information processing apparatus and an information processing method, a data communication system, a recording medium, and a program, and more particularly, to an information processing apparatus and a data processing apparatus suitable for transmitting and receiving data encoded using inter-frame correlation. The present invention relates to an information processing method, a data communication system, a recording medium, and a program.
[0002]
[Prior art]
As an adaptive rate control method in data transmission / reception using RTP (Real Time Transfer Protocol) / RTCP (RTP Control Protocol), for example, a device on a data receiving side transmits a packet loss rate to a device on a data transmitting side. A method has been proposed in which a transmission-side device estimates a data transmission state and controls a transmission rate on the basis of information described in an RTCP RR (Receiver Report) packet for notifying a jitter or the like (for example, And Patent Document 1).
[0003]
[Patent Document 1]
JP-A-2002-204278
[0004]
According to this method, it is possible to reduce the data transmission rate when the packet is lost or the jitter is increased due to network congestion, using values such as the packet loss rate and the jitter included in the RR packet. Become.
[0005]
[Problems to be solved by the invention]
For example, when data such as MPEG (Moving Picture Coding Experts Group / Moving Picture Experts Group) that is subjected to compression processing using inter-frame correlation is transmitted, a macroblock error occurs due to packet loss. It may propagate over several frames.
[0006]
In general, when a macroblock in which a packet loss has occurred is referred to as a reference frame, the time required to recover the error caused by the macroblock is generated by intra-frame encoding, and other frames are generated. It depends on the interval between so-called intra-frames (also called I-frames) that can be decoded without reference.
[0007]
For example, as shown in FIG. 1, in MPEG2, a GOP (Group Of Pictures) includes an I frame obtained by performing a DCT operation on data before compression for each block, and an I frame that is earlier than the current frame already encoded. Motion-compensated inter-frame prediction of P-frame and P-frame obtained by encoding a motion vector and a motion vector, and motion-compensated inter-frame prediction of both I-frames and P-frames before and after the already-encoded current frame It is composed of a B frame in which an error signal and a motion vector are encoded.
[0008]
Among them, the I frame is the only frame that does not need the information of the previous frame at the time of decoding because of intraframe coding. Therefore, even if a macroblock error occurs due to packet loss, the propagation of the error ends in the next I frame.
[0009]
The interval between I frames is, for example, 0.5 seconds in MPEG1 and MPEG2, and 1 second to several seconds in MPEG4. That is, when a packet loss occurs in a macroblock in an I frame, the same time as the I frame interval is required to recover the error.
[0010]
To cope with this problem, for example, a method of retransmitting the information of the macroblock in which an error has occurred may be considered, but in order to realize this method, the data receiving side requests the necessary macroblock from the transmitting side. Also, since the transmitting side needs to have a configuration capable of retransmitting the macroblock in which the packet has been lost, the control becomes complicated and the cost increases.
[0011]
In order to cope with this problem, a method of shortening the interval between I frame generations is also conceivable. The problem that it cannot be used occurs.
[0012]
The present invention has been made in view of such a situation, and it is possible to effectively prevent the occurrence of an error due to packet loss or the like, without providing a complicated configuration, and with the least possible I-frame generation amount. In the event that a problem occurs, it is possible to recover at an early stage.
[0013]
[Means for Solving the Problems]
A first information processing apparatus according to the present invention receives an encoding unit that encodes supplied data using inter-frame correlation, and receives, from another information processing apparatus, predetermined information regarding a data communication state in a network. A receiving unit, a control unit that controls the encoding unit based on the predetermined information received by the receiving unit, and executes an intra-frame encoding, and the encoding unit performs the encoding using the inter-frame correlation. Transmission means for transmitting data to another information processing device via a network.
[0014]
The predetermined information may be information for instructing transmission of an intra frame generated by intra-frame encoding, and the control unit includes an encoding unit when the predetermined information is received by the receiving unit. The means can be controlled to perform intra-frame coding.
[0015]
The predetermined information may be control information for controlling a data transmission rate by the transmission unit, and the control unit receives the predetermined information by the reception unit, and transmits the predetermined information to the data transmission unit. If the instruction to lower the rate has been issued, the encoding means can be controlled to execute intra-frame encoding.
[0016]
The data may be an RTP packet conforming to RFC1889, and the predetermined information may be an RTCP RR packet conforming to RFC1889.
[0017]
The control means calculates a packet loss rate based on the predetermined information received by the receiving means, and based on the calculation result, determines whether or not to cause the coding means to execute the intra-frame coding. Can be
[0018]
The control means calculates a variation in data delay based on the predetermined information received by the receiving means, and determines whether or not to cause the coding means to perform intra-frame coding based on the result. You can make it.
[0019]
The data may be an RTP packet conforming to RFC1889, and the predetermined information may be an RTCP APP packet conforming to RFC1889.
[0020]
A first information processing method according to the present invention includes: a reception control step of controlling reception of predetermined information relating to a data communication state in a network from another information processing apparatus; and a predetermined reception control step of controlling reception by the processing of the reception control step. An encoding control step of controlling the encoding means based on the information to execute intra-frame encoding.
[0021]
The program recorded on the first recording medium of the present invention can be received from another information processing apparatus by a reception control step of controlling reception of predetermined information relating to a data communication state in a network, and a reception control step. An encoding control step of controlling an encoding unit based on the controlled predetermined information to execute intra-frame encoding.
[0022]
A first program according to the present invention includes a reception control step of controlling reception of predetermined information relating to a data communication state in a network from another information processing apparatus, and a predetermined information whose reception is controlled by the processing of the reception control step. And a coding control step of controlling the coding means to perform intra-frame coding.
[0023]
In a first information processing apparatus, an information processing method, and a program according to the present invention, supplied data is encoded using inter-frame correlation, and predetermined information regarding data communication status in a network is transmitted from another information processing apparatus. Are received, and intra-frame encoding is performed based on the received predetermined information.
[0024]
The second information processing apparatus of the present invention is a receiving unit that receives predetermined information including a data transmission state transmitted from another information processing apparatus, based on the predetermined information received by the receiving unit, A generation unit for generating information for instructing another information processing apparatus to transmit an intra frame generated by intra-frame encoding, and information for instructing transmission of an intra frame generated by the generation unit, via a network. Transmission means for transmitting the information to another information processing apparatus.
[0025]
The data may be an RTP packet conforming to RFC1889, and the information instructing the transmission of the intra frame may be an RTCP RR packet conforming to RFC1889.
[0026]
The data may be an RTP packet conforming to RFC1889, and the information instructing the transmission of the intra frame may be an RTCP APP packet conforming to RFC1889.
[0027]
The data may be an RTP packet conforming to RFC1889, and the transmitting means may include an RTCP packet conforming to RFC1889 if the time until the next transmission time of the RTCP RR packet conforming to RFC1889 is longer than a predetermined time. The information for instructing the transmission of the intra frame can be transmitted by the RTCP APP packet to be transmitted. If the time until the next transmission time of the RTCP RR packet is shorter than a predetermined time, the intranet is transmitted by the RTCP RR packet. Information for instructing transmission of a frame can be transmitted.
[0028]
The data may be an RTP packet conforming to RFC1889, and the transmitting means may be capable of changing the sending time of the RTCP RR packet conforming to RFC1889, and the RTCP RR packet If the time until the next scheduled transmission time is longer than the predetermined time, the transmission time of the RTCP RR packet can be advanced so that the information for instructing the transmission of the intra frame can be transmitted.
[0029]
According to a second information processing method of the present invention, a reception control step of controlling reception of predetermined information including a data transmission state transmitted from another information processing apparatus, and reception is controlled by the processing of the reception control step And generating information for instructing another information processing apparatus to transmit an intra frame generated by intra-frame encoding based on the predetermined information.
[0030]
The program recorded on the second recording medium according to the present invention includes a reception control step of controlling reception of predetermined information including a data transmission state transmitted from another information processing apparatus, and a process of the reception control step. And generating information for instructing another information processing apparatus to transmit an intra frame generated by intra-frame encoding, based on the predetermined information whose reception is controlled by the information processing apparatus.
[0031]
A second program according to the present invention includes: a reception control step of controlling reception of predetermined information including a data transmission state transmitted from another information processing apparatus; and a reception control step of controlling reception by the processing of the reception control step. And generating a command for instructing another information processing apparatus to transmit an intra-frame generated by intra-frame encoding based on the information of (i).
[0032]
In the second information processing apparatus, the information processing method, and the program according to the present invention, predetermined information including a data transmission state is received, and based on the predetermined information, another information processing apparatus transmits an intra-frame code to another information processing apparatus. Information for instructing transmission of an intra frame generated by the conversion is generated and transmitted.
[0033]
The first data communication system according to the present invention is configured such that the first information processing device receives first information including a data transmission state transmitted from the second information processing device, Generating means for generating, based on the first information received by the first receiving means, second information for instructing the second information processing apparatus to transmit an intra frame generated by intra-frame encoding; And first transmission means for transmitting the second information generated by the generation means to the second information processing apparatus via a network, wherein the second information processing apparatus Encoding means for encoding using inter-frame correlation, second receiving means for receiving second information from the first information processing device, and second information received by the second receiving means. Control the encoding means based on the Control means to be executed and data encoded by the encoding means using the inter-frame correlation and first information including the data transmission status are transmitted to the first information processing device via the network. And a second transmitting means.
[0034]
In the first data communication system of the present invention, the first information including the data transmission status transmitted from the second information processing device is received by the first information processing device, and the received first information is transmitted. Is generated on the basis of the information of the second information processing apparatus, the second information for instructing the second information processing apparatus to transmit an intra frame generated by intra-frame encoding is generated, and the generated second information is transmitted via a network. Is transmitted to the second information processing device, and the supplied data is encoded by using the inter-frame correlation, and the second information is received from the first information processing device. Intra-frame encoding is performed based on the received second information, and the encoded data and the first information including the data transmission status are transmitted to the first information processing unit via the network. Sent to the device.
[0035]
In a second data communication system according to the present invention, the first information processing device includes a first receiving unit that receives data transmitted from the second information processing device, and a second information processing device. Encoding means for encoding the supplied data using inter-frame correlation, the first information processing apparatus comprising: A second receiving unit for receiving information on the data communication status from the processing device, and controlling the encoding unit based on the information on the data communication status received by the second receiving unit to perform intra-frame encoding. Control means for executing the data, and second transmitting means for transmitting data encoded by the encoding means using the inter-frame correlation to the first information processing device via a network. That.
[0036]
In the second data communication system of the present invention, the data transmitted from the second information processing device is received by the first information processing device, and the information about the data communication status in the network is transmitted to the second information processing device. Is transmitted, in the second information processing device, the supplied data is encoded using inter-frame correlation, information on the data communication status is received from the first information processing device, and the received data communication Intra-frame encoding is performed based on the information on the situation, and the encoded data is transmitted to the first information processing device via the network.
[0037]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, embodiments of the present invention will be described with reference to the drawings.
[0038]
FIG. 2 is a block diagram showing a configuration of the data transmitting device 31 and the data receiving device 32 in the first embodiment of the data communication system to which the present invention is applied.
[0039]
The data transmitting device 31 and the data receiving device 32 transmit and receive data according to RTP (Real-time Transport Protocol), which is a real-time data transfer protocol. The RTP is designed to transfer video and audio data in a form suitable for real time, assuming that the RTP is used, for example, in an application for remote conference using video and audio data. In RTP, data is divided into packets in units of time and transmitted. RTP is a UDP (User Datagram Protocol) type protocol in which measures against packet loss and transmission time are not performed, and is usually used as a set with a communication status report by RTCP (RTP Control Protocol).
[0040]
The network 33 may be, for example, a LAN (Local Area Network) operated in the organization, or a large-scale network in which an unspecified number of networks such as the so-called Internet are connected. Alternatively, a dedicated line for connecting a predetermined transmitting device and a receiving device may be used.
[0041]
The encoder 41 of the data transmitting apparatus 31 compresses (encodes, for example, audio, image, video, or a mixture thereof) using inter-frame correlation such as MPEG1, MPEG2, or MPEG4. (Encoding and inter-frame encoding) to generate a frame including the I frame described with reference to FIG. In addition, when the encoder 41 receives the supply of the control signal from the encoding control unit 44, the encoder 41 generates an I frame regardless of the order of frames generated by normal encoding, and supplies the I frame to the data transmission unit 42.
[0042]
The amount of encoded data generated by encoding by the encoder 41 is controlled by the transmission rate control unit 45. That is, since the amount of data to be transmitted is limited, the data transmission rate is controlled.
[0043]
When transmitting the data supplied from the encoder 41 as an RTP packet, the data transmission unit 42 adds an RTCP Sender Report (SR) packet every predetermined time, such as 5 seconds, and transmits the RTP packet via the network 33. To the data receiving device 32. The SR packet is a report for transmission / reception statistics on the data transmission side, and describes information indicating the status of data transmission.
[0044]
FIG. 3 shows the format of the SR packet.
[0045]
The header is version information (V), which is 2-bit information for identifying the version of RTP, and 1-bit padding bit (P), which is information indicating whether or not this packet contains one or more padding octets. A 5-bit reception report count (RC (reception report count)) indicating the number of reception report blocks included in the packet, and a constant 200 indicating an SR packet for identifying an SR packet and an RR packet are described. It consists of an 8-bit packet type (PT (packet type)), a packet length (length), which is 16-bit information indicating the length of the RTCP packet, and a source SSRC identifier (SSRC of sender). ing.
[0046]
The sender information (Sender Info) includes a 64-bit NTP time stamp (Network Time protocol time stamp), a 32-bit RTP time stamp (Real-time Transport Protocol time stamp), and a 32-bit transmission packet number information (sender's). packet count) and 32-bit transmission data amount information (sender's octet count).
[0047]
The report block includes a 32-bit SSRC identifier number (SSRC_n (source identifier)), a fraction lost indicating a packet loss rate, a cumulative number of packet losses (cumulative number of packets lost), and a maximum value of a sequence number (extended distance). sequence number received), an interarrival jitter indicating a change in an arrival time interval of an RTP packet, an LSR (last SR) which is information of a time stamp at the time of receiving the latest SR packet, and a delay from the LSR. It is composed of DLSR (delay sine last SR (DLSR)).
[0048]
Further, the SR packet is further provided with special extension areas (profile-specific extensions).
[0049]
As described above, the SR packet includes the NTP time stamp and the RTP time stamp.
[0050]
The data receiving unit 43 receives an RR (Receive Report) packet in RTCP or an APP (Application defined RTCP) packet transmitted from the data receiving device 32. The RR packet is a report for reception statistics from the data receiving side. An APP packet is a packet for application extension. When the RR packet or the APP packet in RTCP includes a refresh command, the data receiving unit 43 supplies the refresh command to the encoding control unit 44.
[0051]
FIG. 4 shows a format of the RR packet.
[0052]
In the format of the RR packet, a constant 201 indicating the RR packet is described in the packet type (PT (packet type)), and the source of the received RTP packet is replaced with the source SSRC identifier (SSRC of sender). Except for the description of the SSRC identifier (SSRC of packet sender), it has the same header structure as the SR packet, and does not include the sender information included in the SR packet. It has a report block and a special extension area.
[0053]
FIG. 5 shows the format of the APP packet.
[0054]
The APP packet has version information (V) which is 2-bit information for identifying the version of RTP, and 1-bit padding bit (P) which is information indicating whether or not the packet includes one or more padding octets. ), A subtype (subtype) for identifying the definition of this packet, an 8-bit packet type (PT (packet type)) in which a constant 204 indicating that this packet is an APP packet, and this RTCP packet A packet length (length) which is 16-bit information indicating the length of the packet, a SSRC identifier, a CSRC (Contributing source) identifier which is a transmission related party (original sender ID when mixed by a mixer), and an ASCII code. APP packet described It is composed of a name (name) uniquely assigned to the packet and information (application dependent data) depending on the application.
[0055]
The encoding control unit 44 generates a control signal for generating and transmitting an I frame based on the refresh command supplied from the data receiving unit 43, and supplies the control signal to the encoder 41.
[0056]
The transmission rate control unit 45 generates a transmission rate control signal based on the information supplied from the data reception unit 43, supplies the transmission rate control signal to the encoder 41, and controls the amount of generated data. Control the transmission rate.
[0057]
Here, the case of performing transmission rate control in the case of general MPEG2 or MPEG4 has been described. For example, if encoding processing using hierarchical encoding is being performed, the transmission rate control unit 45 performs data transmission. By controlling the section 42, the transmission rate is controlled.
[0058]
The data receiving unit 51 of the data receiving device 32 receives the SR packet and the RTP packet from the data transmitting device 31 via the network 33, and processes the data of the RTP packet, for example, video, audio, text, and the like, into data processing. To the unit 52. In addition, the data receiving unit 51 supplies the information indicating the reception time of the RTP packet, particularly, when the RTP packet is an I frame, to the refresh command generation unit 53, and also describes the information in the RTP packet. A refresh instruction generation unit that determines whether or not to generate a refresh instruction, such as a time stamp, a packet size, a sequence number, and time information described in an SR packet. 53.
[0059]
The data processing unit 52 performs a process on the data supplied from the data receiving unit 51. Specifically, the data processing unit 52 executes, for example, a decoding process, a descrambling process, a display process, or an audio playback process on the supplied data.
[0060]
The data transmission unit 54 generates the RR packet (RR packet not including the refresh command) described with reference to FIG. 4 for the RTP packet transmitted from the data transmission device 31, and transmits the data via the network 33. Transmit to the device 31.
[0061]
The refresh command generator 53 determines whether to generate a fresh command based on the information supplied from the data receiver 51, generates a refresh command as necessary, and supplies the refresh command to the refresh command transmitter 55. .
[0062]
The refresh command transmitting unit 55 transmits the refresh command supplied from the refresh command generating unit 53 to the data transmitting device 31 via the network 33 as an RTCP APP packet or an RR packet.
[0063]
In the data transmission device 31 and the data reception device 32 in FIG. 2, the data transmission and reception are described as being performed in a plurality of blocks. However, the data transmission device 31 includes a data transmission unit 42 and a data reception unit 43. The process to be executed may be executed by one transmission / reception function, or the data receiving device 32 may execute the process executed by the data receiving unit 51, the data transmitting unit 54, and the refresh command transmitting unit 55. It may be configured to be able to be executed by one or two transmission / reception functions.
[0064]
Next, operations of the data transmitting device 31 and the data receiving device 32 will be described.
[0065]
The encoder 41 of the data transmission device 31 receives input of, for example, audio, image, video, text data, or a mixture thereof from a data input unit (not shown), and encodes the data by a method such as MPEG2 or MPEG4. Then, the data is supplied to the data transmission unit 42.
[0066]
The data transmission unit 42 divides the data supplied from the encoder 41 into packets in units of time, adds SR packets at predetermined time intervals, and transmits the packets to the data reception device 32 via the network 33.
[0067]
The data receiving unit 51 of the data receiving device 32 receives the RTP packet and the RTCP SR packet from the data transmitting device 31 via the network 33. The data receiving unit 51 supplies data such as video, audio, and text to the data processing unit 52 and, at the same time, receives the RTP packet at the reception time, and particularly, when the RTP packet is an I frame, receives the I frame. , A time stamp, a sequence number, time information described in the SR packet, and the like are supplied to the refresh instruction generation unit 53.
[0068]
In addition, when a packet loss such as discarding of a frame due to a bit error in an access line or discarding of a packet on a bottleneck link occurs in a received packet, the data receiving unit 51 transmits information on the packet loss to a refresh command. It is supplied to the generation unit 53.
[0069]
FIG. 6 is a functional block diagram showing the function of the refresh instruction generation unit 53.
[0070]
The data acquisition unit 81 obtains the time information included in the SR packet, the reception time of the RTP packet, the time stamp, the packet size, the sequence number, and the like from the data reception unit 51 to determine the data delay and the packet loss rate. Is obtained and supplied to the packet loss rate calculation unit 82 and the one-way delay calculation unit 84. In particular, when the RTP packet is an I frame, information indicating the reception time of the I frame is provided. It is acquired and supplied to the I-frame interval measuring unit 86. Further, the data acquisition unit 81 supplies the reception time of the I frame to the I frame reception time prediction unit 89.
[0071]
The packet loss rate calculator 82 accumulates, for example, the number of received packets and the number of packet losses in a predetermined time (for example, several hundred msec to 1 sec), and calculates the packet loss rate by the following equation (1).
[0072]
Packet loss rate L = number of packet losses / (number of received packets + number of packet losses) (1)
[0073]
The packet loss rate recording unit 83 receives and supplies the packet loss rate calculated by the packet loss rate calculation unit 82, and records it.
[0074]
The one-way delay calculation unit 84 calculates a one-way delay (OWD: One Way Delay) based on the NTP value of the SR packet or based on the I-frame interval. The one-way delay recording unit 85 receives the one-way delay value calculated by the one-way delay calculation unit 84 and records the value.
[0075]
The I-frame interval measuring section 86 measures the I-frame interval based on the supplied reception time of the I-frame. The I-frame interval recording unit 87 receives a supply of the value of the I-frame interval measured by the I-frame interval measuring unit 86, and records a predetermined number of I-frame intervals, for example, 20 from the latest.
[0076]
The refresh command issuing unit 88 refers to the timer 90, compares the packet loss rate recorded in the packet loss rate recording unit 83 with a predetermined threshold at a predetermined timing, and determines that the packet loss rate is larger than the predetermined threshold. In this case, the value of the I frame interval recorded in the I frame interval recording unit 87 is supplied to the I frame reception time prediction unit 89. If the time to the predicted I-frame reception time obtained by the processing of the I-frame reception time prediction unit 89 is longer than the one-way delay time recorded in the one-way delay recording unit 85, a refresh command is generated. , To the refresh command transmitting unit 55.
[0077]
The I-frame reception time predicting unit 89 predicts the next I-frame reception time when no refresh command has been issued, based on the supplied I-frame reception time and the information of the I-frame interval. It is supplied to the instruction issuing unit 88.
[0078]
In this way, the refresh command generation unit 53 generates a refresh command based on the reception time of the received I frame and the information included in the RTP packet and the SR packet.
[0079]
Upon receiving the supply of the refresh command from the refresh command generation unit 53, the refresh command transmitting unit 55 obtains the transmission timing of the next RR packet, and the time until the transmission timing of the next RR packet is, for example, 1 second. When the time is equal to or longer than a predetermined time, a refresh command is transmitted to the data transmitting apparatus 31 using the APP packet, and the time until the transmission timing of the next RR packet is within a predetermined time, for example, 200 ms. , RR packet to transmit a refresh command to the data transmitting device 31.
[0080]
The data receiving unit 43 of the data transmitting device 31 receives the RR packet or the APP packet, and supplies the RR packet or the APP packet to the encoding control unit 44 when the received RR packet or the APP packet includes a refresh command. Further, the data receiving unit 43 supplies the transmission rate control unit 45 with information such as a time stamp, a packet loss rate, and jitter included in the received RR packet or APP packet.
[0081]
The transmission rate control unit 45 generates a transmission rate control signal based on the information supplied from the data receiving unit 43, and supplies the transmission rate control signal to the encoder 41.
[0082]
When receiving the input of the refresh command from the data receiving unit 43, the encoding control unit 44 encodes and generates the next frame regardless of the normal frame order (for example, according to MPEG2, the order of IBBPBBP...). A control signal for controlling the encoding so that the frame to be processed becomes an I frame is generated and supplied to the encoder 41.
[0083]
The encoder 41 generates an I-frame by performing intra-frame encoding when a refresh command is issued based on the control signal supplied from the encoding control unit 44, and otherwise sets a predetermined I-frame. Encoding is performed based on the encoding method, and the encoded data is supplied to the data transmitting unit 42. Further, the encoder 41 determines the generation amount of the encoded data based on the control of the transmission rate control unit 45.
[0084]
The data transmitting unit 42 divides the supplied encoded data for each time to generate an RTP packet, and transmits the RTP packet to the data receiving device 32 via the network 33. In addition, the data transmission unit 42 generates the above-described SR packet at predetermined time intervals, for example, 5 seconds, and transmits the generated SR packet to the data reception device 32 via the network 33.
[0085]
In this way, the data transmitting device 31 is less likely to cause an error due to packet loss in the data receiving device 32, or if an error occurs, so as not to impose a load on the network 33. I frames can be generated and transmitted to the data receiving device 32 regardless of the frame order in the normal encoding process so that the propagation time is shortened.
[0086]
Next, the refresh command issuing process 1 executed by the data receiving device 32 will be described with reference to the flowcharts of FIGS.
[0087]
In step S11, the data receiving unit 51 of the data receiving device 32 receives the RTP packet and the RTCP SR packet transmitted from the data transmitting device 31 via the network 33 and includes the RTP packet and the RTCP SR packet, for example, The moving image data or the like according to MPEG2 is supplied to the data processing unit 52, and among the information included in the RTP packet and the RTCP SR packet, information necessary for determining whether or not to generate a refresh command is determined. , To the refresh instruction generation unit 53. The data acquisition unit 81 of the refresh instruction generation unit 53 includes information necessary for processing of the packet loss rate calculation unit 82, the one-way delay calculation unit 84, the I-frame interval measurement unit 86, and the I-frame reception time prediction unit 89. Supply each.
[0088]
In step S12, the one-way delay calculator 84 determines whether or not the NTP of the SR packet is available. For example, when the NTP of the SR packet is clearly shifted or the SR packet cannot be received, it is determined that the NTP of the SR packet is not available.
[0089]
If it is determined in step S12 that the NTP of the SR packet is available, in step S13, the one-way delay calculation unit 84 obtains a one-way delay based on the RTP time stamp, and the one-way delay recording unit 85 To supply. The one-way delay recording unit 85 records the supplied one-way delay.
[0090]
If it is determined in step S12 that the NTP of the SR packet is not available, the one-way delay calculating unit 84 notifies the refresh instruction issuing unit 88 that the NTP of the SR packet is not available. In S14, the refresh command issuing unit 88 determines whether the refresh command has not been issued yet.
[0091]
If it is determined in step S14 that the refresh instruction has already been issued, the process proceeds to step S18 described below. If it is determined in step S14 that the refresh command has not been issued yet, it is necessary to issue the refresh command. In step S15, the refresh command issuing unit 88 issues the refresh command and issues It is supplied to the command transmitting unit 55. The refresh command transmitting unit 55 determines whether or not the transmission timing of the next RR packet is within a predetermined time, such as 200 ms, which is sufficiently shorter than the normal RR packet transmission interval.
[0092]
In step S15, when it is determined that the transmission timing of the next RR packet is within a predetermined time, in step S16, the refresh command transmitting unit 55 transmits the supplied refresh command to the network by the next RR packet. The data is transmitted to the data transmitting device 31 via the recording device 33, and the transmission time of the refresh command is recorded in the internal recording unit.
[0093]
If it is determined in step S15 that the transmission timing of the next RR packet is not within the predetermined time, in step S17, the refresh command transmitting unit 55 issues a refresh command using an APP packet whose transmission timing is not regulated in advance. The supplied refresh command is transmitted as an APP packet to the data transmitting device 31 via the network 33, and the transmission time of the refresh command is recorded in an internal recording unit.
[0094]
After the processing of step S13, step S16, or step S17 ends, in step S18, the I-frame interval measuring unit 86 determines whether or not the received data is an I-frame.
[0095]
If it is determined in step S18 that the received data is an I-frame, in step S19, the I-frame interval measuring unit 86 measures the I-frame interval based on the reception time of the immediately preceding I-frame. Then, it is supplied to the I frame interval recording section 87. The I-frame interval recording unit 87 records the supplied new I-frame interval measurement value.
[0096]
In step S20, the one-way delay calculation unit 84 determines whether or not the NTP of the SR packet is available in step S12. When it is determined in step S20 that the NTP of the SR packet is available, in step S13, since the one-way delay is obtained based on the RTP time stamp, the processing of step S21 is skipped, and the processing is performed. The process proceeds to step S22.
[0097]
If it is determined in step S20 that the NTP of the SR packet is not available, since the one-way delay has not yet been determined, in step S21, the one-way delay calculation unit 84 performs the I-frame interval measurement in step S19. A one-way delay is calculated based on the I-frame interval calculated by the unit 86 and recorded in the I-frame interval recording unit 87.
[0098]
That is, when it is determined that the NTP of the SR packet is not available, the data transmission device 31 transmits an I frame because the refresh command has been transmitted to the data transmission device 31. Therefore, if the I-frame interval is different from the normal value, it can be determined that the I-frame is an I-frame issued in response to the refresh command, so that an apparent RTT (Round Trip Time) should be obtained. Can be. Therefore, in step S21, the one-way delay can be obtained by dividing the apparent RTT by two.
[0099]
In step S22, the packet loss rate calculation unit 82 calculates the packet loss rate using the above-described formula (1) based on the supplied information, and supplies the packet loss rate to the packet loss rate recording unit 83. The packet loss rate recording unit 83 records the supplied packet loss rate.
[0100]
In step S23, the refresh command issuing unit 88 determines whether the packet loss rate recorded in the packet loss rate recording unit 83 is equal to or greater than a predetermined threshold. Here, the predetermined threshold value is predetermined based on a packet loss rate at which a decoding error due to a packet loss occurs in the data receiving device 32 and, in some cases, the error may be propagated. Is set. If it is determined in step S23 that the packet loss rate is not equal to or larger than the predetermined threshold, the process returns to step S11, and the subsequent processes are repeated.
[0101]
If it is determined in step S23 that the packet loss rate is equal to or greater than the predetermined threshold, in step S24, the refresh command issuing unit 88 changes the value of the I frame interval recorded in the I frame interval recording unit 87 to I It is supplied to the frame reception time prediction section 89, and controls the I frame reception time prediction section 89 to predict the reception time of the next I frame. The I-frame reception time predicting unit 89 determines that the refresh command has not been issued based on the supplied I-frame reception time supplied from the data acquisition unit 81 and the I-frame interval supplied from the refresh command issuing unit 88. In this case, the reception time of the next I frame to be received is predicted.
[0102]
In step S25, the refresh command issuing unit 88 records the one-way delay recorded in the one-way delay recording unit 85, the I-frame interval recorded in the I-frame interval recording unit 87, and the packet loss rate recording unit 83. The packet loss rate and the arrival time of the next I frame when no refresh command is issued, predicted by the I frame reception time prediction unit 89 in step S24, are read out.
[0103]
In step S26, the refresh command issuing unit 88 determines whether to transmit a refresh command based on the value read in step S25. The refresh command issuing unit 88 determines that the refresh command is to be transmitted when the packet loss rate is equal to or more than a predetermined value and the time until the predicted reception time of the next I frame is longer than the one-way delay.
[0104]
If it is determined in step S26 that the refresh command is not to be transmitted, the process returns to step S11, and the subsequent processes are repeated. When it is determined in step S26 that the refresh command is to be transmitted, in step S27, the refresh command issuing unit 88 issues a refresh command, and the refresh command transmitting unit 55 sets the transmission timing of the next RR packet to, for example, 200 ms. For example, it is determined whether or not the time is within a predetermined time that is sufficiently shorter than the transmission interval of the normal RR packet.
[0105]
If it is determined in step S27 that the transmission timing of the next RR packet is within a predetermined time, in step S28, the refresh command transmitting unit 55 sets the supplied refresh command as the next RR packet, and , The transmission time of the refresh command is recorded in the internal recording unit, the process returns to step S11, and the subsequent processes are repeated.
[0106]
If it is determined in step S27 that the transmission timing of the next RR packet is not within the predetermined time, in step S29, the refresh command transmitting unit 55 issues a refresh command using an APP packet whose transmission timing is not specified in advance. The supplied refresh command is transmitted as an APP packet to the data transmission device 31 via the network 33, and the transmission time of the refresh command is recorded in the internal recording unit. The process proceeds to step S11. Return, and the subsequent processing is repeated.
[0107]
Through such processing, a refresh command is generated and transmitted to the data transmitting device 31 using a suitable packet among the RR packet and the APP packet.
[0108]
Further, in the processing described with reference to FIGS. 7 and 8, when there is a time until the transmission timing of the RR packet, the description has been made assuming that the refresh command is transmitted using the APP packet. By controlling, the refresh command may be transmitted to the data transmitting apparatus 31 at a timing different from the conventional RR packet transmission interval.
[0109]
The refresh command issuing process 2 when controlling the transmission time of the RR packet will be described with reference to FIGS. 9 and 10.
[0110]
In steps S41 to S45, the same processes as those in steps S11 to S15 described with reference to FIG. 7 are executed.
[0111]
That is, the RTP packet and the RTCP SR packet transmitted from the data transmitting device 31 are received, and it is determined whether or not the NTP of the SR packet is available. If it is determined that the NTP of the SR packet is available, a one-way delay is determined based on the RTP timestamp, and if it is determined that the NTP of the SR packet is not available, the refresh command is issued by 1 It is determined whether or not it has been issued again. Then, if it is determined that the refresh instruction has already been issued, the process proceeds to step S48 described later. Therefore, it is determined whether or not the transmission timing of the next RR packet is within a predetermined time, such as 200 ms, which is sufficiently shorter than the normal RR packet transmission interval.
[0112]
In step S45, when it is determined that the transmission timing of the next RR packet is within the predetermined time, in step S46, the refresh command transmitting unit 55 converts the supplied refresh command into the next RR packet as the next RR packet. The data is transmitted to the data transmitting device 31 via the recording device 33, and the transmission time of the refresh command is recorded in the internal recording unit.
[0113]
If it is determined in step S45 that the transmission timing of the next RR packet is not within the predetermined time, in step S47, the refresh command transmitting unit 55 changes the RR packet issuance timing and refreshes with the RR packet. An instruction is issued, the supplied refresh instruction is transmitted as an RR packet to the data transmitting device 31 via the network 33, and the transmission time of the refresh instruction is recorded in an internal recording unit.
[0114]
After the processing of step S43, step S46, or step S47 is completed, in step S48 to step S57, processing similar to step S18 to step S27 described with reference to FIGS. 7 and 8 is performed.
[0115]
That is, it is determined whether or not the received data is an I frame. If the received data is determined to be an I frame, the I frame interval is measured based on the reception time of the immediately preceding I frame, It is determined whether the NTP of the SR packet was available. If it is determined that the NTP of the SR packet is not available, the one-way delay is not yet obtained, and thus the one-way delay is obtained from the apparent RTT based on the I-frame interval.
[0116]
Then, the packet loss rate is calculated, and it is determined whether the packet loss rate is equal to or greater than a predetermined threshold. If it is determined that the packet loss rate is not equal to or greater than the predetermined threshold, the process returns to step S41, and the subsequent processes are repeated. If it is determined that the packet loss rate is equal to or greater than the predetermined threshold, the reception time of the next I frame is predicted, and the one-way delay, the frame interval, the packet loss rate, and the arrival time of the next I frame are read. Then, it is determined whether or not to transmit the refresh command. If it is determined not to transmit the refresh command, the process returns to step S41, and the subsequent processes are repeated. When it is determined that the refresh command is to be transmitted, it is determined whether the transmission timing of the next RR packet is within a predetermined time, such as 200 ms, which is sufficiently shorter than the normal RR packet transmission interval.
[0117]
In step S57, when it is determined that the transmission timing of the next RR packet is within the predetermined time, in step S58, the refresh command transmitting unit 55 converts the supplied refresh command into the next RR packet as the next RR packet. 33, the transmission time of the refresh command is recorded in the internal recording unit, the process returns to step S41, and the subsequent processes are repeated.
[0118]
If it is determined in step S57 that the transmission timing of the next RR packet is not within the predetermined time, in step S59, the refresh command transmitting unit 55 changes the RR packet issuance timing and refreshes with the RR packet. An instruction is issued, the supplied refresh instruction is transmitted as an RR packet to the data transmitting device 31 via the network 33, and the transmission time of the refresh instruction is recorded in an internal recording unit. Returning to S41, the subsequent processing is repeated.
[0119]
By such processing, the data receiving device 32 transmits the refresh command to the data transmitting device 31 at a timing at which error propagation does not continue for a long time by controlling the transmission timing of the RR packet without using the APP packet. can do.
[0120]
Next, referring to the flowchart of FIG. 11, the refresh command issued in the refresh command issuing process 1 described with reference to FIGS. 7 and 8 or the refresh command issued in the refresh command issuing process 2 described with reference to FIGS. An I-frame insertion process 1 executed by the data transmitting device 31 for receiving an instruction will be described.
[0121]
In step S71, the data receiving unit 43 of the data transmitting device 31 determines whether an RR packet or an APP packet has been received. If it is determined in step S71 that the RR packet or the APP packet has not been received, the process of step S71 is repeated until it is determined that the RR packet or the APP packet has been received.
[0122]
When it is determined in step S71 that the RR packet or the APP packet has been received, in step S72, the data receiving unit 43 determines whether the received RR packet or the APP packet includes a refresh command. to decide. For example, when the process executed by the data receiving device 32 is the refresh command issuing process 1 described with reference to FIGS. 7 and 8, both the RR packet and the APP packet received by the data receiving unit 43 are refreshed. Although the command may be included, if the process executed by the data receiving device 32 is the refresh command issuing process 2 described with reference to FIGS. 9 and 10, the RR received by the data receiving unit 43 Only packets may contain a refresh instruction. If it is determined in step S72 that the refresh command is not included, the process returns to step S71, and the subsequent processes are repeated.
[0123]
If it is determined in step S72 that a refresh command is included, the data receiving unit 43 supplies the refresh command to the encoding control unit 44 in step S73. The encoding control unit 44 controls the encoding performed by the encoder 41, generates an I frame, and supplies the I frame to the data transmitting unit 42. Then, the data transmitting unit 42 transmits the I frame in the RTP packet to the data receiving device 32 via the network 33, and the process returns to Step S71, and the subsequent processes are repeated.
[0124]
Unless the control of inserting an I frame into the RTP packet to be transmitted is performed, frames generated at normal decoding timing (for example, in the case of MPEG2, the frames are generated in the order of IBBPBBPBBP...) RTP packet is transmitted to the data receiving device 32 via the network 33.
[0125]
By such processing, in the data transmission device 31, decoding is controlled based on the refresh command, and an I-frame is generated as necessary and transmitted as an RTP packet to the data reception device 32. For example, all the data to be transmitted are I-frames, or the frequency of occurrence of I-frames is increased to prevent error propagation due to packet loss or the like without imposing a load on the network, or to shorten the error propagation time. It is possible to do.
[0126]
Next, FIG. 12 is a block diagram showing a configuration of a data transmitting apparatus 101 and a data receiving apparatus 102 in the second embodiment of the data communication system to which the present invention is applied.
[0127]
Parts corresponding to those in FIG. 2 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.
[0128]
That is, the data transmitting apparatus 101 of FIG. 12 has the same configuration as the data transmitting apparatus 31 described with reference to FIG. 2 except that an encoding control unit 111 is provided instead of the encoding control unit 44 of FIG. Have The encoding control unit 111 determines whether insertion of an I frame is necessary based on the rate control command generated and transmitted in the data receiving device 102, and generates a control signal for controlling the encoding process by the encoder 41. .
[0129]
Also, the data receiving apparatus 102 of FIG. 12 substitutes for the refresh command generation unit 53 and the refresh command transmission unit 55 of FIG. 2 to predict congestion of the network 33 and, if necessary, generate a rate control command. 2, except that a rate control command transmission unit 122 for transmitting the rate control command generated by the congestion prediction unit 121 to the data transmission device 101 via the network 33 is provided. It has the same configuration as the data receiving device 32 described.
[0130]
Next, operations of the data transmitting apparatus 101 and the data receiving apparatus 102 will be described.
[0131]
The encoder 41 of the data transmitting apparatus 101 receives, for example, audio, image, video, text data, or a mixture of these data from a data input unit (not shown), and based on the control of the encode control unit 111, Is encoded and supplied to the data transmission unit 42. The amount of data generated at this time is controlled by the transmission rate control unit 45.
[0132]
The data transmission unit 42 divides the data supplied from the encoder 41 into packets in units of time, adds an SR packet every predetermined time, and transmits the packets to the data receiving device 102 via the network 33.
[0133]
The data receiving unit 51 of the data receiving device 102 receives the RTP packet and the RTCP SR packet from the data transmitting device 101 via the network 33. The data receiving unit 51 supplies, for example, data such as video, audio, and text to the data processing unit 52, and also receives the reception time, time stamp, sequence number, and time information included in the SR packet of the RTP packet. , To the congestion prediction unit 121.
[0134]
Further, when a packet loss such as discarding of a frame due to a bit error in an access line or discarding of a packet on a bottleneck link occurs in a received packet, the data receiving unit 51 transmits information on the packet loss to the congestion prediction. To the unit 121.
[0135]
The congestion prediction unit 121 obtains a data delay and a packet loss rate based on the information supplied from the data reception unit 51, and generates a rate control command as needed.
[0136]
FIG. 13 shows a functional block diagram of the congestion prediction unit 121.
[0137]
The data acquisition unit 131 obtains, from the data reception unit 51, data delay and a packet loss rate such as time information included in the SR packet, a reception time of the RTP packet, a time stamp, a packet size, and a sequence number. Is obtained and supplied to the data operation unit 132.
[0138]
The data operation unit 132 refers to the time information supplied from the timer 133 and, for a predetermined time such as 5 seconds or 10 seconds, for example, a one-way delay in data transmission from the data transmitting apparatus 101 to the data receiving apparatus 102, The sum of the number of losses, the number of received packets, and the number of bytes of received data is calculated to generate statistical information.
[0139]
If the times according to NTP of the data transmitting apparatus 101 and the data receiving apparatus 102 match, the absolute delay time is mapped to the NTP time accuracy by mapping the RTP time stamp included in the RTCP SR packet to NTP. Can be obtained by However, actually, there are many environments in which NTP cannot be used.
[0140]
Next, a method of measuring a delay when NTP cannot be used, that is, when the data transmitting apparatus 101 and the data receiving apparatus 102 do not match the time according to NTP will be described.
[0141]
First, the absolute delay for each packet can be obtained by the following equation (2).
[0142]

[0143]
Here, the brackets [] indicate which clock the time refers to (the same applies hereinafter).
[0144]
When the NTP times of the data transmitting apparatus 101 and the data receiving apparatus 102 are synchronized and the NTP information is correctly included in the RTCP SR packet, the following equation (3) holds.
[0145]

[0146]
Then, from the time stamp in the SR packet and the mapping information of the NTP, a mapping function f1 from the NTP to the RTP time stamp associating the transmission time and the time stamp of the RTP packet is obtained as shown in Expression (4). The delay of the RTP packet can be obtained by the equation (5), which is replaced with the RTP time stamp of 3).
[0147]
RTP time stamp = f1 [NTP] (4)

[0148]
On the other hand, for example, because the NTP of the SR packet is clearly shifted or the SR packet cannot be received, the mapping of the RTP time stamp to the NTP cannot be performed. Is not obtained, it is necessary to measure the delay with a clock inside the data receiving apparatus 102. In this case, the delay of the RTP packet is actually obtained by measuring the relative delay based on the arrival time of a certain RTP packet.
[0149]
For example, assume that the time stamp of the first RTP packet is TS1, and the reception time (in ms) of the first RTP packet is TR1. Both the time stamp of the first RTP packet and the reception time of the first RTP packet are times measured by a reference different from NTP. Here, the reception time is multiplied by the RTP reference clock in order to match the time unit of the time stamp of the first RTP packet and the reception time of the first RTP packet. In data communication by RTP Payload format for MPEG4 Elementary Stream (RFC3016), a clock of 90 KHz is used as a reference clock. Therefore, if the unit of the reception time is ms, the following equation (6) may be used to convert the time into the reference clock of the time stamp.
[0150]
Reception time [reference clock] = TR1 (ms) × 90 (6)
[0151]
Here, when the first RTP packet to the n-th RTP packet are RTP1 to RTPn, and the respective time stamps are TS1 to TSn, if there is no congestion or packet loss in the network, the following equation (7) ) Is almost a constant.
[0152]

[0153]
When TSn− (TRn × 90) represented by the equation (7) is measured by a reference clock of the RTP using a clock included in the data receiving apparatus 102, the data transmission apparatus 101 and the data receiving apparatus 102 perform the measurement. The time difference. However, the jitter and the clock shift between transmission and reception are ignored in equation (7).
[0154]
However, it is not known how much the time difference corresponds to the absolute time. That is, even if the value of the equation (7) is 0, the magnitude of the packet delay cannot be obtained. Therefore, the data operation unit 132 sets the time difference between the data transmitting apparatus 101 and the data receiving apparatus 102 including the packet delay at a certain time as the initial terminal-to-terminal time difference, and, based on the time difference, calculates the relative time by the equation (8). Measure the delay.
[0155]

[0156]
Here, TR is the reception time (unit: ms) of the RTP packet, and TS is the RTP time stamp.
[0157]
By the way, as described above, in equation (7), jitter and clock shift between transmission and reception are ignored, and it is assumed that the difference between the time stamps between transmission and reception obtained from equation (7) is constant. In practice, the sum of the time difference between the terminals and the packet delay shown in equation (7) decreases monotonically slowly due to the difference between the basic clocks of the data transmitting apparatus 101 and the data receiving apparatus 102, or It shows either tendency of monotonous increase. For example, as shown in FIG. 14, a situation occurs in which the delay gradually decreases with respect to the reception time.
[0158]
In such a case, if the initial time difference calculated using the equation (8) is kept used as the initial value, the obtained relative delay also monotonically decreases or monotonically increases. Therefore, the initial time difference calculated using equation (8) needs to be updated periodically.
[0159]
Further, the data calculation unit 132 further receives the supply of the number of packet losses, the number of received packets, and the amount of received data (the number of received bytes). The total sum of the relative delay of the RTP packet, the number of packet losses, the number of received packets, and the total amount of received data (the number of received bytes) in time are obtained.
[0160]
Next, the relative delay increase rate calculating unit 134 uses the following equation (9) based on the number of received packets within a predetermined time and the sum of the relative delay of the RTP packet supplied from the data calculating unit 132, The average relative delay of the RTP packet for a predetermined period (between time t1 and time t2) is calculated.
[0161]
(Equation 1)

[0162]
Further, the relative delay increase rate calculation unit 134 compares the average relative delay of the RTP packet calculated using Expression (9) with the average relative delay of the immediately preceding section, and calculates the average relative delay increase rate D. calculate.
[0163]
Then, the packet loss rate calculation unit 135 obtains the packet loss rate L from the information supplied from the data calculation unit 132.
[0164]
Generally, packet loss of an RTP packet occurs due to discarding of a frame due to a bit error in an access line or discarding of a packet in a bottleneck link. Bit errors in the access line can be prevented to some extent by error correction or retransmission in the link layer (particularly in the case of a wireless link). However, in an actual wireless link, a certain Packet loss occurs. In addition, when a packet is discarded in a bottleneck link, packet loss occurs due to overflow of the Queue of the bottleneck link. For example, in a situation in which a FIFO (First In First Out) type Queue is used. Therefore, packet loss occurs due to the overflow.
[0165]
That is, it is assumed that packet loss due to a bit error in the access line occurs with a certain probability. Therefore, lowering the rate on the transmission side does not necessarily lower the packet loss rate. However, when packets are discarded on the bottleneck link, packet loss will continue to occur unless the transmission side rate is reduced.
[0166]
Here, the packet loss rate calculation unit 135 calculates the packet loss rate L using Expression (1) in the same manner as the packet loss rate calculation unit 82 in FIG. 6 described above.
[0167]
The state transition control unit 136 controls the reception rate based on the average relative delay increase rate D of the RTP packets and the packet loss rate L supplied from the relative delay increase rate calculation unit 134 and the packet loss rate calculation unit 135. It is determined whether or not to make a state transition.
[0168]
The state transition control unit 136 uses Expression (10) to periodically compare the average relative delay increase rate with a predetermined threshold H1, and lowers the reception rate when the average relative delay increase rate is larger than the threshold H1. To the control state.
[0169]
Average relative delay increase rate D> threshold H1 (10)
[0170]
Further, the state transition control unit 136 uses the equation (11) to calculate the packet loss rate within a predetermined short time (for example, several hundred msec to 1 sec, which is sufficiently shorter than the case where the average relative delay increase rate is obtained). , And a predetermined threshold value H2, and if the packet loss rate is greater than the threshold value H2, the control state is shifted so as to reduce the reception rate.
[0171]
Packet loss rate L> threshold H2 (11)
[0172]
When the transmission band of the data transmission path between the data transmitting apparatus 101 and the data receiving apparatus 102 has a certain size, the length of the queue in the network becomes relatively short, and the state of the data transmission path becomes unstable. , The packet loss rate is likely to increase, and therefore, Expression (11) is easily satisfied in Expressions (10) and (11). On the other hand, when the transmission band of the data transmission path between the data transmission apparatus 101 and the data reception apparatus 102 is relatively small, the length of the queue in the network becomes relatively long, and the state of the data transmission path is not good. Since the average relative delay increase rate D tends to increase by becoming stable, the equation (10) is easily satisfied in the equations (10) and (11).
[0173]
As described above, by determining whether it is necessary to reduce the reception rate based on the two evaluation expressions of Expressions (10) and (11), even if the data transmission band is large, the evaluation value can be reduced. Even in this case, it is possible to control the data reception rate optimal for the state of the data transmission path.
[0174]
When the data transmission state is stable, the state transition control unit 136 sets the reception rate to be constant or increases the reception rate when the data transmission state is stable, based on the comparison results obtained by the equations (10) and (11). May transition.
[0175]
FIG. 15 shows a state transition diagram by the state transition control unit 136.
[0176]
The states include an UP state in which the data reception rate is controlled to be increased, a Down state in which the data reception rate is controlled to be decreased, and a Hold state in which the data reception rate is controlled not to be changed.
[0177]
At the beginning of the control, the state is a Hold state. The state transition control unit 136 calculates the average relative delay increase rate D ≦ the threshold value H1 and the packet loss rate L based on the comparison results obtained by Expressions (10) and (11) while the Hold state is maintained for the fixed time T. If ≤ threshold H2, the state transits to the UP state, and if average relative delay increase rate D> threshold H1 or packet loss rate L> threshold H2, the state transits to the Down state.
[0178]
In the UP state, the state transition control unit 136 changes the state when the average relative delay increase rate D ≦ the threshold value H1 and the packet loss rate L ≦ the threshold value H2 based on the comparison result by the equations (10) and (11). If the average relative delay increase rate D> threshold value H1 or the packet loss rate L> threshold value H2 holds in the UP state, the state changes to the Down state.
[0179]
In the Down state, the state transition control unit 136 determines, based on the comparison result obtained by the equations (10) and (11), that the average relative delay increase rate D ≦ the threshold H1 and the packet loss rate L ≦ the threshold H2, The state transits to the Hold state, and when the average relative delay increase rate D> the threshold value H1 or the packet loss rate L> the threshold value H2, the state is maintained as the Down state.
[0180]
The state transition control unit 136 outputs a state transition signal indicating the current state to the reception rate setting unit 137.
[0181]
The reception rate setting unit 137 sets a reception rate based on the state transition signal supplied from the state transition control unit, and generates a rate control command.
[0182]
When the state is the Down state, the reception rate setting unit 137 calculates the reset rate R by the following equation (12).
[0183]
Reset rate R = reception rate × C1 (12)
[0184]
Here, C1 in Expression (12) is a predetermined constant of 1 or less, or a value of 1 or less that is separately obtained by calculating the reciprocal of the average relative delay increase rate.
[0185]
Then, when the state is the UP state, the reception rate setting unit 137 calculates the reset rate R by the following equation (13).
[0186]
Reset rate R = reception rate × C2 (13)
[0187]
Here, C2 is one or more predetermined constants or one or more values calculated by calculating the reciprocal of the packet loss rate L.
[0188]
Regarding the upper limit of the reception rate, the link speed of the bottleneck link (data transmission speed in a network condition where packet loss and delay does not increase) is measured in advance using HTTP, packet pairs, or the like. May be set as the upper limit, or may not be set.
[0189]
As described above, the data reception rate is controlled based on the state shown in the state transition diagram of FIG. 15, so that the data reception rate is optimally controlled according to the condition of the data transmission path at that time. FIG. 16 is a diagram showing the relationship between the state shown in the state transition diagram of FIG. 15 and the data reception rate.
[0190]
The state of the data transmission path may fluctuate depending on the time. According to the data receiving rate control by the data receiving apparatus 102 to which the present invention is applied, the data receiving rate is not controlled to converge to a constant value, but the condition of the data transmission path is good, and the delay and packet loss are reduced. In a state in which no occurrence occurs, the state becomes an UP state, and the data transfer rate is controlled so as to increase.
[0191]
In this way, the congestion prediction unit 121 generates a rate control command as needed. The congestion prediction unit 121 outputs the generated rate control command to the rate control command transmission unit 122.
[0192]
The rate control command transmitting unit 122 transmits the rate control command to the data transmitting apparatus 101 as an RTCP APP packet.
[0193]
The data receiving unit 43 of the data transmitting apparatus 101 receives the rate control command, and supplies the received rate control command to the encoding control unit 111 and the transmission rate control unit 45.
[0194]
The encoding control unit 111 receives the supply of the rate control command from the data receiving unit 43. If the rate control command instructs to lower the rate, an error may occur on the data receiving side due to packet loss or the like. Therefore, the encoding of the encoder 41 is controlled to generate an I frame.
[0195]
The transmission rate control unit 45 generates a transmission rate control signal based on the rate control command supplied from the data reception unit 43, and supplies the transmission rate control signal to the data generation unit 41.
[0196]
The data generation unit 41 changes the data generation amount as needed based on the control of the transmission rate control unit 45. Therefore, the data rate of the data transmitted by the data transmission unit 42 is controlled. In addition, the data generation unit 41 generates an I frame as needed, regardless of the normal encoding order, based on the control of the encoding control unit 111. The data generated by the data generation unit 41 in this way is supplied to the data transmission unit 42, divided into packets, and transmitted.
[0197]
In this way, the data transmitting apparatus 101 can transmit packet data at an optimal transmission rate according to the state of the network, and can insert an I-frame into the data to be transmitted as necessary. Here, the data receiving apparatus 102 is described as transmitting the rate control command as an RTCP APP packet to the data transmitting apparatus 101. However, the data receiving apparatus 102 transmits the rate control command to the data transmitting apparatus 101 using the RTCP RR packet. It is needless to say that the information may be transmitted to the user.
[0198]
Next, the statistical information generation processing executed by the data receiving apparatus 102 will be described with reference to the flowchart in FIG.
[0199]
In step S111, the data receiving unit 51 determines whether or not a packet transmitted from the data transmitting device 101 via the network 33 has been received. If it is determined in step S111 that the packet has not been received, the process of step S111 is repeated until it is determined that the packet has been received.
[0200]
If it is determined in step S111 that the packet has been received, the data receiving unit 51 determines the reception time, time stamp, packet size, sequence number of the received RTP packet, NTP information of the received RTCP SR packet, and the like. Information necessary for obtaining the data delay and the packet loss rate is supplied to the congestion prediction unit 121. In step S112, the data calculation unit 132 of the congestion prediction unit 121 stores the reception time of the received RTP packet in an internal memory. Record.
[0201]
In step S113, the data calculation unit 132 of the congestion prediction unit 121 determines the one-way delay time, the number of packet losses, the number of received packets, and the number of received packets in the data transmission from the data transmitting apparatus 101 to the data receiving apparatus 102, as described above. The number of bytes is calculated, and in step S114, the one-way delay time, the number of packet losses, the number of received packets, and the total sum of the number of received bytes are updated. After the processing in step S114 ends, the processing returns to step S111, and the subsequent processing is repeated.
[0202]
Through such processing, statistical information is generated in the data calculation unit 132 of the congestion prediction unit 121.
[0203]
Next, a rate control process executed based on the statistical information generated in the statistical information generation process described with reference to FIG. 17 will be described with reference to the flowchart in FIG.
[0204]
In step S131, the state transition control unit 136 sets the control state to the Hold state.
[0205]
In step S132, the packet loss rate calculation unit 135 obtains the number of packet losses and the number of received packets from the statistical information from the data calculation unit 132, and calculates the packet loss rate L using Expression (1).
[0206]
In step S133, the relative delay increase rate calculation unit 134 acquires the total sum of the relative delay of the RTP packet within a predetermined time from the statistical information from the data calculation unit 132, and calculates the average of the RTP packet using Expression (9). A relative delay is calculated, and an average relative delay increase rate D is calculated by comparing the average relative delay of the RTP packet with the average relative delay of the immediately preceding section.
[0207]
In step S134, the state transition control unit 136 determines whether a fixed time has elapsed in the same state. If it is determined in step S134 that the predetermined time has not elapsed in the same state, the process returns to step S132, and the subsequent processes are repeated.
[0208]
If it is determined in step S134 that a predetermined time has elapsed in the same state, in step S135, the state transition control unit 136 increases the packet loss rate L calculated in step S132 and the average relative delay increase calculated in step S133. Based on the rate D, as described with reference to FIG. 15, the control state is changed as necessary, and a state transition signal is supplied to the reception rate setting unit 137.
[0209]
In step S136, the reception rate setting unit 137 determines whether to change the reception bit rate based on the state transition signal supplied from the state transition control unit 136. If the state transition signal indicates the Hold state and it is determined in step S136 that the reception bit rate is not changed, the process returns to step S132, and the subsequent processing is repeated.
[0210]
If the state transition signal indicates the UP state or the Down state, and it is determined in step S136 that the reception bit rate is to be changed, in step S137, the reception rate setting unit 137 determines whether the above-described equation (12) or (13) ) Is used to calculate a bit rate, generate a rate control command, and supply the rate control command to the rate control command transmitting unit 122. The rate control command transmitting unit 122 transmits the rate control command to the data transmitting device 101 via the network 33.
[0211]
After the processing in step S137 ends, the processing returns to step S132, and the subsequent processing is repeated.
[0212]
Through such processing, the data receiving apparatus 102 generates a rate control command according to the state of the data transmission path (network 33), and transmits it to the data transmitting apparatus 101.
[0213]
Next, an I-frame insertion process 2 executed by the data transmitting apparatus 101 that has received the rate control command will be described with reference to the flowchart in FIG.
[0214]
In step S151, the data receiving unit 43 of the data transmitting apparatus 101 determines whether an RR packet or an APP packet has been received from the data receiving apparatus 102 via the network 33. If it is determined in step S151 that no RR packet or APP packet has been received, the process of step S151 is repeated until it is determined that an RR packet or APP packet has been received.
[0215]
If it is determined in step S151 that an RR packet or an APP packet has been received, in step S152, the data receiving unit 43 determines whether the received RR packet or the APP packet includes a rate control command. Judge. If it is determined in step S152 that the received RR packet or APP packet does not include the rate control command, the process returns to step S151, and the subsequent processing is repeated.
[0216]
If it is determined in step S152 that the received RR packet or APP packet contains a rate control command, the data receiving unit 43 transmits the rate control command to the transmission rate control unit 45 and the encode control unit 111. In step S153, the transmission rate control unit 45 controls the encoder 41 based on the rate control command to reduce or increase the amount of generated data, thereby transmitting the data from the data transmission unit 42. The data transmission bit rate.
[0219]
In step S154, the encoding control unit 111 acquires the transmission time of the next I frame in the normal encoding order from the encoder 41. In step S155, the encoding control unit 111 determines whether the rate control command instructs to lower the rate. Then, it is determined whether or not to execute the I-frame insertion processing based on whether or not the time until the transmission time of the I-frame in the normal encoding order is longer than a predetermined time, such as one second.
[0218]
The encoding control unit 111 determines that the time until the transmission time of the I frame in the normal encoding order is longer than a predetermined time, such as 1 second, and that the rate control command instructs to lower the rate. , I frame insertion processing is executed. If it is determined in step S155 that the I-frame insertion process is not to be performed, the process returns to step S151, and the subsequent processes are repeated.
[0219]
If it is determined in step S155 that the I-frame insertion process is to be performed, in step S156, the encoding control unit 111 controls the encoder 41 to generate an I-frame regardless of the normal encoding order. , And insert an I frame into the RTP packet to be transmitted. After the process of step S156 ends, the process returns to step S151, and the subsequent processes are repeated.
[0220]
Through such processing, the data transmitting apparatus 101 that has received the rate control command controls the rate of data to be transmitted, and in a normal encoding order, the time until transmission of the next I frame is longer than the predetermined time. If the command is long and the rate control command instructs to lower the rate, an I-frame is generated and transmitted to the data receiving apparatus 102. It is possible to prevent a situation in which the propagation of the data continues for a long time.
[0221]
It goes without saying that the method of generating the rate control command by the data receiving apparatus 102 may be a method other than the method described above. The data transmission device 101 receives the data regardless of the type of the rate control command transmitted by the data reception device 102, controls the rate of data to be transmitted as needed, generates an I frame, Can be sent.
[0222]
In the first embodiment, in the data receiving device 32, a refresh command is generated, and the data transmitting device receiving the refresh command generates an I-frame as necessary regardless of the normal encoding order, Transmitted to the receiving device 32. Further, in the second embodiment, in the data receiving apparatus 102, a rate control command is generated, and in addition to controlling the data transmission rate, the data transmitting apparatus receiving the rate control command controls the data transmission rate. Regardless, an I frame was generated and transmitted to the data receiving apparatus 102 as needed.
[0223]
On the other hand, instead of generating a command for the data transmitting device in the data receiving device, the data transmitting device generates an instruction based on the RR packet generated and transmitted by the data receiving device. It may be determined whether or not it is necessary to insert a frame, and if necessary, an I frame may be generated and transmitted regardless of the normal encoding order.
[0224]
FIG. 20 is a block diagram illustrating a configuration of the data transmitting device 161 and the data receiving device 162 according to the third embodiment.
[0225]
Parts corresponding to those in FIG. 2 are denoted by the same reference numerals, and description thereof will be omitted as appropriate.
[0226]
That is, the data transmission device 161 in FIG. 20 determines whether to generate and transmit an I frame based on the information included in the RR packet regardless of the normal encoding order, instead of the encoding control unit 44. It has a configuration similar to that of the data transmission device 31 of FIG. 2 except that an encode control unit 171 for determining whether or not a control signal for generating an I frame is output to the encoder 41 is provided if necessary. The data receiving device 162 in FIG. 20 has the same configuration as the data receiving device 32 in FIG. 2 except that the refresh command generating unit 53 and the refresh command transmitting unit 55 are omitted.
[0227]
Next, the operation of the data transmitting device 161 and the data receiving device 162 will be described.
[0228]
The encoder 41 of the data transmitting device 161 receives input of, for example, audio, image, video, text data, or a mixture of these from a data input unit (not shown), and encodes the data by a method such as MPEG2 or MPEG4. Then, the data is supplied to the data transmission unit 42. The encoder 41 determines the amount of data to generate based on the control of the transmission rate control unit 45. By controlling the amount of data generated, the transmission rate is controlled.
[0229]
The data transmission unit 42 divides the data supplied from the encoder 41 into packets in units of time, adds SR packets at predetermined time intervals, and transmits the packets to the data receiving device 162 via the network 33.
[0230]
The data receiving unit 51 of the data receiving device 162 receives the RTP packet and the RTCP SR packet from the data transmitting device 31 via the network 33. The data receiving unit 51 supplies data such as video, audio, and text to the data processing unit 52, for example.
[0231]
The data transmitting unit 54 transmits the RTCP RR packet to the data transmitting device 161 via the network 33.
[0232]
The data receiving unit 43 of the data transmitting device 161 receives the RR packet, and supplies information such as time stamp information, packet loss, and jitter included in the received RR packet to the encoding control unit 171.
[0233]
The encoding control unit 171 compares a value such as a packet loss rate or a jitter with a predetermined threshold based on the information supplied from the data receiving unit 43, and based on the comparison result, regardless of the normal frame order. A control for determining whether or not the next frame to be generated by encoding is an I-frame, and, if necessary, controlling the encoding so that the next frame to be generated by encoding is an I-frame. A signal is generated and supplied to the encoder 41. Further, the encoding control unit 171 supplies the transmission rate control unit 45 with a result of comparison between the values such as the packet loss rate and the jitter and a predetermined threshold.
[0234]
The transmission rate control unit 45 determines whether or not the transmission rate needs to be changed based on the information supplied from the encoding control unit 171 and controls the data generation amount when the transmission rate needs to be changed. A control signal is generated and supplied to the encoder 41.
[0235]
The encoder 41 encodes the supplied data while inserting an I frame as necessary based on the control signal supplied from the encoding control unit 44. In other cases, the encoder 41 performs predetermined encoding. Encoding is performed based on the encoding method, and the encoded data is supplied to the data transmitting unit 42.
[0236]
The data transmitting unit 42 divides the supplied encoded data for each time to generate an RTP packet, and transmits the RTP packet to the data receiving device 162 via the network 33. Further, the data transmitting unit 42 generates the above-described SR packet at predetermined time intervals, such as 5 seconds, for example, and transmits the generated SR packet to the data receiving device 162 via the network 33.
[0237]
In this manner, the data transmitting device 161 is generated by the normal encoding process so that an error due to packet loss is less likely to occur in the data receiving device 162 or even if an error occurs, the error propagation time is shortened. Regardless of the order of the frames, an I frame can be appropriately generated and transmitted to the data receiving device 32.
[0238]
Next, an I-frame insertion process 3 executed by the data transmitting device 161 based on information included in the RR packet will be described with reference to a flowchart of FIG.
[0239]
In step S181, the data receiving unit 43 of the data transmitting device 161 determines whether an RR packet transmitted from the data receiving device 162 has been received. If it is determined in step S181 that the RR packet has not been received, the process of step S181 is repeated until it is determined that the RR packet has been received.
[0240]
If it is determined in step S181 that the RR packet has been received, the data receiving unit 43 supplies the information described in the RR packet to the encode control unit 171. In step S182, the encode control unit 171 Based on the supplied data, for example, the packet loss rate is calculated using the above-described equation (1), and it is determined whether the packet loss rate is equal to or greater than a predetermined threshold. If it is determined in step S182 that the packet loss rate is equal to or greater than the predetermined threshold, the process proceeds to step S184.
[0241]
When it is determined in step S182 that the packet loss rate is equal to or smaller than the predetermined threshold, in step S183, the encoding control unit 171 determines whether the jitter is equal to or larger than the predetermined threshold. If it is determined in step S183 that the jitter is equal to or smaller than the predetermined threshold, the process returns to step S181, and the subsequent processes are repeated.
[0242]
If it is determined in step S182 that the packet loss rate is equal to or greater than the predetermined threshold, or if it is determined in step S183 that the jitter is equal to or greater than the predetermined threshold, in step S184, the encoding control unit 171 determines The transmission rate control unit 45 is notified that the packet loss rate is equal to or greater than a predetermined threshold or that the jitter is equal to or greater than a predetermined threshold. The transmission rate control unit 45 controls the processing by the encoder 41 to reduce the amount of generated data, thereby lowering the transmission bit rate of the data transmitted from the data transmission unit 42.
[0243]
Here, the encoding control unit 171 has been described as notifying the transmission rate control unit 45 that the packet loss rate is equal to or greater than a predetermined threshold or that the jitter is equal to or greater than a predetermined threshold. It goes without saying that the rate control unit 45 may execute the same processing as in steps S182 and S183, so that the transmission rate can be reduced as necessary as a result.
[0244]
In step S185, the encoding control unit 171 acquires the transmission time of the next I frame transmitted from the data transmission unit 42.
[0245]
In step S186, the encoding control unit 171 determines whether or not to execute the I frame insertion process based on the packet loss rate, the jitter, and the time until the transmission time of the next I frame. The encoding control unit 171 determines that one of the packet loss rate and the jitter is equal to or more than a predetermined threshold value, and that the time until the transmission time of the next I frame is longer than a predetermined time such as one second. If it is longer, an I-frame insertion process is executed. If it is determined in step S186 that the I-frame insertion process is not to be performed, the process returns to step S181, and the subsequent processes are repeated.
[0246]
In addition, the encoding control unit 171 determines that the packet loss rate and the jitter are both equal to or greater than a predetermined threshold, and that the time until the transmission time of the next I frame is a predetermined time such as 1 second. If it is longer, the I-frame insertion process may be executed.
[0247]
When it is determined in step S186 that the I-frame insertion process is to be performed, in step S187, the encoding control unit 171 controls the encoder 41 to execute the intra-frame encoding and transmit the data to be transmitted from the data transmitting unit 42. Then, the process returns to step S181, and the subsequent processes are repeated.
[0248]
By such a process, the data transmitting device 161 is generated by the normal encoding process so that an error due to packet loss is unlikely to occur in the data receiving device 162 or even if an error occurs, the error propagation time is shortened. Regardless of the order of the frames, the I frame can be generated as appropriate and transmitted to the data receiving device 162.
[0249]
Here, based on the information described in the RTCP RR packet transmitted from data receiving apparatus 162 to data transmitting apparatus 161, data transmitting apparatus 161 determines whether or not to insert an I frame into the data to be transmitted. Although described as being determined, the data receiving apparatus 162 determines whether or not the data transmitting apparatus 161 inserts an I frame into data other than the RTCP RR packet (for example, an RTCP APP packet). May be described and transmitted.
[0250]
In the above, the case of the data transmitting device 31, the data transmitting device 101, or the data transmitting device 161 transmitting packet data and the data receiving device 32, the data receiving device 102, or the data receiving device 162 receiving packet data is described. However, the present invention is also applicable to the case where packet data is transmitted / received between data transmitting / receiving apparatuses capable of transmitting / receiving packet data.
[0251]
The series of processes described above can also be executed by software. The software is a computer in which a program constituting the software is built in dedicated hardware, or a general-purpose personal computer that can execute various functions by installing various programs. For example, it is installed from a recording medium.
[0252]
As shown in FIG. 22, this recording medium is a magnetic disk 221 (including a flexible disk) on which the program is recorded and an optical disk 222 (CD-ROM), which are distributed to provide the user with the program, separately from the computer. A package medium including a ROM (Compact Disk-Read Only Memory), a DVD (including a Digital Versatile Disk), a magneto-optical disk 223 (including an MD (Mini-Disk) (trademark)), a semiconductor memory 224, or the like. Is done.
[0253]
FIG. 22 illustrates a configuration example of a personal computer 201 that executes the above processing. The CPU (Central Processing Unit) 211 of the personal computer 201 stores a program stored in a ROM (Read Only Memory) 212 or a program loaded into a RAM (Random Access Memory) 213 from a HDD (Hard Disc Drive) 218. Therefore, various processes are executed. The RAM 213 also appropriately stores data necessary for the CPU 211 to execute various processes.
[0254]
The CPU 211, the ROM 212, and the RAM 213 are mutually connected via an internal bus 214. The input / output interface 215 is also connected to the internal bus 214.
[0255]
The input / output interface 215 includes an input unit 216 including a keyboard and a mouse, a display including a CRT (Cathode Ray Tube), an LCD (Liquid Crystal Display), an output unit 217 including a speaker, and a hard disk. A network interface 220 including an HDD 218, a modem, a terminal adapter, and the like is connected. The network interface 220 performs communication processing via a network such as the Internet, for example.
[0256]
A drive 219 is connected to the input / output interface 215 as necessary, and a magnetic disk 221, an optical disk 222, a magneto-optical disk 223, a semiconductor memory 224, or the like is appropriately mounted, and a computer program read therefrom is loaded. Are installed in the HDD 218 as needed.
[0257]
In this specification, the steps of describing a program recorded on a recording medium may be performed in chronological order in the order described, or may not be performed in chronological order. This also includes processes executed individually.
[0258]
In the present specification, the system represents the entire device including a plurality of devices.
[0259]
【The invention's effect】
As described above, according to the present invention, an encoded frame can be transmitted. In particular, an intra frame generated by intra-frame encoding can be transmitted according to a data communication situation.
[0260]
According to another aspect of the present invention, in addition to receiving data, based on a data communication state, information for instructing transmission of an intra frame generated by intra-frame coding is generated, and the data is transmitted to a data transmission source. Can be sent.
[Brief description of the drawings]
FIG. 1 is a diagram for describing an I-frame interval.
FIG. 2 is a block diagram showing a configuration of a data transmitting device and a data receiving device of a first data communication system to which the present invention has been applied.
FIG. 3 is a diagram illustrating an SR packet.
FIG. 4 is a diagram illustrating an RR packet.
FIG. 5 is a diagram illustrating an APP packet.
FIG. 6 is a block diagram illustrating a configuration of a refresh instruction generation unit in FIG. 2;
FIG. 7 is a flowchart illustrating a refresh instruction issuing process 1;
FIG. 8 is a flowchart illustrating a refresh instruction issuing process 1;
FIG. 9 is a flowchart illustrating a refresh instruction issuing process 2;
FIG. 10 is a flowchart illustrating a refresh instruction issuing process 2;
FIG. 11 is a flowchart illustrating I-frame insertion processing 1;
FIG. 12 is a block diagram showing a configuration of a data transmitting device and a data receiving device of a second data communication system to which the present invention has been applied.
13 is a block diagram illustrating a configuration of a congestion prediction unit in FIG.
FIG. 14 is a diagram illustrating a simple decrease in delay due to a reception time.
FIG. 15 is a diagram illustrating a state transition.
FIG. 16 is a diagram illustrating a state transition and a data transmission rate.
FIG. 17 is a flowchart illustrating a statistical information generation process.
FIG. 18 is a flowchart illustrating a rate control process.
FIG. 19 is a flowchart illustrating I-frame insertion processing 2;
FIG. 20 is a block diagram showing a configuration of a data transmitting device and a data receiving device of a third data communication system to which the present invention has been applied.
FIG. 21 is a flowchart illustrating I-frame insertion processing 3;
FIG. 22 is a block diagram illustrating a configuration of a personal computer.
[Explanation of symbols]
31 data transmitter, 32 data receiver, 33 network, 41 encoder, 42 data transmitter, 43 data receiver, 44 encode controller, 45 transmission rate controller, 51 data receiver, 52 data processor, 53 refresh command Generation unit, 54 data transmission unit, 55 refresh command transmission unit, 81 data acquisition unit, 82 packet loss rate calculation unit, 84 one-way delay calculation unit, 86 I frame interval measurement unit, 89 refresh command issue unit, 89 I frame reception Time prediction unit, 101 data transmission device, 102 data reception device, 111 encoding control unit, 121 congestion prediction unit, 122 rate control command transmission unit, 132 data calculation unit, 134 relative delay increase rate calculation unit, 135 packet loss rate calculation unit , 136 state transition control unit, 13 Reception rate setting unit, 161 data transmission apparatus, 162 a data receiving apparatus, 171 the encoding control unit

Claims (20)

In an information processing apparatus that transmits data to another information processing apparatus via a network,
Encoding means for encoding the supplied data using inter-frame correlation,
From the other information processing device, receiving means for receiving predetermined information regarding the data communication status in the network,
A control unit that controls the encoding unit based on the predetermined information received by the reception unit to execute intra-frame encoding;
An information processing apparatus comprising: a transmission unit configured to transmit the data encoded by the encoding unit using the inter-frame correlation to the another information processing device via the network. The predetermined information is information that instructs transmission of an intra frame generated by the intra-frame encoding,
2. The information processing apparatus according to claim 1, wherein the control unit controls the encoding unit to execute intra-frame encoding when the predetermined information is received by the receiving unit. . The predetermined information is control information for controlling a transmission rate of the data by the transmission unit,
The control unit controls the encoding unit when the predetermined information is received by the reception unit and the predetermined information instructs to reduce a transmission rate of the data. 2. The information processing apparatus according to claim 1, wherein encoding is performed. The data is an RTP packet conforming to RFC1889,
The information processing apparatus according to claim 1, wherein the predetermined information is an RTCP RR packet conforming to RFC1889. The control unit calculates a packet loss rate based on the predetermined information received by the reception unit, and determines whether to cause the encoding unit to execute intra-frame encoding based on the result. The information processing apparatus according to claim 4, wherein the determination is performed. The control unit calculates a variation in the delay of the data based on the predetermined information received by the receiving unit, and based on the result, causes the encoding unit to execute intra-frame encoding. The information processing apparatus according to claim 4, wherein the determination is made. The data is an RTP packet conforming to RFC1889,
The information processing apparatus according to claim 1, wherein the predetermined information is an RTCP APP packet conforming to RFC1889. An information processing device that includes an encoding unit that encodes the supplied data using inter-frame correlation, and that transmits the data encoded by the encoding unit to another information processing device via a network. In the information processing method,
From the other information processing apparatus, a reception control step of controlling reception of predetermined information on a data communication state in the network,
An encoding control step of controlling the encoding means to execute intra-frame encoding based on the predetermined information whose reception is controlled by the processing of the reception control step. Method. Using an encoding unit that encodes the supplied data using inter-frame correlation, a process of transmitting the data encoded by the encoding unit to another information processing apparatus via a network is performed by a computer. A program to be executed by
From the other information processing apparatus, a reception control step of controlling reception of predetermined information on a data communication state in the network,
An encoding control step of controlling the encoding means to execute intra-frame encoding based on the predetermined information whose reception is controlled by the processing of the reception control step. A recording medium on which a readable program is recorded. Using an encoding unit that encodes the supplied data using inter-frame correlation, a process of transmitting the data encoded by the encoding unit to another information processing apparatus via a network is performed by a computer. A program to be executed by
From the other information processing apparatus, a reception control step of controlling reception of predetermined information on a data communication state in the network,
An encoding control step of controlling the encoding means to execute intra-frame encoding based on the predetermined information whose reception is controlled by the processing of the reception control step. From another information processing device, via a network, in an information processing device that receives data encoded using inter-frame correlation,
A receiving unit that receives predetermined information including a transmission status of the data, transmitted from the other information processing apparatus,
Based on the predetermined information received by the receiving unit, based on the predetermined information, the other information processing apparatus, generating means for generating information for instructing transmission of an intra frame generated by intra-frame encoding,
An information processing apparatus, comprising: transmission means for transmitting information for instructing transmission of the intra frame generated by the generation means to the another information processing apparatus via the network. The data is an RTP packet conforming to RFC1889,
The information instructing the transmission of the intra frame is RTCP conforming to RFC1889.
The information processing apparatus according to claim 11, wherein the information processing apparatus is an RR packet. The data is an RTP packet conforming to RFC1889,
The information processing apparatus according to claim 11, wherein the information instructing the transmission of the intra frame is an RTCP APP packet conforming to RFC1889. The data is an RTP packet conforming to RFC1889,
When the time until the next transmission time of the RTCP RR packet conforming to RFC1889 is longer than a predetermined time, the transmitting unit transmits information for instructing the transmission of the intra frame by an RTCP APP packet conforming to RFC1889. 12. The information according to claim 11, wherein when the time until the next transmission time of the RTCP RR packet is shorter than a predetermined time, information for instructing transmission of the intra frame is transmitted by the RTCP RR packet. Information processing device. The data is an RTP packet conforming to RFC1889,
The transmitting means can change the transmission time of the RTCP RR packet conforming to RFC1889. The information processing apparatus according to claim 11, wherein information for instructing transmission of the intra frame is transmitted at an earlier transmission time. From another information processing apparatus, via a network, in an information processing method of an information processing apparatus that receives data encoded using inter-frame correlation,
A reception control step of controlling reception of predetermined information including a transmission state of the data transmitted from the other information processing apparatus,
A generation step of generating information for instructing the other information processing apparatus to transmit an intra frame generated by intra-frame encoding, based on the predetermined information whose reception is controlled by the processing of the reception control step; An information processing method comprising: A program that executes processing for controlling reception of data encoded using inter-frame correlation, transmitted from another information processing apparatus via a network,
A reception control step of controlling reception of predetermined information including a transmission state of the data transmitted from the other information processing apparatus,
A generation step of generating information for instructing the other information processing apparatus to transmit an intra frame generated by intra-frame encoding, based on the predetermined information whose reception is controlled by the processing of the reception control step; A recording medium on which a computer-readable program is recorded. A program that executes processing for controlling reception of data encoded using inter-frame correlation, transmitted from another information processing apparatus via a network,
A reception control step of controlling reception of predetermined information including a transmission state of the data transmitted from the other information processing apparatus,
A generation step of generating information for instructing the other information processing apparatus to transmit an intra frame generated by intra-frame encoding, based on the predetermined information whose reception is controlled by the processing of the reception control step; A program characterized by including: A first information processing device that receives data from another information processing device via a network,
In a data communication system configured with a second information processing device that transmits data to another information processing device via the network,
The first information processing device includes:
A first information including a transmission status of the data transmitted from the second information processing apparatus;
First receiving means for receiving the information of
On the basis of the first information received by the first receiving means, second information for instructing the second information processing apparatus to transmit an intra frame generated by intra-frame encoding is generated. Generating means;
First transmission means for transmitting the second information generated by the generation means to the second information processing device via the network,
The second information processing device includes:
Encoding means for encoding the supplied data using inter-frame correlation,
Second receiving means for receiving the second information from the first information processing device;
Control means for controlling the encoding means based on the second information received by the second receiving means to execute intra-frame encoding;
The data encoded by the encoding unit using the inter-frame correlation and the first information including the transmission status of the data are transmitted to the first information processing device via the network. A data communication system comprising: a second transmission unit. A first information processing device that receives data from another information processing device via a network,
In a data communication system configured with a second information processing device that transmits data to another information processing device via the network,
The first information processing device includes:
First receiving means for receiving the data transmitted from the second information processing device;
A first transmission unit configured to transmit information on a data communication state in the network to the second information processing apparatus;
The second information processing device includes:
Encoding means for encoding the supplied data using inter-frame correlation,
Second receiving means for receiving the information on the data communication status from the first information processing device;
Control means for controlling the encoding means to execute intra-frame encoding based on the information on the data communication status received by the second receiving means; and using inter-frame correlation by the encoding means. And a second transmitting unit for transmitting the data encoded by the first information processing apparatus to the first information processing apparatus via the network.

Images