JP2004193992A - Information processing system, information processor, information processing method, recording medium and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an information processing system in which an error can be corrected and an idle capacity of a band of a network can be predicted by using an idle region of the band of the network which efficiently transmits/receives information through the network and to provide an information processor, an information processing method, a recording medium and a program. <P>SOLUTION: A picture transmission device 11 packetizes a plurality of layers of hierarchy encoding data and transmits a packet to a picture reception device 12 in a layer unit. The picture reception device 12 transmits a confirmation response including a transmission time of the packet and ID of the lost packet to the picture transmission device 11. The picture transmission device 11 calculates an optimum transmission rate in accordance with the confirmation response from the picture reception device 12, sets the number of transmission packets based on the transmission rate and sets the layer to be transmitted so that the number of the transmission packets becomes not larger than that which is set. The packets which are wastefully transmitted can be eliminated and communication is made efficient. The invention can be applied to an electronic apparatus connected to the network. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

【００７７】
式（１）において、Ｔは理想的な送信レート、ｓはパケットサイズ、ｐはパケットロス率、ｔはTCP（Transmission Control Protocol）のタイムアウト時間（通常は、RTTの４倍）である。送信レート算出部１２１は、式（１）により、送信レートＴを算出した後、算出した送信レートＴを、パケット数算出部１２２に通知する。その後、処理はステップＳ１０５に進む。
【００７８】
ステップＳ１０５において、パケット数算出部１２２は、送信パケット数を以下の式（この式を式（２）とする）により、算出する。
【００７９】
ｎ＝（ｆ×ｓ）／Ｔ
【００８０】
式（２）において、ｎは送信パケット数、ｆはフレームレート、ｓはパケットサイズ、ＴはステップＳ１０４で算出された送信レートである。パケット数算出部１２２は、送信パケット数ｎを算出した後、算出した送信パケット数ｎを、階層設定部１２３に通知する。その後、処理はステップＳ１０６に進む。
【００８１】
ステップＳ１０６において、階層設定部１２３は、変数ＬをＬ＝０に初期化する。その後、処理はステップＳ１０７に進む。ステップＳ１０７において、階層設定部１２３は、以下の式（この式を式（３）とする）が成立するか否かを判定する。
【００８２】
（Ｌ＋１）×５≦ｎ
【００８３】
式（３）において、「５」は、１つのレイヤに含まれるパケットの個数を表している。すなわち、図２においては、１つのレイヤは、５個のパケットに分割されているので、式（３）において「５」とされている。そして、式（３）が成立する場合、処理はステップＳ１０８に進む。
【００８４】
ステップＳ１０８において、階層設定部１２３は、変数Ｌが４以上であるか否かを判定し、４以上ではない（４未満である）と判定した場合、処理はステップＳ１０９に進む。なお、ステップＳ１０８における判定の基準「４」は、レイヤの個数を表している。すなわち、図２において、階層符号化データは、レイヤＬ１乃至Ｌ４の４個のレイヤにより構成されていたため、ステップＳ１０８において、判定の基準が「４」とされる。
【００８５】
ステップＳ１０９において、階層設定部１２３は、変数Ｌの値を１だけインクリメントする。その後、処理はステップＳ１０７に戻り、ステップＳ１０７以降の処理が繰り返される。
【００８６】
ステップＳ１０７において、階層設定部１２３が、式（３）が成立しないと判定した場合、ステップＳ１０８の処理はスキップされ、処理はステップＳ１１０に進む。また、ステップＳ１０８において、階層設定部１２３が、変数Ｌは４以上であると判定した場合、処理はステップＳ１１０に進む。
【００８７】
例えば、ステップＳ１０５で、ｎ＝１７と算出された場合、ステップＳ１０６乃至ステップＳ１０９の処理は、以下のようになる。
【００８８】
まず、ステップＳ１０６においてＬ＝０に初期化され、ステップＳ１０７において、式（３）に、変数Ｌ＝０、およびｎ＝１７が代入され、式（３）が成立するか否かが判定される。変数Ｌ＝０の場合、式（３）は成立するので、処理はステップＳ１０８に進む。ステップＳ１０８において、変数Ｌが４以上であるか否かが判定されるが、変数Ｌ＝０なので、変数Ｌは４以上ではないと判定され、処理はステップＳ１０９に進む。ステップＳ１０９において、変数Ｌが１だけインクリメントされ、Ｌ＝１に設定される。
【００８９】
その後、処理はステップＳ１０７に戻り、式（３）に、変数Ｌ＝１、およびｎ＝１７が代入され、式（３）が成立するか否かが判定される。変数Ｌ＝１の場合、式（３）は成立するので、処理はステップＳ１０８に進む。ステップＳ１０８において、変数Ｌが４以上であるか否かが判定されるが、変数Ｌ＝１なので、変数Ｌは４以上ではないと判定され、処理はステップＳ１０９に進む。ステップＳ１０９において、変数Ｌが１だけインクリメントされ、変数Ｌ＝２に設定される。
【００９０】
その後、処理はステップＳ１０７に戻り、式（３）に、変数Ｌ＝２、およびｎ＝１７が代入され、式（３）が成立するか否かが判定される。変数Ｌ＝２の場合、式（３）は成立するので、処理はステップＳ１０８に進む。ステップＳ１０８において、変数Ｌが４以上であるか否かが判定されるが、変数Ｌ＝２なので、変数Ｌは４以上ではないと判定され、処理はステップＳ１０９に進む。ステップＳ１０９において、変数Ｌが１だけインクリメントされ、変数Ｌ＝３に設定される。
【００９１】
その後、処理はステップＳ１０７に戻り、式（３）に、変数Ｌ＝３、およびｎ＝１７が代入され、式（３）が成立するか否かが判定される。変数Ｌ＝３の場合、式（３）は成立しないので、処理はステップＳ１１０に進む。
【００９２】
ステップＳ１１０において、階層設定部１２３は、Ｌ＋１層より上位のレイヤのデータパケットを廃棄する。ステップＳ１１１において、階層設定部１２３は、下位のＬ層までのパケットを、画像受信装置１２に送信するデータパケットとして設定する。
【００９３】
例えば、ステップＳ１０６乃至ステップＳ１０９の処理により、変数ＬがＬ＝１に設定されていた場合、階層設定部１２３は、ステップＳ１１０において、図２に示される４つのレイヤのうち、レイヤＬ２乃至Ｌ４のパケット、すなわちパケットＰ６乃至Ｐ２０のパケットを廃棄し、ステップＳ１１１において、図２に示されるレイヤＬ１のパケット、すなわちパケットＰ１乃至Ｐ５を、画像受信装置１２に送信するデータパケットとして設定する。
【００９４】
また、例えば、ステップＳ１０６乃至ステップＳ１０９の処理により、変数ＬがＬ＝２に設定されていた場合、階層設定部１２３は、ステップＳ１１０において、図２に示される４つのレイヤのうち、レイヤＬ３およびＬ４のパケット、すなわちパケットＰ１１乃至Ｐ２０のパケットを廃棄し、ステップＳ１１１において、図２に示されるレイヤＬ１およびＬ２のパケット、すなわちパケットＰ１乃至Ｐ１０を、画像受信装置１２に送信するデータパケットとして設定する。
【００９５】
また、例えば、ステップＳ１０６乃至ステップＳ１０９の処理により、変数ＬがＬ＝３に設定されていた場合、階層設定部１２３は、ステップＳ１１０において、図２に示される４つのレイヤのうち、レイヤＬ４のパケット、すなわちパケットＰ１６乃至Ｐ２０のパケットを廃棄し、ステップＳ１１１において、図２に示されるレイヤＬ１乃至Ｌ３のパケット、すなわちパケットＰ１乃至Ｐ１５を、画像受信装置１２に送信するデータパケットとして設定する。
【００９６】
また、例えば、ステップＳ１０６乃至ステップＳ１０９の処理により、変数ＬがＬ＝４に設定されていた場合、階層設定部１２３は、ステップＳ１１０において、パケットを廃棄せず、ステップＳ１１１において、図２に示されるレイヤＬ１乃至Ｌ４のパケット、すなわちパケットＰ１乃至Ｐ２０を、画像受信装置１２に送信するデータパケットとして設定する。
【００９７】
ステップＳ１１１の処理の後、処理はステップＳ１０１に戻り、ステップＳ１０１以降の処理が繰り返される。
【００９８】
このようにして、画像送信装置１１から画像受信装置１２に送信する階層符号化データの階層が決定される。
【００９９】
階層設定部１２３は、以上の処理による設定に基づいて、パケット生成部１０５により生成されたデータパケットのうち、画像受信装置１２に送信するデータパケットのみをパケット生成部１０５に選択させ、それ以外のデータパケットを廃棄させる。その後、選択されたデータパケットのみが、パケット生成部１０５からパケット送信部１０６に供給される。
【０１００】
次に、図９のフローチャートを参照して、図６のステップＳ２の処理、すなわち、データパケットを、パケット送信部１０６から画像受信装置１２に送信する送信処理について説明する。
【０１０１】
パケット送信部１０６は、パケット生成部１０５から、送信すべきパケットとして設定されたデータパケットの供給を受けて、ステップＳ１５１において、送信するデータパケットの階層ｉをｉ＝１に設定する。その後、処理はステップＳ１５２に進む。
【０１０２】
ステップＳ１５２において、パケット送信部１０６は、階層ｉのデータパケットのパケットヘッダに、送信時刻を記録して、データパケットを画像受信装置１２に送信する。その後、処理はステップＳ１５３に進む。
【０１０３】
ステップＳ１５３において、パケット送信部１０６は、階層ｉを１だけインクリメントする。その後、処理はステップＳ１５４に進む。
【０１０４】
ステップＳ１５４において、パケット送信部１０６は、階層ｉが、図８のフローチャートのステップＳ１０６乃至ステップＳ１０９の処理により設定された変数Ｌより大きいか否かを判定し、階層ｉが変数Ｌより大きくない場合（階層ｉが変数Ｌ以下の場合）、処理はステップＳ１５２に戻り、ステップＳ１５２以降の処理が繰り返される。そして、ステップＳ１５４において、階層ｉが変数Ｌより大きいと判定された場合、パケット送信部１０６は、データパケットの送信を終了する。
【０１０５】
例えば、変数ＬがＬ＝３であった場合、まず、ステップＳ１５１でｉ＝１に設定され、ステップＳ１５２において、ｉ＝１に対応するレイヤ、すなわち図２においてレイヤＬ１のパケットＰ１乃至Ｐ５が、画像受信装置１２に送信される。その後、ステップＳ１５３において、ｉが１だけインクリメントされ、ｉ＝２に設定される。そして、ステップＳ１５４において、ｉがＬより大きいか否かが判定されるが、ｉ＝２であり、ｉはＬより大きくないので、処理はステップＳ１５２に戻る。
【０１０６】
その後、ステップＳ１５２において、ｉ＝２に対応するレイヤ、すなわち図２においてレイヤＬ２のパケットＰ６乃至Ｐ１０が、画像受信装置１２に送信される。その後、ステップＳ１５３において、ｉが１だけインクリメントされ、ｉ＝３に設定される。そして、ステップＳ１５４において、ｉがＬより大きいか否かが判定されるが、ｉ＝３であり、ｉはＬより大きくないので、処理はステップＳ１５２に戻る。
【０１０７】
その後、ステップＳ１５２において、ｉ＝３に対応するレイヤ、すなわち図２においてレイヤＬ３のパケットＰ１１乃至Ｐ１５が、画像受信装置１２に送信される。その後、ステップＳ１５３において、ｉが１だけインクリメントされ、ｉ＝４に設定される。そして、ステップＳ１５４において、ｉがＬより大きいか否かが判定されるが、ｉ＝４であり、ｉはＬより大きいので、処理が終了される。
【０１０８】
以上のようにして、画像送信装置１１から画像受信装置１２に対して、１階層分ずつ、データパケットが送信される。
【０１０９】
以上のようにして、データパケットを送信するようにした場合、画像送信装置１１から画像受信装置１２に送信されるデータパケットの個数は、例えば、図１０のように制御される。
【０１１０】
図１０は、図１と同様、縦軸が、ウィンドウサイズを表し、横軸が時間軸を表している。図１０のグラフにおいては、時刻ｔ０乃至ｔ１の間、レイヤＬ１に相当する５個のパケットＰ１乃至Ｐ５が送信するデータパケットとして設定され、時刻ｔ１乃至ｔ２の間、レイヤＬ１およびＬ２に相当する１０個のパケットＰ１乃至Ｐ１０が送信するデータパケットとして設定され、時刻ｔ２乃至ｔ３の間、レイヤＬ１乃至Ｌ３に相当する１５個のパケットＰ１乃至Ｐ１５が送信するデータパケットとして設定され、時刻ｔ３乃至ｔ４の間、レイヤＬ１およびＬ２に相当する１０個のパケットＰ１乃至Ｐ１０が送信するデータパケットとして設定され、時刻ｔ４乃至ｔ５の間、レイヤＬ１乃至Ｌ３に相当する１５個のパケットＰ１乃至Ｐ１５が送信するデータパケットとして設定され、時刻ｔ５乃至ｔ６の間、レイヤＬ１およびＬ２に相当する１０個のパケットＰ１乃至Ｐ１０が送信するデータパケットとして設定され、時刻ｔ６乃至ｔ７の間、レイヤＬ１乃至Ｌ３に相当する１５個のパケットＰ１乃至Ｐ１５が送信するデータパケットとして設定され、時刻ｔ７以降、レイヤＬ１およびＬ２に相当する１０個のパケットＰ１乃至Ｐ１０が送信するデータパケットとして設定されている。
【０１１１】
図１１は、図１０に示される、本発明を適用した場合の送信パケット数の推移を、図１の従来例と比較した図である。
【０１１２】
図１１において、図１の従来例の送信パケットの個数の推移が、点線で示されている。また、図１０に示される、本発明を適用した場合の送信パケット数の推移が、実線で示されている。図１１において、斜線で示された範囲のパケットは、従来例において、画像送信装置１１から画像受信装置１２に送信されても、画質の向上が期待できないデータを含むパケットである。
【０１１３】
各レイヤを構成するデータパケットを、一まとめにして、画像送信装置１１から画像受信装置１２に送信することにより、画像受信装置１２は、受信されたデータパケットのデータを全て利用して、より高画質な画像を提供することができ、画像送信装置１１から画像受信装置１２に対して無駄に送信するデータを無くすことが可能となる。
【０１１４】
また、図１１において、時刻ｔ０乃至ｔ１の間、従来のレート制御では、送信パケット数が５個未満であるため、画像受信装置１２は、これらのデータパケットを受信しても、画像を再現することができない。また、図１１において、時刻ｔ２乃至ｔ３の間、時刻ｔ４乃至ｔ５の間、および時刻ｔ６乃至ｔ７の間においては、送信パケット数は９個であるため、これらのデータパケットを受信した画像受信装置１２は、受信したデータパケットのうち、４個を利用することができなかった。
【０１１５】
それに対して、本発明においては、時刻ｔ０から、送信パケット数が１つのレイヤ分である５個に設定されているため、画像受信装置１２は、データパケットの受信開始直後から、データをデコードして画像を再生することができる。本発明においては、レイヤ単位で、パケットを一まとめに増減させることにより、無駄なデータパケットの送受信をなくすことができる。
【０１１６】
ところで、図１１において、斜線で示された部分は、送信可能なパケット数の余裕を表しているということもできる。そこで、この余裕を利用して、冗長データを送信するようにしても良い。
【０１１７】
図１２は、送信可能なパケット数の余裕を利用して、エラー訂正用の冗長データを送信するようにした場合における、画像送信装置１１のレート制御処理を表すフローチャートである。
【０１１８】
図１２のフローチャートの、ステップＳ２０１乃至ステップＳ２１１の処理は、図８のステップＳ１０１乃至ステップＳ１１１の処理と同様であるため、説明を省略する。図１２においては、ステップＳ２１１の処理の後、ステップＳ２１２において、階層設定部１２３は、ｎ−（Ｌ×５）個分の冗長データを、送信するパケットとして設定する。すなわち、階層設定部１２３は、ステップＳ２０５で算出された送信パケット数ｎから、ステップＳ２１１で設定された、送信するレイヤのパケット数を差し引いた余りの個数分だけ、エラー訂正用の冗長データを、送信するパケットとして設定する。
【０１１９】
その後、処理はステップＳ２０１に戻り、ステップＳ２０１以降の処理が繰り返される。
【０１２０】
以上のように、ステップＳ２０５で算出された送信パケット数ｎから、ステップＳ２１１で設定された、送信するレイヤのパケット数を差し引いた余りの個数分だけ、エラー訂正用の冗長データを、送信するパケットとして設定した場合、階層設定部１２３は、パケット生成部１０５に、ｎ−（Ｌ×５）個分のエラー訂正用の冗長データを、送信するパケットとして作成するように指令する。パケット生成部１０５は、階層設定部１２３からの指令に従って、ｎ−（Ｌ×５）個分のエラー訂正用の冗長データを、送信するパケットとして作成する。
【０１２１】
以上のように、余り個数分だけ、エラー訂正用の冗長データを送信することにより、画像受信装置１２は、パケットの欠損が発生しても、エラー訂正用の冗長データから、欠損したパケットのデータに対応するデータを作成することができる可能性がある。従って、画像受信装置１２は、画像送信装置１１に対して、欠損したパケットの再送要求をしなければならない可能性を減少させることができる。
【０１２２】
ところで、ネットワーク１０が優先制御を提供する場合、優先度を設定して、全データパケットを送信し、優先度の低いデータパケットが何個、画像受信装置１２に届いたかを測定することにより、ネットワーク１０にどの程度、帯域が空いているかを推測することができる。
【０１２３】
例えば、理想的な送信パケット数ｎがｎ＝１２である場合、画像送信装置１１は、レイヤＬ１およびＬ２の１０個のデータパケット（パケットＰ１乃至Ｐ１０）を、高優先度で送信し、レイヤＬ３およびＬ４のデータパケットを低優先度で送信する。そして、画像受信装置１２からの応答パケットに基づいて、画像受信装置１２に届いたデータパケットの個数を特定する。例えば、画像受信装置１２に、７個のデータパケットが届いた場合、画像送信装置１１は、あと７個分のデータパケットを送信するだけの帯域の余裕があると判断し、それ以降の理想的な送信パケット数ｎの値をｎ＝１０＋７＝１７と設定する。このようにしても良い。
【０１２４】
次に、図１３のフローチャートを参照して、優先度の高いレイヤのデータを優先して送信する場合の画像送信装置１１のレート制御処理について説明する。
【０１２５】
図１３のフローチャートの、ステップＳ２３１乃至ステップＳ２３９の処理は、図８のステップＳ１０１乃至ステップＳ１０９の処理と同様であるため、説明を省略する。図１３においては、ステップＳ２３８の処理の後、ステップＳ２４０において、階層設定部１２３は、下位のＬ層までのレイヤのデータパケットを、高優先度で画像受信装置１２に送信するパケットとして設定する。例えば、変数Ｌ＝２である場合、階層設定部１２３は、図２のレイヤＬ１およびＬ２のデータパケットを、高優先度のパケットとして設定する。
【０１２６】
その後、ステップＳ２４１において、階層設定部１２３は、（Ｌ＋１）層以上のレイヤのデータパケットを、低優先度で画像受信装置１２に送信するパケットとして設定する。例えば、変数Ｌ＝２である場合、階層設定部１２３は、図２のレイヤＬ３およびＬ４のデータパケットを、低優先度のパケットとして設定する。
【０１２７】
ステップＳ２４１の処理の後、ステップＳ２４２において、階層設定部１２３は、応答パケットに基づいて、送信パケット数ｎを設定する。すなわち、応答パケットには、損失パケットＩＤが含まれているので、この損失パケットＩＤを基に、何個のデータパケットが、画像受信装置１２に届いたかを求めることができる。そこで、階層設定部１２３は、損失パケットＩＤを基に、低優先度で送信したデータパケットのうち、画像受信装置１２に届いたデータパケットの個数を算出し、算出した値を、現在設定されている送信パケット数ｎに足し算して、新たな送信パケット数ｎを算出する。
【０１２８】
その後、処理はステップＳ２３６に戻り、ステップＳ２３６以降の処理が繰り返される。
【０１２９】
以上のように、優先度の高いレイヤのデータを優先して、送信するようにしてもよい。
【０１３０】
なお、図１３のフローチャートの処理においては、ネットワーク１０の帯域の空き容量を推測することはできるが、たとえ１つのレイヤ分、余分にデータパケットが画像受信装置１２に届いたとしても、それが、あるレイヤを構成する全パケットが揃うとは限らない。そこで、現在高優先度で送信しているレイヤより、１つ上位のレイヤのデータパケットだけ、低優先度で送信するようにしても良い。次に、図１４のフローチャートを参照して、この場合の画像送信装置１１のレート制御処理について説明する。
【０１３１】
図１４のフローチャートの、ステップＳ２６１乃至ステップＳ２７０、並びにステップＳ２７２の処理は、図１３のステップＳ２３１乃至ステップＳ２４０、並びにステップＳ２４２の処理と同様であるため、説明を省略する。図１４においては、ステップＳ２７０の処理の後、ステップＳ２７１において、階層設定部１２３は、（Ｌ＋１）層のレイヤのデータパケットを、低優先度で画像受信装置１２に送信するパケットとして設定する。例えば、変数Ｌ＝２である場合、階層設定部１２３は、図２のレイヤＬ３のデータパケットを、低優先度のパケットとして設定する。その後、処理はステップＳ２７２に進む。
【０１３２】
以上のようにすることにより、例えば、レイヤＬ１を構成するパケットＰ１乃至Ｐ５を高優先度で送信し、レイヤＬ２を構成するパケットＰ６乃至Ｐ１０を低優先度で送信した結果、レイヤＬ１およびＬ２を構成する全パケットＰ１乃至Ｐ１０が、画像受信装置１２に届いた場合、それ以降、レイヤＬ１およびＬ２を構成するパケットＰ１乃至Ｐ１０を、高優先度で送信し、レイヤＬ３を構成するパケットＰ１１乃至Ｐ１５を低優先度で送信することになり、結果的に、より多くの階層符号化データを送信することが可能となる。従って、外部機器が、画像受信装置１２から階層符号化データを取得し再生した場合、より高精細な画像を表示することが可能となる。
【０１３３】
以上、説明したように、本発明によれば、単位時間当りに送信可能なパケット数が増加した場合、レイヤ（階層）の区切り単位分だけ、送信可能なパケット数が増加するまで、送信パケット数の増加を行なわないようにし、単位時間当りに送信可能なパケット数が減少した場合、送信可能なパケット数以下になるように、レイヤ（階層）の区切り単位分だけ、パケット数を減少させるようにしたので、無駄に送受信するデータパケットをなくすことができ、より効率的に、データの送受信を行なうことが可能となる。また、画像受信装置１２の側でも、受信したデータパケットから、利用することができないデータパケットを選り分ける処理を省略することが可能となる。
【０１３４】
なお、以上の説明においては、図２に示されるような、４つのレイヤを有し、１つのレイヤが５個のパケットに分割される階層符号化データの場合を例にしているが、階層符号化データであれば、レイヤの数、および１レイヤ当りのパケット数は、図２に示されるような個数でなくても良い。
【０１３５】
また、本発明は、実時間ストリーミングに適用することも可能である。
【０１３６】
上述した一連の処理は、ハードウェアにより実行させることもできるが、ソフトウェアにより実行させることもできる。一連の処理をソフトウェアにより実行させる場合には、そのソフトウェアを構成するプログラムが、専用のハードウェアに組み込まれているコンピュータ、または、各種のプログラムをインストールすることで、各種の機能を実行させることが可能な、例えば汎用のパーソナルコンピュータなどに記録媒体からインストールされる。
【０１３７】
図１５は、画像送信装置１１、または画像受信装置１２をソフトウェアにより実現する場合の情報処理装置の一実施の形態の構成を示している。パーソナルコンピュータ４００のCPU４０１は、パーソナルコンピュータ４００の動作の全体を制御する。また、CPU４０１は、バス４０４および入出力インターフェース４０５を介してユーザからキーボードやマウスなどからなる入力部４０６から指令が入力されると、それに対応してROM(Read Only Memory)４０２に格納されているプログラムを実行する。あるいはまた、CPU４０１は、ドライブ４１０に接続された磁気ディスク４２１、光ディスク４２２、光磁気ディスク４２３、または半導体メモリ４２４から読み出され、記憶部４０８にインストールされたプログラムを、RAM(Random Access Memory)４０３にロードして実行する。これにより、上述した画像送信装置１１、または画像受信装置１２の機能が、ソフトウェアにより実現されている。さらに、CPU４０１は、通信部４０９を制御して、外部と通信し、データの授受を実行する。また、入出力インタフェース４０５には、ディスプレイやスピーカなどにより構成される出力部４０７も接続されている。
【０１３８】
プログラムが記録されている記録媒体は、図１５に示すように、コンピュータとは別に、ユーザにプログラムを提供するために配布される、プログラムが記録されている磁気ディスク４２１（フレキシブルディスクを含む）、光ディスク４２２（CD-ROM(Compact Disk-Read Only Memory)，DVD（Digital Versatile Disk）を含む）、光磁気ディスク４２３（MD（Mini-Disc）を含む）、もしくは半導体メモリ４２４などよりなるパッケージメディアにより構成されるだけでなく、コンピュータに予め組み込まれた状態でユーザに提供される、プログラムが記録されているROM４０２や、記憶部４０８に含まれるハードディスクなどで構成される。
【０１３９】
尚、本明細書において、記録媒体に記録されるプログラムを記述するステップは、記載された順序に沿って時系列的に行われる処理は、もちろん、必ずしも時系列的に処理されなくとも、並列的あるいは個別に実行される処理を含むものである。
【０１４０】
また、本明細書において、システムとは、複数の装置により構成される装置全体を表すものである。
【０１４１】
【発明の効果】
以上のように、第１の本発明によれば、ネットワークを介して、情報を送受信することができる。また、第１の本発明によれば、より効率的に、情報を送受信することができる。さらに、第１の本発明によれば、ネットワークの帯域の空き領域を利用して、エラー訂正を行なうことができる。また、第１の本発明によれば、ネットワークの帯域の空き容量を予測することができる。
【０１４２】
第２の本発明によれば、ネットワークを介して、情報を送信することができる。また、第２の本発明によれば、より効率的に、情報を送信することができる。さらに、第２の本発明によれば、ネットワークの帯域の空き領域を利用して、エラー訂正を行なうことができる。また、第２の本発明によれば、ネットワークの帯域の空き容量を予測することができる。
【図面の簡単な説明】
【図１】従来の送信パケット数の制御を説明するための図である。
【図２】JPEG2000の符号化方式により符号化された１フレーム分のパケットの例を示す図である。
【図３】本発明を適用した情報処理システムの構成例を示すブロック図である。
【図４】画像送信装置の構成例を示すブロック図である。
【図５】画像受信装置の構成例を示すブロック図である。
【図６】画像送信装置の画像送信処理、および画像受信装置の画像受信処理を説明するフローチャートである。
【図７】画像受信装置の確認応答処理を説明するフローチャートである。
【図８】画像送信装置のレート制御処理を説明するフローチャートである。
【図９】図６のステップＳ２を詳細に説明するフローチャートである。
【図１０】本発明における送信パケット数の制御を説明する図である。
【図１１】本発明における送信パケット数の制御を従来の送信パケット数の制御と比較する図である。
【図１２】画像送信装置のレート制御処理を説明する他のフローチャートである。
【図１３】画像送信装置のレート制御処理を説明する、さらに他のフローチャートである。
【図１４】画像送信装置のレート制御処理を説明するフローチャートである。
【図１５】本発明を適用したコンピュータの一実施の形態の構成例を示すブロック図である。
【符号の説明】
１０ ネットワーク， １１ 画像送信装置， １２ 画像受信装置， １０１ 操作部， １０２ 制御部， １０３ 記録媒体， １０４ 入力部， １０５ パケット生成部， １０６ パケット送信部， １０７ パケット受信部， １０８ レート制御部， １２１ 送信レート算出部， １２２ 階層設定部， １５１ 操作部， １５２ 制御部， １５３ パケット受信部， １５４ 記録媒体， １５５ 状態測定部， １５６ パケット生成部， １５７ パケット送信部， ４０１ CPU， ４０２ ROM， ４０３ RAM， ４０４ バス， ４０５ 入出力インタフェース， ４０６ 入力部， ４０７ 出力部， ４０８ 記憶部， ４０９ 通信部， ４１０ ドライブ， ４２１ 磁気ディスク， ４２２ 光ディスク， ４２３ 光磁気ディスク， ４２４ 半導体メモリ[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an information processing system, an information processing apparatus and method, a recording medium, and a program, and more particularly to an information processing system, an information processing apparatus and method, a recording medium, and a program that can transmit and receive information more efficiently. .
[0002]
[Prior art]
The control of the data transmission rate determines the amount of transmission data per unit time. On the Internet, which is a packet communication network, if the packet size (data amount per packet) is constant, the transmission rate can be controlled by controlling the number of packets transmitted per unit time.
[0003]
An example of controlling the transmission rate by controlling the number of packets transmitted per unit time is a sliding window algorithm defined in RFC (Request For Comments) 2001. This sliding window algorithm will be described with reference to FIG.
[0004]
In FIG. 1, the vertical axis represents the window size. In the sliding window algorithm, the transmitting terminal device can transmit as many packets as are included in the “window”. The window is variable in size, and the vertical axis of the graph in FIG. 1 represents the size of this window. In FIG. 1, the horizontal axis is a time axis.
[0005]
At time t0 in FIG. 1, the transmitting terminal first sets the window size to the size of one packet, and transmits one packet to the receiving terminal. When receiving the acknowledgment of the packet reception from the receiving terminal device, the transmitting terminal device sets the window size to two packets and transmits the two packets to the receiving terminal device. In this way, the transmitting terminal increases the size of the window and increases the number of packets that can be transmitted one by one each time an acknowledgment of packet reception is received from the receiving terminal. In this way, the transmitting terminal device gradually increases the number of packets transmitted per unit time.
[0006]
Here, at time t1, it is assumed that an acknowledgment indicating that at least a part of the packet transmitted from the transmitting terminal device has not been received by the receiving terminal device (a packet loss has occurred) has been notified from the receiving terminal device. . At this time (t1), the transmitting terminal device reduces the size of the window to half, and reduces the number of packets that can be transmitted to half (in FIG. 1, the number is reduced from 19 to 9). The fraction is rounded down.) Thereafter, when the transmitting terminal device receives acknowledgments from the receiving terminal device for all the packets in the window, it increases the size of the window and increases the number of packets that can be transmitted one by one.
[0007]
Then, at time t2, if a packet loss occurs again, the transmitting terminal device reduces the size of the window to half and the number of transmittable packets to half in the same manner as described above. Thereafter, when the transmitting terminal device receives acknowledgments from the receiving terminal device for all the packets in the window, the transmitting terminal device increases the size of the window and increases the number of packets that can be transmitted one by one. Thereafter, processing is performed at time t3 in the same manner as at time t1 and time t2.
[0008]
In the sliding window algorithm, the optimum data transmission rate is determined as described above.
[0009]
As described above, in order to quickly determine the optimal data transmission rate in accordance with the state of the network, when the transmission rate increases, the number of transmission packets is added (in the above example, one packet at a time). Generally, an AIMD (Additive Increase Multiple Decrease) algorithm that multiplies the number of transmission packets by a value less than 1 (1/2 in the above example) when the transmission rate decreases is generally known. (For example, see Non-Patent Document 1).
[0010]
[Non-patent document 1]
D. Chiu and R. Jain, “Analysis of the Increase / Decrease Algorithms for Congestion Avoidance in Computer Networks,” Journal of Computer Networks and ISDN, Vol. 17, No. 1, June 1989 pp.1-14
[0011]
[Problems to be solved by the invention]
Incidentally, data to be transmitted may be hierarchically encoded data. For example, JPEG (Joint Photographic Experts Group) 2000 is an example of hierarchically encoded data.
[0012]
As an example, when the hierarchically encoded data of JPEG2000 is hierarchically encoded by four layers, and five packets are required to transmit data of one layer, the hierarchical code of the entire four layers is required. The coded data requires 4 × 5 = 20 packets. FIG. 2 shows an example of data for one frame hierarchically encoded by JPEG2000.
[0013]
In FIG. 2, “Layer L1”, “Layer L2”, “Layer L3”, and “Layer L4” on the left side represent four layers. The layer L1 is the lowest layer, and becomes a higher layer in the order of the layer L2, the layer L3, and the layer L4.
[0014]
In FIG. 2, blocks on the right side of “Layer L1,” “Layer L2,” “Layer L3,” and “Layer L4” each represent a packet. That is, “P1”, “P2”, “P3”, “P4”, “P5”, “P6”, “P7”, “P8”, “P9”, “P10”, “P11”, “P12”, Each block described as “P13”, “P14”, “P15”, “P16”, “P17”, “P18”, “P19”, and “P20” represents a packet.
[0015]
In FIG. 2, the data of the layer L1 is divided into packets of "P1,""P2,""P3,""P4," and "P5," and the data of the layer L2 is divided into "P6,""P7." , “P8”, “P9”, and “P10” packets, and the data of layer L3 is divided into “P11”, “P12”, “P13”, “P14”, and “P15” packets. , Layer L4 data is divided into packets of “P16”, “P17”, “P18”, “P19”, and “P20”.
[0016]
In JPEG2000, when only the data of the layer L1 is decoded and displayed, a coarse image (this image is referred to as image 1) is displayed. When the data of the layer L1 and the layer L2 are decoded and displayed, an image is displayed. 1 is displayed (this image is referred to as image 2), and when the data of layer L1, layer L2, and layer L3 are decoded and displayed, an image finer than image 2 (this image is referred to as image 3) ) Is displayed, and when the data of the layer L1, the layer L2, the layer L3, and the layer L4 are decoded and displayed, an image finer than the image 3 (this image is referred to as an image 4) is displayed.
[0017]
However, in FIG. 2, when the data of all the packets constituting one layer are prepared, a clear image can be displayed. For example, only when all the packets P1 to P5 are collected, the data of the layer L1 can be decoded and a clear image can be displayed. Further, only when all the packets P1 to P10 are collected, the data of the layer L1 and the data of the layer L2 can be decoded and a clear image can be displayed. Similarly, only when all the packets P1 to P15 are collected, the data of the layer L1, the layer L2, and the layer L3 can be decoded, and a clear image can be displayed. Further, only when all the packets P1 to P20 are collected, the data of the layer L1, the layer L2, the layer L3, and the layer L4 can be decoded, and a clear image can be displayed.
[0018]
Therefore, for example, the image quality of the image when the data of the packets P1 to P6 is decoded does not improve much as compared with the image quality of the image when the data of the layer L1, that is, the data of the packets P1 to P5 are decoded. That is, even if there is data of the packet P6, the effect due to the data cannot be expected. Similarly, for example, the image quality of the image when the data of the packets P1 to P7 is decoded does not significantly improve as compared with the image quality of the image when the data of the layer L1, that is, the data of the packets P1 to P5 are decoded. That is, even if there is data of the packets P6 and P7, the effect due to the data cannot be expected. Similarly, for example, the image quality of the image when the data of the packets P1 to P8 is decoded does not improve much as compared with the image quality of the image when the data of the layer L1, that is, the data of the packets P1 to P5 are decoded. That is, even if there is data of the packets P6 to P8, the effect due to the data cannot be expected. Similarly, for example, the image quality of the image when the data of the packets P1 to P9 is decoded does not significantly improve as compared with the image quality of the image when the data of the layer L1, that is, the data of the packets P1 to P5 are decoded. That is, even if there is data of the packets P6 to P9, the effect due to the data cannot be expected.
[0019]
When transmitting and receiving such hierarchically encoded data by the sliding window algorithm, the transmitting terminal apparatus sequentially starts from the packet P1, which is a packet with a small number (P1, P2, P3, P4, P5,..., P19, P20 in order). ), Send the packet. That is, the transmitting terminal device preferentially transmits the packet with the smaller number in FIG. 2, that is, the packet of the layer L1, to the receiving terminal device. Transmit to the receiving terminal device.
[0020]
Here, if the window size is the size of five packets, the transmitting terminal transmits packets P1 to P5 to the receiving terminal, and the receiving terminal transmits packets P1 to P5 received from the transmitting terminal. An image corresponding to the layer L1 can be displayed using all the data of P5. Also, if the window size is the size of ten packets, the transmitting terminal device transmits the packets P1 to P10 to the receiving terminal device, and the receiving terminal device receives the packets P1 to P10 received from the transmitting terminal device. , Images corresponding to the layers L1 and L2 can be displayed. If the window size is the size of 15 packets, the transmitting terminal device transmits the packets P1 to P15 to the receiving terminal device, and the receiving terminal device transmits the packets P1 to P15 received from the transmitting terminal device. , Images corresponding to the layers L1, L2, and L3 can be displayed. If the window size is the size of 20 packets, the transmitting terminal device transmits the packets P1 to P20 to the receiving terminal device, and the receiving terminal device receives the packets P1 to P20 received from the transmitting terminal device. , The images corresponding to the layers L1, L2, L3, and L4 can be displayed.
[0021]
However, when all the packets constituting one layer are not prepared, even if the data of the packets included in the layer is used, it is not expected to improve the image quality. Therefore, there is no point in transmitting those packets. Therefore, there is a problem that packets to be transmitted are wasted.
[0022]
That is, for example, when the window size is the size of six packets, the transmitting terminal device transmits the packets P1 to P6 to the receiving terminal device, but the receiving terminal device receives the packet P1 received from the transmitting terminal device. Even if the data of the packet P6 among the data P6 to P6 is used, the improvement of the image quality cannot be expected, so that the transmission of the packet P6 is useless. Similarly, when the window size is the size of seven packets, the transmitting terminal device transmits the packets P1 to P7 to the receiving terminal device, but the receiving terminal device receives the packet P1 received from the transmitting terminal device. Even if the data of the packets P6 and P7 among the data of the packets P6 and P7 are used, the improvement of the image quality cannot be expected, so that the transmission of the packets P6 and P7 is useless. Similarly, when the window size is, for example, the size of eight packets, the transmitting terminal device transmits the packets P1 to P8 to the receiving terminal device, but the receiving terminal device receives the packets from the transmitting terminal device. Even if the data of the packets P6, P7, and P8 among the packets P1 to P8 is used, the improvement of the image quality cannot be expected, so that the transmission of the packets P6, P7, and P8 is useless. Similarly, when the window size is, for example, the size of nine packets, the transmitting terminal transmits packets P1 to P9 to the receiving terminal, but the receiving terminal receives the packets from the transmitting terminal. Even if the data of the packets P6, P7, P8, and P9 among the packets P1 to P9 is used, it is not possible to expect an improvement in the image quality. Therefore, it is useless to transmit the packets P6, P7, P8, and P9. .
[0023]
As described above, when transmitting and receiving the hierarchically encoded data by the sliding window algorithm, if the packets to be transmitted from the transmitting terminal device to the receiving terminal device are not complete for one layer, the transmitting terminal device transmits to the receiving terminal device. There is a problem that a part of the packet to be used is wasted.
[0024]
Also, the receiving terminal device needs to select an unusable packet from the received packets.
[0025]
The present invention has been made in view of such a situation, and has as its object to transmit and receive information more efficiently.
[0026]
[Means for Solving the Problems]
In the information processing system according to the present invention, the first information processing apparatus packetizes the hierarchically coded data to generate a first packet, and converts the first packet generated by the generation means into a second packet. A first transmitting unit for transmitting the first packet to the information processing device, a second packet including information on a transmission time of the first packet by the first information processing device, and information on the loss of the first packet. First receiving means for receiving from the processing device, calculating means for calculating a reference value of the number of packets of the first packet to be transmitted based on the second packet received by the first receiving means, and calculating means Setting means for setting the hierarchy of the hierarchically coded data to be packetized by the generation means based on the reference value calculated by the second information processing apparatus. Packet Receiving means for receiving the hierarchically encoded data from the first packet received by the second receiving means, and receiving the first packet received by the second receiving means from the first packet received by the second receiving means. And a second transmitting unit configured to transmit the second packet generated by the generating unit to the first information processing apparatus.
[0027]
An information processing apparatus according to the present invention generates a first packet by packetizing hierarchically encoded data, and a transmitting unit that transmits the first packet generated by the generating means to another information processing apparatus. Receiving means for receiving, from another information processing apparatus, a second packet including information on the transmission time of the first packet by the information processing apparatus and the loss of the first packet, and a second packet received by the receiving means. Calculating means for calculating a first number, which is a reference value of the number of packets of the first packet to be transmitted, based on the second packet; and generating means for generating a packet based on the first number calculated by the calculating means. Setting means for setting the layer of the layer encoded data to be converted.
[0028]
The setting unit sets a hierarchy of the hierarchical data to be packetized by the generation unit so that the number of packets of the first packet generated by the generation unit is equal to or less than the first number. You can make it.
[0029]
In the setting unit, when the layer encoded data of the layer set as the layer to be packetized is packetized, a second number that is the number of packets of the first packet generated by the generation unit is: Setting the hierarchy to be packetized so as to be equal to or less than the first number, and calculating a third number which is a remainder obtained by subtracting a second number from the first number; The means can further cause the redundant data to be packetized by a third number to generate the first packet.
[0030]
The setting means may include, when packetizing the hierarchically encoded data of the layer to be transmitted with priority, the second number which is the number of packets of the first packet generated by the generating means, Setting the hierarchy to be transmitted preferentially so as to be less than or equal to 1 and calculating a third number which is a remainder obtained by subtracting a second number from the first number; Means for packetizing the hierarchically encoded data of the layer to be transmitted preferentially to generate the first packet, and for the third number, the hierarchically encoded data of the layer not to be transmitted preferentially by the third number; Packetization can be performed to generate a first packet.
[0031]
According to the information processing method of the present invention, a generation step of packetizing hierarchically encoded data to generate a first packet, and transmitting the first packet generated by the processing of the generation step to another information processing apparatus. A transmitting step, a receiving time of transmitting the first packet by the information processing apparatus, and a receiving step of receiving a second packet including information on the loss of the first packet from another information processing apparatus; A calculating step of calculating a first number, which is a reference value of the number of packets of the first packet to be transmitted, based on the received second packet; and a first number calculated by the processing of the calculating step. A setting step of setting a layer of the layer coded data to be packetized by the processing of the generation step.
[0032]
The program of the recording medium according to the present invention includes a generation step of packetizing hierarchically encoded data to generate a first packet for transmission to another information processing apparatus, and transmitting the first packet by the information processing apparatus. When the second packet including the time and the information on the loss of the first packet is received from another information processing apparatus, the number of packets of the first packet to be transmitted is calculated based on the received second packet. A calculating step of calculating a reference value; and a setting step of setting a layer of the layer coded data to be packetized by the processing of the generating step based on the reference value calculated by the processing of the calculating step. .
[0033]
The program according to the present invention is configured to transmit the hierarchically encoded data, which is the hierarchically encoded data, to another information processing apparatus by a packet communication. Generating a first packet by packetizing the hierarchically encoded data, and transmitting a second packet including information on a transmission time of the first packet by the information processing apparatus and loss of the first packet, A calculation step of calculating a reference value of the number of packets of the first packet to be transmitted based on the received second packet when received from another information processing apparatus; and a reference value calculated by the processing of the calculation step. A setting step of setting a layer of the layer-encoded data to be packetized by the processing of the generation step based on .
[0034]
In the information processing system of the present invention, in the first information processing device, the hierarchically encoded data is packetized to generate a first packet, and the generated first packet is transmitted to the second information processing device. Is transmitted from the second information processing device, the second packet including the transmission time of the first packet by the first information processing device and the information on the loss of the first packet is received from the second information processing device. A reference value of the number of packets of the first packet to be transmitted is calculated based on the second packet, and a layer of the layer coded data to be packetized is set based on the calculated reference value. Further, in the second information processing device, the first packet is received from the first information processing device, hierarchically encoded data is obtained from the received first packet, and the received first packet is received. , A second packet is created, and the created second packet is transmitted to the first information processing device.
[0035]
In the information processing apparatus and method, the recording medium, and the program according to the present invention, the hierarchically encoded data is packetized to generate a first packet, and the generated first packet is transmitted to another information processing apparatus. A second packet, which is transmitted and includes a transmission time of the first packet by the information processing device and information on the loss of the first packet, is received from another information processing device, based on the received second packet. Then, a first number, which is a reference value of the number of packets of the first packet to be transmitted, is calculated, and based on the calculated first number, a layer of hierarchically encoded data to be packetized is set.
[0036]
The present invention can be applied to electronic devices connected to a network.
[0037]
BEST MODE FOR CARRYING OUT THE INVENTION
FIG. 3 is a diagram showing a configuration of an embodiment of an information processing system to which the present invention is applied.
[0038]
In FIG. 3, an image transmitting device 11 and an image receiving device 12 are connected via a network 10 including the Internet. The image transmitting device 11 generates and generates a packet including hierarchically encoded data encoded by an encoding method such as JPEG2000 (in the following description, a packet including hierarchically encoded data is referred to as a data packet). The data packet is transmitted to the image receiving device 12 via the network 10 in real time. In addition, the image transmitting apparatus 11 transmits a packet including information on a transmission time at which the image transmitting apparatus 11 transmitted a data packet and a packet loss from the image receiving apparatus 12 (in the following description, from the image receiving apparatus 12 to the image transmitting apparatus 11). The transmitted packet is referred to as a response packet), and the transmission rate of the data packet is controlled based on the information.
[0039]
The image receiving device 12 acquires hierarchically encoded data from the data packet received from the image transmitting device 11 and records it on a predetermined recording medium, and also transmits information on the transmission time and packet loss of the data packet by the image transmitting device 11. A response packet including the response packet is generated, and the response packet is transmitted to the image transmission device 11.
[0040]
Next, FIG. 4 illustrates an example of the internal configuration of the image transmission device 11. 4, an operation unit 101 receives an input of an operation from a user, and notifies the control unit 102 of operation information corresponding to the received operation. The control unit 102 controls the operation of each unit of the image transmission device 11 based on a preset program and operation information from the operation unit 101.
[0041]
The recording medium 103 records hierarchically encoded data encoded by an encoding method such as JPEG2000, and supplies the hierarchically encoded data to the packet generation unit 105 as appropriate. The input unit 104 receives an input of hierarchically encoded data from an external device (for example, an imaging device or a microphone) not shown, and supplies the input hierarchically encoded data to the packet generation unit 105.
[0042]
The packet generation unit 105 divides the layer coded data supplied from the recording medium 103 or the input unit 104 according to the control of the layer setting unit 123, and generates a data packet including the divided data. For example, as illustrated in FIG. 2, the packet generation unit 105 divides the hierarchically coded data into a plurality of layers, and generates a data packet including each data. Then, the packet generation unit 105 discards the data packets instructed to be discarded from the hierarchy setting unit 123 among the generated data packets, and supplies the remaining packets that have not been discarded to the packet transmission unit 106. I do. The packet generation unit 105 adds a packet header of Transmission Control Protocol / Internet Protocol (TCP / IP) or User Datagram Protocol / Internet Protocol (UDP / IP) to the generated data packet. The packet header includes a sequence number for identifying each data packet (hereinafter, a sequence number included in the packet header is referred to as a packet ID).
[0043]
When the data packet generated by the packet generation unit 105 is supplied, the packet transmission unit 106 transmits the data packet to the image reception device 12 via the network 10 at a predetermined timing. Note that the packet transmission unit 106 is controlled by the hierarchy setting unit 123. Also, when transmitting a data packet, the packet transmitting unit 106 records the current time in the packet header of the data packet as the transmission time of the data packet.
[0044]
The packet receiving unit 107 receives the response packet from the image receiving device 12 and supplies this to the transmission rate calculating unit 121.
[0045]
The rate control unit 108 controls the packet generation unit 105 and the packet transmission unit 106 based on the response packet received by the packet reception unit 107, and adjusts the data packet transmission rate to an optimal value. The transmission rate calculation unit 121 in the rate control unit 108 calculates an ideal transmission rate of data packets based on the response packet supplied from the packet reception unit 107, and notifies the calculation result to the packet number calculation unit 122. . The packet number calculation unit 122 calculates the ideal number of packets to be transmitted to the image receiving device 12 based on the notification from the transmission rate calculation unit 121, and notifies the hierarchy setting unit 123. The layer setting unit 123 sets the layer (layer) of the data packet to be transmitted to the image receiving device 12 based on the transmission rate notified from the packet number calculation unit 122.
[0046]
Next, FIG. 5 illustrates an example of the internal configuration of the image receiving device 12. In FIG. 5, an operation unit 151 receives an input of an operation from a user, and notifies the control unit 152 of operation information corresponding to the received operation. The control unit 152 controls the operation of each unit of the image receiving device 12 based on a preset program and operation information from the operation unit 151.
[0047]
The packet receiving unit 153 receives the data packet transmitted from the image transmitting device 11 via the network 10, records the hierarchically encoded data included in the data packet on the recording medium 154, and The added packet header is read, and the information contained in the packet header, that is, information on the packet ID and the transmission time of the data packet is supplied to the state measuring unit 155.
[0048]
The recording medium 154 includes, for example, a hard disk, and records the hierarchically encoded data supplied from the packet receiving unit 153, and reads out the recorded hierarchically encoded data by an external device (not shown).
[0049]
The state measurement unit 155 receives information on the packet ID and the transmission time of the data packet from the packet reception unit 153. Then, the state measuring unit 155 determines the presence or absence of a lost packet based on the packet ID supplied from the packet receiving unit 153, and if there is a lost packet, acquires the sequence number of the lost packet as the lost packet ID. Further, state measurement section 155 acquires the reception time at which the data packet was received. Then, the state measurement unit 155 supplies the packet generation unit 156 with the data packet transmission time and the packet ID read from the packet header, and the data packet reception time and the lost packet ID acquired by itself.
[0050]
When the data packet transmission time and the packet ID read from the packet header and the data packet reception time and the lost packet ID obtained by itself are supplied from the state measurement unit 155, Is generated, and the response packet is supplied to the packet transmission unit 157.
[0051]
The packet transmission unit 157 transmits the response packet supplied from the packet generation unit 156 to the image transmission device 11 via the network 10. When transmitting the response packet, the packet transmission unit 157 records the current time as the transmission time of the response packet in the packet header of the response packet.
[0052]
Next, an image transmission process of the image transmission device 11 and an image reception process of the image reception device will be described with reference to a flowchart of FIG. In the following description, a case where hierarchical encoded data encoded by the JPEG2000 encoding method shown in FIG. 2 is adopted as hierarchical encoded data transmitted from the image transmitting apparatus 11 to the image receiving apparatus 12 will be described. explain. That is, as shown in FIG. 2, the layer coded data has four layers (layers), and each layer is divided into five packets.
[0053]
In step S1 of FIG. 6, the packet generation unit 105 of the image transmission device 11 packetizes the hierarchically encoded data supplied from the recording medium 103 or the input unit 104 according to the control of the hierarchical setting unit 123, and converts the data packet into a packet. Generate. In step S2, the packet transmitting unit 106 transmits the data packet generated in step S1 by the packet generating unit 105 to the image receiving device 12 via the network 10. Thereafter, the process returns to step S1, and the processes after step S1 are repeated. A detailed description of the transmission processing in step S2 will be described later.
[0054]
In step S11, the packet receiving unit 153 of the image receiving device 12 receives the data packet transmitted by the image transmitting device 11 in step S2. In step S12, the packet receiving unit 153 of the image receiving device 12 reads the hierarchically encoded data from the data packet received in step S11, and records the hierarchically encoded data on the recording medium 154. Thereafter, the process returns to step S11, and the processes after step S11 are repeated.
[0055]
As described above, the data packet is transmitted from the image transmitting device 11 to the image receiving device 12 in real time, and is recorded on the recording medium 154. The hierarchically encoded data recorded on the recording medium 154 is appropriately read out by an external device, decoded, and an image is reproduced.
[0056]
In the present invention, a response packet is transmitted from the image receiving device 12 to the image transmitting device 11, and the transmission rate of the data packet is controlled based on the response packet.
[0057]
Next, the acknowledgment processing of the image receiving apparatus 12, that is, the processing in which the image receiving apparatus 12 creates a response packet and transmits it to the image transmitting apparatus 11, will be described with reference to the flowchart of FIG.
[0058]
In step S51, the control unit 152 of the image receiving apparatus 12 monitors the packet receiving unit 153 and waits until the packet receiving unit 153 receives a data packet. When the packet receiving unit 153 receives a data packet, the process is performed. Proceed to step S52. In step S52, the state measuring unit 155 acquires the current time from a built-in internal clock. In step S53, the state measuring unit 155 acquires the transmission time of the data packet and the packet ID, which are the information added as the packet header, from the packet receiving unit 153, and determines the packet loss based on the packet ID. Determine the presence or absence.
[0059]
That is, since the packet ID is a sequence number, it is determined whether or not the packet ID of the data packet received this time is a serial number with the packet ID of the data packet received last time. The presence or absence can be determined. For example, if the packet ID of the data packet received last time is “10” and the packet ID of the data packet received this time is “11”, it is continuous with “10” and “11”. 155 determines that there is no data packet loss. For example, if the packet ID of the previously received data packet is “10” and the packet ID of the data packet received this time is “12”, “10” and “12” are not continuous, and The measurement unit 155 determines that the data packet has been lost.
[0060]
If the state measurement unit 155 determines that there is no data packet loss in step S53, the process proceeds to step S54.
[0061]
In step S54, the state measurement unit 155 supplies the reception time of the data packet acquired in step S52, the transmission time and the packet ID of the data packet acquired from the packet header of the data packet to the packet generation unit 156. The packet generation unit 156 generates a response packet supplied from the state measurement unit 155 and including the data packet reception time, the data packet transmission time and the packet ID as information, and supplies the response packet to the packet transmission unit 157. I do. Thereafter, the process proceeds to step S56.
[0062]
If the state measurement unit 155 determines in step S53 that there is a data packet loss, the process proceeds to step S55.
[0063]
In step S55, the state measuring unit 155 specifies the packet ID of the lost packet. For example, when the packet ID of the data packet received last time is “10” and the packet ID of the data packet received this time is “12”, the state measuring unit 155 specifies that the lost packet ID is “11”. I do. The state measurement unit 155 supplies the reception time of the data packet acquired in step S52, the transmission time and packet ID of the data packet acquired from the packet header of the data packet, and the lost packet ID to the packet generation unit 156. The packet generation unit 156 generates a response packet including the data packet reception time, the data packet transmission time and the packet ID, and the lost packet ID supplied from the state measurement unit 155 as information, and transmits the response packet to the packet transmission unit. To the unit 157. Thereafter, the process proceeds to step S56.
[0064]
In step S56, the packet transmitting unit 157 transmits the response packet supplied from the packet generating unit 156 to the image transmitting device 11 via the network 10. Immediately before transmitting the response packet, the packet transmitting unit 157 acquires the current time from the internal clock contained therein, and records the current time in the packet header as the transmission time of the response packet.
[0065]
Thereafter, the process returns to step S51, and the processes after step S51 are repeated.
[0066]
As described above, the response packet is transmitted from the image receiving device 12 to the image transmitting device 11.
[0067]
The image transmitting device 11 controls the data packet transmission rate based on the response packet received from the image receiving device 12.
[0068]
Next, the rate control process of the image transmitting apparatus 11, that is, the control of the data packet transmission rate will be described with reference to the flowchart of FIG.
[0069]
In step S101, the control unit 102 monitors the packet receiving unit 107 and waits until the packet receiving unit 107 receives a response packet. Then, when the packet receiving unit 107 receives the response packet, the processing proceeds to step S102.
[0070]
In step S102, the transmission rate calculation unit 121 of the rate control unit 108 acquires the response packet received by the packet reception unit 107 from the packet reception unit 107, and sets the reception time at which the packet reception unit 107 received the response packet, Obtained from the internal clock. Then, the transmission rate calculation unit 121 calculates the RTT (Round Trip Time) from the obtained reception time of the response packet, the transmission time of the data packet included in the response packet, the reception time of the data packet, and the transmission time of the response packet. ) Is calculated.
[0071]
That is, the RTT is the time required for the data packet transmitted from the image transmitting device 11 to be received by the image receiving device 12 (this time is referred to as T1), and the response packet transmitted from the image receiving device 12 It is the sum (T1 + T2) of the time until receiving by the device 11 (this time is defined as T2).
[0072]
Here, the time T1 until the data packet transmitted from the image transmission device 11 is received by the image reception device 12 is obtained by subtracting the transmission time of the data packet from the reception time of the data packet. The time T2 until the response packet transmitted from the image receiving device 12 is received by the image transmitting device 11 is obtained by subtracting the response packet transmission time from the response packet reception time.
[0073]
As described above, after the transmission rate calculation unit 121 acquires the RTT, the processing proceeds to step S103.
[0074]
In step S103, the transmission rate calculation unit 121 calculates a packet loss rate per unit time based on the lost packet ID included in the response packet.
[0075]
In step S104, the transmission rate calculation unit 121 calculates the transmission rate T. As a method of calculating the transmission rate, for example, TFRC (TCP-Friendly Rate Control) can be used. The TFRC is described in detail in S. Floyd, M. Handley, J. Pandye and J. Widmer, "Equation-Based Congestion Control for Unicast Applications", Proceedings of ACM SIGCOMM 2000, May 2000. In the TFRC, an ideal transmission rate at which a data packet is transmitted from the image transmission device 11 is predicted by using the packet loss rate and the RTT according to the following equation (this equation is represented by equation (1)).
[0076]
(Equation 1)

[0077]
In Expression (1), T is an ideal transmission rate, s is a packet size, p is a packet loss rate, and t is a timeout period of TCP (Transmission Control Protocol) (normally, four times RTT). After calculating the transmission rate T according to Equation (1), the transmission rate calculation unit 121 notifies the calculated transmission rate T to the packet number calculation unit 122. Thereafter, the process proceeds to step S105.
[0078]
In step S105, the number-of-packets calculation unit 122 calculates the number of transmission packets by the following equation (this equation is defined as equation (2)).
[0079]
n = (f × s) / T
[0080]
In equation (2), n is the number of transmission packets, f is the frame rate, s is the packet size, and T is the transmission rate calculated in step S104. After calculating the number of transmission packets n, the packet number calculation unit 122 notifies the hierarchy setting unit 123 of the calculated number of transmission packets n. Thereafter, the process proceeds to step S106.
[0081]
In step S106, the hierarchy setting unit 123 initializes the variable L to L = 0. Thereafter, the process proceeds to step S107. In step S107, the hierarchy setting unit 123 determines whether the following equation (this equation is referred to as equation (3)) is satisfied.
[0082]
(L + 1) × 5 ≦ n
[0083]
In Expression (3), “5” represents the number of packets included in one layer. That is, in FIG. 2, since one layer is divided into five packets, it is set to “5” in equation (3). Then, when Expression (3) is satisfied, the process proceeds to Step S108.
[0084]
In step S108, the hierarchy setting unit 123 determines whether or not the variable L is 4 or more. When it is determined that the variable L is not 4 or more (less than 4), the process proceeds to step S109. Note that the criterion "4" for the determination in step S108 indicates the number of layers. That is, in FIG. 2, since the hierarchically encoded data is composed of four layers, L1 to L4, the determination criterion is set to “4” in step S108.
[0085]
In step S109, the hierarchy setting unit 123 increments the value of the variable L by one. Thereafter, the process returns to step S107, and the processes after step S107 are repeated.
[0086]
If the hierarchy setting unit 123 determines in step S107 that Expression (3) is not satisfied, the process of step S108 is skipped, and the process proceeds to step S110. If the hierarchy setting unit 123 determines in step S108 that the variable L is 4 or more, the process proceeds to step S110.
[0087]
For example, if it is calculated in step S105 that n = 17, the processing in steps S106 to S109 is as follows.
[0088]
First, in step S106, L = 0 is initialized, and in step S107, the variables L = 0 and n = 17 are substituted into equation (3), and it is determined whether or not equation (3) holds. . If the variable L = 0, equation (3) holds, and the process proceeds to step S108. In step S108, it is determined whether or not the variable L is 4 or more. Since the variable L = 0, it is determined that the variable L is not 4 or more, and the process proceeds to step S109. In step S109, the variable L is incremented by 1 and set to L = 1.
[0089]
Thereafter, the process returns to step S107, and the variable L = 1 and n = 17 are substituted into the equation (3), and it is determined whether or not the equation (3) is satisfied. If variable L = 1, equation (3) holds, and the process proceeds to step S108. In step S108, it is determined whether or not the variable L is 4 or more. Since the variable L = 1, it is determined that the variable L is not 4 or more, and the process proceeds to step S109. In step S109, the variable L is incremented by 1, and the variable L is set to L = 2.
[0090]
Thereafter, the process returns to step S107, and the variable L = 2 and n = 17 are substituted into the expression (3), and it is determined whether or not the expression (3) is satisfied. If variable L = 2, equation (3) holds, and the process proceeds to step S108. In step S108, it is determined whether or not the variable L is 4 or more. Since the variable L is 2, it is determined that the variable L is not 4 or more, and the process proceeds to step S109. In step S109, the variable L is incremented by one, and the variable L is set to three.
[0091]
Thereafter, the process returns to step S107, and the variable L = 3 and n = 17 are substituted into the equation (3), and it is determined whether or not the equation (3) is satisfied. If variable L = 3, equation (3) does not hold, and the process proceeds to step S110.
[0092]
In step S110, the hierarchy setting unit 123 discards a data packet of a layer higher than the (L + 1) th layer. In step S111, the hierarchy setting unit 123 sets the packets up to the lower L layers as data packets to be transmitted to the image receiving device 12.
[0093]
For example, when the variable L is set to L = 1 by the processing of steps S106 to S109, the hierarchy setting unit 123 determines in step S110 that the layers L2 to L4 of the four layers shown in FIG. The packets, that is, the packets P6 to P20 are discarded, and the packet of the layer L1 shown in FIG. 2, that is, the packets P1 to P5 is set as a data packet to be transmitted to the image receiving device 12 in step S111.
[0094]
Further, for example, when the variable L is set to L = 2 by the processing of Steps S106 to S109, the layer setting unit 123 determines in Step S110 that the layer L3 and the layer L3 of the four layers shown in FIG. The L4 packet, that is, the packets P11 to P20 are discarded, and in step S111, the packets of the layers L1 and L2 shown in FIG. 2, that is, the packets P1 to P10 are set as data packets to be transmitted to the image receiving device 12. .
[0095]
Further, for example, when the variable L is set to L = 3 by the processing of steps S106 to S109, the hierarchy setting unit 123 determines in step S110 that the layer L4 of the four layers shown in FIG. The packets, that is, the packets P16 to P20 are discarded, and the packets of the layers L1 to L3 shown in FIG. 2, that is, the packets P1 to P15 are set as the data packets to be transmitted to the image receiving device 12 in step S111.
[0096]
Further, for example, when the variable L is set to L = 4 by the processing of steps S106 to S109, the layer setting unit 123 does not discard the packet in step S110, and in FIG. The packets of the layers L1 to L4, that is, packets P1 to P20 are set as data packets to be transmitted to the image receiving device 12.
[0097]
After the process of step S111, the process returns to step S101, and the processes after step S101 are repeated.
[0098]
In this manner, the hierarchy of the hierarchically encoded data transmitted from the image transmission device 11 to the image reception device 12 is determined.
[0099]
The hierarchy setting unit 123 causes the packet generation unit 105 to select only data packets to be transmitted to the image receiving device 12 from among the data packets generated by the packet generation unit 105 based on the settings made by the above processing. Discard the data packet. After that, only the selected data packet is supplied from the packet generation unit 105 to the packet transmission unit 106.
[0100]
Next, the process of step S2 of FIG. 6, that is, the transmission process of transmitting a data packet from the packet transmitting unit 106 to the image receiving device 12 will be described with reference to the flowchart of FIG.
[0101]
The packet transmission unit 106 receives the supply of the data packet set as the packet to be transmitted from the packet generation unit 105, and sets the layer i of the data packet to be transmitted to i = 1 in step S151. Thereafter, the process proceeds to step S152.
[0102]
In step S152, the packet transmitting unit 106 records the transmission time in the packet header of the data packet of the layer i, and transmits the data packet to the image receiving device 12. Thereafter, the process proceeds to step S153.
[0103]
In step S153, the packet transmitting unit 106 increments the hierarchy i by one. Thereafter, the process proceeds to step S154.
[0104]
In step S154, the packet transmitting unit 106 determines whether or not the layer i is larger than the variable L set in steps S106 to S109 in the flowchart of FIG. If the hierarchy i is equal to or less than the variable L, the process returns to step S152, and the processes from step S152 are repeated. If it is determined in step S154 that the hierarchy i is larger than the variable L, the packet transmitting unit 106 ends the transmission of the data packet.
[0105]
For example, when the variable L is L = 3, first, in step S151, i = 1 is set. In step S152, the layer corresponding to i = 1, that is, the packets P1 to P5 of the layer L1 in FIG. The image is transmitted to the image receiving device 12. Then, in step S153, i is incremented by 1 and i = 2. Then, in step S154, it is determined whether or not i is greater than L. Since i = 2 and i is not greater than L, the process returns to step S152.
[0106]
Thereafter, in step S152, the layer corresponding to i = 2, that is, the packets P6 to P10 of the layer L2 in FIG. Then, in step S153, i is incremented by 1 and i is set to 3. Then, in step S154, it is determined whether or not i is greater than L. Since i = 3 and i is not greater than L, the process returns to step S152.
[0107]
After that, in step S152, the layer corresponding to i = 3, that is, the packets P11 to P15 of the layer L3 in FIG. Then, in step S153, i is incremented by 1 and i = 4 is set. Then, in step S154, it is determined whether or not i is larger than L. However, since i = 4 and i is larger than L, the process is terminated.
[0108]
As described above, the data packets are transmitted from the image transmitting device 11 to the image receiving device 12 for each layer.
[0109]
When the data packets are transmitted as described above, the number of data packets transmitted from the image transmitting device 11 to the image receiving device 12 is controlled, for example, as shown in FIG.
[0110]
In FIG. 10, as in FIG. 1, the vertical axis represents the window size, and the horizontal axis represents the time axis. In the graph of FIG. 10, five packets P1 to P5 corresponding to the layer L1 are set as data packets to be transmitted during the time t0 to t1, and 10 packets corresponding to the layers L1 and L2 during the time t1 to t2. Packets P1 to P10 are set as data packets to be transmitted, and from time t2 to time t3, fifteen packets P1 to P15 corresponding to layers L1 to L3 are set as data packets to be transmitted. , 10 packets P1 to P10 corresponding to the layers L1 and L2 are set as data packets to be transmitted, and between time t4 and t5, 15 packets P1 to P15 corresponding to the layers L1 to L3 transmit data. It is set as a packet and corresponds to layers L1 and L2 between time t5 and time t6. 10 packets P1 to P10 are set as data packets to be transmitted, and from time t6 to t7, 15 packets P1 to P15 corresponding to layers L1 to L3 are set as data packets to be transmitted, and after time t7. , 10 packets P1 to P10 corresponding to the layers L1 and L2 are set as data packets to be transmitted.
[0111]
FIG. 11 is a diagram in which the transition of the number of transmission packets when the present invention is applied shown in FIG. 10 is compared with the conventional example of FIG.
[0112]
In FIG. 11, the transition of the number of transmission packets in the conventional example of FIG. 1 is indicated by a dotted line. Further, the transition of the number of transmission packets when the present invention is applied shown in FIG. 10 is indicated by a solid line. In FIG. 11, packets in a range indicated by hatching are packets including data that cannot be expected to improve image quality even when transmitted from the image transmission device 11 to the image reception device 12 in the conventional example.
[0113]
By transmitting the data packets constituting each layer collectively from the image transmitting device 11 to the image receiving device 12, the image receiving device 12 uses all the data of the received data packets and It is possible to provide a high-quality image, and it is possible to eliminate unnecessary data transmitted from the image transmitting apparatus 11 to the image receiving apparatus 12.
[0114]
Also, in FIG. 11, during the time t0 to t1, in the conventional rate control, since the number of transmission packets is less than 5, the image receiving apparatus 12 reproduces an image even when receiving these data packets. I can't. In FIG. 11, the number of transmission packets is nine between times t2 and t3, between times t4 and t5, and between times t6 and t7. No. 12, four of the received data packets could not be used.
[0115]
On the other hand, in the present invention, since the number of transmission packets is set to 5 corresponding to one layer from time t0, the image receiving device 12 decodes the data immediately after starting the reception of the data packet. Images can be played back. In the present invention, unnecessary data packet transmission / reception can be eliminated by collectively increasing / decreasing packets in layer units.
[0116]
By the way, in FIG. 11, it can be said that the hatched portion indicates the margin of the number of packets that can be transmitted. Therefore, redundant data may be transmitted by using this margin.
[0117]
FIG. 12 is a flowchart illustrating a rate control process of the image transmitting apparatus 11 in a case where redundant data for error correction is transmitted using a margin of the number of transmittable packets.
[0118]
The processing of steps S201 to S211 in the flowchart of FIG. 12 is the same as the processing of steps S101 to S111 in FIG. In FIG. 12, after the processing in step S211, in step S212, the hierarchy setting unit 123 sets n- (L × 5) redundant data as packets to be transmitted. That is, the hierarchy setting unit 123 sets the error correction redundant data by the number of remainders obtained by subtracting the number of packets of the layer to be transmitted set in step S211 from the number of transmission packets n calculated in step S205. Set as a packet to be sent.
[0119]
Thereafter, the process returns to step S201, and the processes after step S201 are repeated.
[0120]
As described above, the redundant data for error correction is transmitted by the number equal to the number obtained by subtracting the number of packets of the transmission layer set in step S211 from the number n of transmission packets calculated in step S205. In this case, the hierarchy setting unit 123 instructs the packet generation unit 105 to create n− (L × 5) pieces of redundant data for error correction as packets to be transmitted. The packet generation unit 105 creates redundant data for error correction for n− (L × 5) as packets to be transmitted in accordance with a command from the hierarchy setting unit 123.
[0121]
As described above, by transmitting redundant data for error correction by the remaining number, the image receiving apparatus 12 can reduce the data of the lost packet from the redundant data for error correction even if the loss of the packet occurs. It may be possible to create data corresponding to. Therefore, the image receiving device 12 can reduce the possibility of having to request the image transmitting device 11 to retransmit a lost packet.
[0122]
By the way, when the network 10 provides priority control, the network is set by setting a priority, transmitting all data packets, and measuring how many low priority data packets have arrived at the image receiving device 12. It is possible to estimate how much bandwidth is available in 10.
[0123]
For example, when the ideal number n of transmission packets is n = 12, the image transmission device 11 transmits ten data packets (packets P1 to P10) of the layers L1 and L2 with high priority, and transmits the layer L3 And the L4 data packet with low priority. Then, based on the response packet from the image receiving device 12, the number of data packets reaching the image receiving device 12 is specified. For example, when seven data packets have arrived at the image receiving device 12, the image transmitting device 11 determines that there is enough bandwidth to transmit the remaining seven data packets, and the subsequent ideal data packets are transmitted. The value of the number of transmission packets n is set as n = 10 + 7 = 17. This may be done.
[0124]
Next, the rate control process of the image transmitting apparatus 11 in the case of preferentially transmitting data of a layer having a higher priority will be described with reference to the flowchart of FIG.
[0125]
The processing in steps S231 to S239 of the flowchart in FIG. 13 is the same as the processing in steps S101 to S109 in FIG. In FIG. 13, after the process of step S238, in step S240, the hierarchy setting unit 123 sets the data packets of the layers up to the lower L layers as packets to be transmitted to the image receiving device 12 with high priority. For example, when the variable L = 2, the hierarchy setting unit 123 sets the data packets of the layers L1 and L2 in FIG. 2 as high-priority packets.
[0126]
After that, in step S241, the hierarchy setting unit 123 sets the data packets of the (L + 1) or higher layers as packets to be transmitted to the image receiving device 12 with low priority. For example, when the variable L = 2, the hierarchy setting unit 123 sets the data packets of the layers L3 and L4 in FIG. 2 as low-priority packets.
[0127]
After the processing in step S241, in step S242, the hierarchy setting unit 123 sets the number n of transmission packets based on the response packet. That is, since the response packet contains the lost packet ID, it is possible to determine how many data packets have reached the image receiving device 12 based on the lost packet ID. Therefore, the hierarchy setting unit 123 calculates the number of data packets that have reached the image receiving device 12 among the data packets transmitted with low priority based on the lost packet ID, and sets the calculated value to the currently set value. The new transmission packet number n is calculated by adding the current transmission packet number n.
[0128]
After that, the process returns to step S236, and the processes after step S236 are repeated.
[0129]
As described above, data of a layer having a higher priority may be transmitted with priority.
[0130]
In the process of the flowchart of FIG. 13, the free space of the band of the network 10 can be estimated, but even if an extra data packet reaches the image receiving device 12 for one layer, Not all packets constituting a certain layer are necessarily prepared. Therefore, only the data packet of the layer one layer higher than the layer currently transmitting with high priority may be transmitted with low priority. Next, the rate control processing of the image transmitting apparatus 11 in this case will be described with reference to the flowchart of FIG.
[0131]
The processes in steps S261 to S270 and step S272 in the flowchart in FIG. 14 are the same as the processes in steps S231 to S240 and step S242 in FIG. In FIG. 14, after the process of step S270, in step S271, the hierarchy setting unit 123 sets the data packet of the (L + 1) th layer as a packet to be transmitted to the image receiving device 12 with low priority. For example, when the variable L = 2, the hierarchy setting unit 123 sets the data packet of the layer L3 in FIG. 2 as a low-priority packet. Thereafter, the process proceeds to step S272.
[0132]
By doing as described above, for example, as a result of transmitting the packets P1 to P5 constituting the layer L1 with high priority and transmitting the packets P6 to P10 constituting the layer L2 with low priority, the layers L1 and L2 are When all the constituent packets P1 to P10 reach the image receiving device 12, thereafter, the packets P1 to P10 forming the layers L1 and L2 are transmitted with high priority, and the packets P11 to P15 forming the layer L3 are transmitted. Is transmitted at a low priority, and as a result, it is possible to transmit more hierarchically encoded data. Therefore, when the external device acquires and reproduces the hierarchically encoded data from the image receiving device 12, it is possible to display a higher definition image.
[0133]
As described above, according to the present invention, when the number of packets that can be transmitted per unit time increases, the number of packets that can be transmitted increases until the number of packets that can be transmitted increases by the number of layers (layers). When the number of packets that can be transmitted per unit time decreases and the number of packets that can be transmitted per unit time is reduced, the number of packets is reduced by the number of layers (layers). Therefore, unnecessary data packets to be transmitted / received can be eliminated, and data can be transmitted / received more efficiently. Further, the image receiving apparatus 12 can also omit the process of selecting an unusable data packet from the received data packets.
[0134]
In the above description, as shown in FIG. 2, the case of hierarchically encoded data in which four layers are included and one layer is divided into five packets is taken as an example. If the data is coded data, the number of layers and the number of packets per layer need not be the numbers shown in FIG.
[0135]
The present invention can also be applied to real-time streaming.
[0136]
The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processing is executed by software, a program constituting the software may be executed by a computer built into dedicated hardware or by installing various programs to execute various functions. It is installed from a recording medium in a possible personal computer or the like, for example.
[0137]
FIG. 15 shows a configuration of an embodiment of an information processing apparatus when the image transmitting apparatus 11 or the image receiving apparatus 12 is realized by software. The CPU 401 of the personal computer 400 controls the entire operation of the personal computer 400. When a user inputs a command from the input unit 406 including a keyboard, a mouse, and the like via the bus 404 and the input / output interface 405, the CPU 401 stores the command in a ROM (Read Only Memory) 402. Execute the program. Alternatively, the CPU 401 reads a program read from the magnetic disk 421, the optical disk 422, the magneto-optical disk 423 or the semiconductor memory 424 connected to the drive 410 and installed in the storage unit 408 into a RAM (Random Access Memory) 403. And run it. Thus, the functions of the image transmitting device 11 or the image receiving device 12 described above are implemented by software. Further, the CPU 401 controls the communication unit 409 to communicate with the outside and exchange data. The input / output interface 405 is also connected to an output unit 407 including a display, a speaker, and the like.
[0138]
As shown in FIG. 15, the recording medium on which the program is recorded is a magnetic disk 421 (including a flexible disk) on which the program is recorded, which is distributed separately from the computer in order to provide the user with the program. An optical disk 422 (including a compact disk-read only memory (CD-ROM), a digital versatile disk (DVD)), a magneto-optical disk 423 (including a mini-disc (MD)), or a package medium including a semiconductor memory 424 or the like In addition to the configuration, the storage unit 408 includes a ROM 402 in which a program is recorded and a hard disk included in the storage unit 408, which are provided to a user in a state of being incorporated in a computer in advance.
[0139]
In this specification, a step of describing a program recorded on a recording medium is performed in a time-series manner in the order described. Alternatively, the processing includes individually executed processing.
[0140]
Also, in this specification, a system refers to an entire device including a plurality of devices.
[0141]
【The invention's effect】
As described above, according to the first aspect of the present invention, information can be transmitted and received via a network. Further, according to the first aspect of the present invention, information can be transmitted and received more efficiently. Further, according to the first aspect of the present invention, it is possible to perform error correction using a free area of a network band. Further, according to the first aspect of the present invention, it is possible to predict the free space in the network bandwidth.
[0142]
According to the second aspect of the present invention, information can be transmitted via a network. According to the second aspect of the present invention, information can be transmitted more efficiently. Further, according to the second aspect of the present invention, it is possible to perform error correction using a free area of a network band. Further, according to the second aspect of the present invention, it is possible to predict the free space in the network bandwidth.
[Brief description of the drawings]
FIG. 1 is a diagram for explaining conventional control of the number of transmission packets.
FIG. 2 is a diagram illustrating an example of a packet for one frame encoded by a JPEG2000 encoding method.
FIG. 3 is a block diagram illustrating a configuration example of an information processing system to which the present invention has been applied.
FIG. 4 is a block diagram illustrating a configuration example of an image transmission device.
FIG. 5 is a block diagram illustrating a configuration example of an image receiving device.
FIG. 6 is a flowchart illustrating an image transmission process of the image transmission device and an image reception process of the image reception device.
FIG. 7 is a flowchart illustrating a confirmation response process of the image receiving apparatus.
FIG. 8 is a flowchart illustrating a rate control process of the image transmission device.
FIG. 9 is a flowchart illustrating Step S2 of FIG. 6 in detail.
FIG. 10 is a diagram illustrating control of the number of transmission packets in the present invention.
FIG. 11 is a diagram comparing control of the number of transmission packets in the present invention with control of the number of transmission packets in the related art.
FIG. 12 is another flowchart illustrating a rate control process of the image transmission device.
FIG. 13 is still another flowchart illustrating the rate control process of the image transmitting apparatus.
FIG. 14 is a flowchart illustrating a rate control process of the image transmission device.
FIG. 15 is a block diagram illustrating a configuration example of an embodiment of a computer to which the present invention has been applied.
[Explanation of symbols]
Reference Signs List 10 network, 11 image transmission device, 12 image reception device, 101 operation unit, 102 control unit, 103 recording medium, 104 input unit, 105 packet generation unit, 106 packet transmission unit, 107 packet reception unit, 108 rate control unit, 121 Transmission rate calculation unit, 122 layer setting unit, 151 operation unit, 152 control unit, 153 packet reception unit, 154 recording medium, 155 state measurement unit, 156 packet generation unit, 157 packet transmission unit, 401 CPU, 402 ROM, 403 RAM , 404 bus, 405 input / output interface, 406 input unit, 407 output unit, 408 storage unit, 409 communication unit, 410 drive, 421 magnetic disk, 422 optical disk, 423 magneto-optical disk, 424 semiconductor memory

Claims (8)

In an information processing system including a first information processing device and a second information processing device for transmitting and receiving hierarchically encoded data that is hierarchically encoded data by packet communication,
The first information processing device includes:
Generating means for packetizing the hierarchically encoded data to generate a first packet;
First transmitting means for transmitting the first packet generated by the generating means to the second information processing device;
First receiving means for receiving, from the second information processing device, a second packet including information on a transmission time of the first packet by the first information processing device and a loss of the first packet; When,
Calculating means for calculating a reference value of the number of packets of the first packet to be transmitted based on the second packet received by the first receiving means;
Setting means for setting a layer of the layer coded data to be packetized by the generation means, based on the reference value calculated by the calculation means,
The second information processing device includes:
Second receiving means for receiving the first packet from the first information processing device;
Acquiring means for acquiring the hierarchically encoded data from the first packet received by the second receiving means;
Creating means for creating the second packet based on the first packet received by the second receiving means;
An information processing system comprising: a second transmission unit that transmits the second packet created by the creation unit to the first information processing device. In an information processing apparatus that transmits hierarchically encoded data, which is hierarchically encoded data, to another information processing apparatus by packet communication,
Generating means for packetizing the hierarchically encoded data to generate a first packet;
Transmitting means for transmitting the first packet generated by the generating means to the other information processing device;
Receiving means for receiving, from the other information processing apparatus, a transmission time of the first packet by the information processing apparatus, and a second packet including information on the loss of the first packet;
Calculating means for calculating a first number, which is a reference value of the number of packets of the first packet to be transmitted, based on the second packet received by the receiving means;
An information processing apparatus comprising: a setting unit configured to set a layer of the layer encoded data to be packetized by the generation unit based on the first number calculated by the calculation unit. The setting unit sets a hierarchy of the hierarchical data to be packetized by the generation unit such that the number of packets of the first packet generated by the generation unit is equal to or less than the first number. The information processing apparatus according to claim 2, wherein: The setting unit, when packetizing the layer encoded data of the layer set as the layer to be packetized, the second number, which is the number of packets of the first packet generated by the generating unit, is Setting the hierarchy to be packetized so as to be equal to or less than a first number, and calculating a third number that is a remainder obtained by subtracting the second number from the first number;
The information processing apparatus according to claim 2, wherein the generation unit generates the first packet by further packetizing redundant data by the third number. The setting means, when packetizing the hierarchically encoded data of the layer to be transmitted with priority, sets the second number, which is the number of packets of the first packet generated by the generating means, to the first number And setting the hierarchy to be transmitted preferentially so as to be less than or equal to the number, and calculating a third number that is a remainder obtained by subtracting the second number from the first number,
The generating means packetizes the layer-encoded data of the layer to be transmitted with priority, generates the first packet, and performs the layer coding of the layer not to be transmitted by priority by the third number. The information processing apparatus according to claim 2, wherein the first packet is generated by packetizing data. In an information processing method of an information processing apparatus for transmitting hierarchically encoded data, which is hierarchically encoded data, to another information processing apparatus by packet communication,
A generation step of packetizing the hierarchically encoded data to generate a first packet;
A transmitting step of transmitting the first packet generated by the processing of the generating step to the other information processing apparatus;
A receiving step of receiving, from the other information processing apparatus, a transmission time of the first packet by the information processing apparatus, and a second packet including information on the loss of the first packet;
A calculating step of calculating a first number, which is a reference value of the number of packets of the first packet to be transmitted, based on the second packet received by the processing of the receiving step;
A setting step of setting a layer of the layer coded data to be packetized by the processing of the generation step based on the first number calculated by the processing of the calculation step. . A program for an information processing device that transmits hierarchically encoded data, which is hierarchically encoded data, to another information processing device by packet communication,
Generating a first packet by packetizing the hierarchically encoded data for transmission to the other information processing apparatus;
When the second packet including the transmission time of the first packet by the information processing device and the information on the loss of the first packet is received from the another information processing device, the received second packet A calculating step of calculating a reference value of the number of packets of the first packet to be transmitted based on a packet;
A setting step of setting a layer of the layer coded data to be packetized by the processing of the generation step based on the reference value calculated by the processing of the calculation step. A recording medium on which a program is recorded. By packet communication, to a computer that controls an information processing device that transmits hierarchically encoded data that is hierarchically encoded data to another information processing device,
Generating a first packet by packetizing the hierarchically encoded data for transmission to the other information processing apparatus;
When the second packet including the transmission time of the first packet by the information processing device and the information on the loss of the first packet is received from the another information processing device, the received second packet A calculating step of calculating a reference value of the number of packets of the first packet to be transmitted based on a packet;
A setting step of setting a layer of the layer encoded data to be packetized by the processing of the generation step, based on the reference value calculated by the processing of the calculation step.

Images