JP4875285B2 - Editing apparatus and method - Google Patents

Description

【００７５】
ランダムアクセスポイント間の（Ｒ_min、Ｂ_min、Ｆ_min）は、例えば、その直前のランダムアクセスポイントヘッダに、上記したシンタクスのように記録される。RAP_startcodeは、ＲＡＰヘッダが存在し、そのヘッダの開始を示すコードである。
【００７６】
closed_GOPは、そのＧＯＰ内の全てのピクチャが他のＧＯＰのピクチャを参照することがなく独立であるか、または、他のＧＯＰのピクチャを参照するという依存関係があるかどうかを示すフラグである。broken_linkは、編集などにより、そのＧＯＰの前後でビットストリームの置き換えが行われた場合、予測の参照画像が存在するか否かを示すフラグである。
【００７７】
NumBuffer_Paramは、求めた特性セット（Ｒ_min、Ｂ_min、Ｆ_min）の数を示す。
Rate[i]、Buffer[i]、F[i]は、それぞれをＲ_min、Ｂ_min、Ｆ_min示す。ここでは、例えば、Ｒ_minは、小さいものから順に記録される。
【００７８】
ビットストリーム全体の（Ｒ_min、Ｂ_min、Ｆ_min）globalは、例えば、そのビットストリームの先頭のシーケンスヘッダに、以下のシンタクスのようにして記録される。
【００７９】

【００８０】
ここで、NumBuffer_Paramは、求めた特性セット（Ｒ_min、Ｂ_min、Ｆ_min）globalの数を示す。Rate[i]、Buffer[i]、F[i]は、それぞれをＲ_min、Ｂ_min、Ｆ_minを示す。ここでは、例えば、Ｒ_minは、小さいものから順に記録される。
【００８１】
バッファ情報付加部７３において、上記のバッファ情報ＢＨが付加された後、最終的な出力ビットストリームＢＳが出力される。
【００８２】
なお、発明の実施の形態ではバッファ情報ＢＨとして、最小ビットレートRmin、最小バッファサイズBminおよび最小遅延量Fminの全てをビットストリームに付加するように説明した。しかし、この例に限らず、最小ビットレートRmin、最小バッファサイズBmin若しくは最小遅延量Fminのうち、少なくとも一つをビットストリームに加えるようにしてもよい。例えば、最小ビットレートRminおよび最小バッファサイズBminの組み合わせをビットストリームに付加するようにしてもよい。
【００８３】
図７に本発明を適用した復号装置の一実施の形態の構成を示す。図７に示した復号装置９０は、図５に示した符号化装置７０に対応するものであり、内部に、図２に示した画像情報復号装置４０を含んでいる。復号装置９０に入力されたビットストリームＢＳは、ビットストリーム解析部９１と復号可能性判定部９２に供給される。
【００８４】
ビットストリーム解析部９１は、ビットストリーム中のバッファ情報ＢＨを復号し、復号可能性判定部９２に出力する。ビットストリーム解析部９１は、ビットストリームをパースし、シーケンスヘッダに記録されている、（Ｒ_min、Ｂ_min、Ｆ_min）globalを復号する。また、各ランダムアクセスポイントヘッダに記録されている、（Ｒ_min、Ｂ_min、Ｆ_min）を復号する。これら情報が復号可能性判定部９２に出力される。
【００８５】
復号可能性判定部９２は、バッファ情報ＢＨおよび画像情報復号装置４０より供給されるデコーダ情報ＤＩに基づいて、入力されたビットストリームがバッファを破綻させること無く復号可能であるかどうかを判定する。デコーダ情報ＤＩは、例えば、デコーダバッファサイズおよび復号ビットレートなどである。
【００８６】
復号可能性判定部９２は、（Ｒ_min、Ｂ_min、Ｆ_min）globalから、図４に示したような特性曲線を作成する。各点の間は線形補間する。この時、デコーダ（復号装置９０）のバッファおよび復号ビットレートが（Ｒ_min、Ｂ_min、Ｆ_min）globalにより作られる特性曲線より上に位置する場合、入力されたビットストリームは、復号可能であると判断することが可能である。従ってこのようなとき、復号可能性判定部９２は、復号可能であると判定し、ビットストリームを画像情報復号装置４０に供給する。
【００８７】
画像情報復号装置４０は、図２に示した画像情報復号装置４０と基本的に同様な構成により、同様な処理を実行し、入力されたビットストリームを復号し、画像情報を図示されていないテレビジョン受像機などに出力する。
【００８８】
ビットストリーム全体を復号可能であるかどうかは上記のように、（Ｒ_min、Ｂ_min、Ｆ_min）globalの特性曲線、デコーダバッファサイズ、復号ビットレートを調べることによって判定することが可能である。
【００８９】
また、ランダムアクセスなどにより、所定のランダムアクセスポイントから特定の区間のみを復号したい場合、同様にして、復号可能性判定部９２は、（Ｒ_min、Ｂ_min、Ｆ_min）から図４に示すような特性曲線を作成する。各点の間は線形補間する。この時、デコーダのバッファおよび復号ビットレートが（Ｒ_min、Ｂ_min、Ｆ_min）により作られる特性曲線より上に位置する場合、ビットストリームは復号可能である。従ってこのようなとき、復号可能性判定部９２は、復号可能であると判定し、ビットストリームを画像情報復号装置４０に供給する。
【００９０】
次にビットストリームの編集を行う際の説明を行う。図８は、本発明を適用したビットストリームの編集を行う編集装置１１０の一実施の形態の構成を示す図である。編集装置１１０が行う編集の例として、入力ビットストリーム１の一部を、別の入力ビットストリーム２に置き換えるスプライスを行う場合を例に挙げて説明する。
【００９１】
ここで、スプライスについて簡単に説明するに、スプライスとは、所定のビットストリームをランダムアクセスポイントにおいて別のビットストリームに置き換えて編集を行うことである。このようなスプライスは、例えば、テレビジョン放送の番組に、コマーシャルの放送を挿入する際などである。この場合、入力ビットストリーム１がテレビジョン放送の番組のビットストリームであり、入力ビットストリーム２がコマーシャルのビットストリームである。
【００９２】
入力ビットストリーム１は、ビットストリーム解析部１１１−１に入力され、入力ビットストリーム２は、ビットストリーム解析部１１１−２に入力される。ビットストリーム解析部１１１−１，１１１−２は、それぞれ入力されたビットストリーム１，２中に含まれているバッファ情報ＢＨ１，２を復号し、ビットストリーム編集部１１２に出力する。
【００９３】
ビットストリーム編集部１１２は、バッファ情報ＢＨ１，２に基づき、所定の編集ポイントで、入力ビットストリーム１に対して入力ビットストリーム２を挿入可能であるか否かを判定する。この時、編集後のビットストリームが、デコーダ（復号装置９０）のバッファを破綻させずに復号可能であるためには、ランダムアクセスポイントとその直前のバッファ占有量の値が同一であるという条件が必要である。
【００９４】
ＭＰＥＧ２，４方式を用いるデコーダは、特定のビットレート、バッファサイズで動作することが想定されていたが、ＪＶＴ方式を用いるデコーダにおいては、図４に示すように、その他のビットレート、バッファサイズであっても、（Ｒ_min、Ｂ_min、Ｆ_min）の特性曲線より上にある場合、復号することが可能であるようにバッファに対する制約が緩和されている。
【００９５】
ビットストリームの編集により、その編集前後でデコード可能性が変化しないようにするためには、編集区間の（Ｒ_min、Ｂ_min、Ｆ_min）が同一であれば良い。従って、ビットストリーム編集部１１２は、編集区間に位置するランダムアクセスポイントヘッダにおける（Ｒ_min、Ｂ_min、Ｆ_min）特性を、入力ビットストリーム１，２に対して作成し、これらの値が一致する場合、その区間をビットストリーム２に置換する。一致しない場合、ビットストリーム１または２に対してパディングビットを挿入して、（Ｒ_min、Ｂ_min、Ｆ_min）が一致するようにした後、入力ビットストリーム２に置換する。
【００９６】
ＪＶＴにおいては、バッファに対する規制が緩和されているが、このことを利用すれば、スプライスにおけるバッファの適合条件を緩和することが可能になる。ＪＶＴにおいては、デコーダのバッファサイズおよび復号ビットレートが（Ｒ_min、Ｂ_min、Ｆ_min）の上に位置する場合、復号可能であることがわかる。従って、元の入力ビットストリーム１の所定の編集区間の（Ｒ_min、Ｂ_min、Ｆ_min）に対して、挿入する入力ビットストリーム２の所定編集区間の（Ｒ_min、Ｂ_min、Ｆ_min）が常に下にある場合、入力ビットストリーム１を復号可能なデコーダは、その区間をビットストリーム２に置換しても復号可能であることになる。
【００９７】
図９にその関係を図示する。曲線１は、入力ビットストリーム１の編集区間での（Ｒ_min、Ｂ_min、Ｆ_min）特性を示す。曲線２は入力ビットストリーム２の編集区間での（Ｒ_min、Ｂ_min、Ｆ_min）特性を示す。デコーダのバッファ、復号ビットレートが、この曲線の上に来る場合、復号可能であることから、図９に示すように曲線２が常に曲線１の下に来るとき、復号可能であることが保証される。
【００９８】
従って、ビットストリーム編集部１１２は、編集区間に位置するランダムアクセスポイントヘッダにおける（Ｒ_min、Ｂ_min、Ｆ_min）特性を、ビットストリーム１，２に対して作成し、ビットストリーム２の特性曲線が、ビットストリーム１の特性曲線の下に来る場合、その区間をビットストリーム２に置換する。
【００９９】
逆に、一致しないような場合、ビットストリーム１または２に対してパディングビットを挿入して、ビットストリーム２の（Ｒ_min、Ｂ_min、Ｆ_min）特性曲線が、ビットストリーム１の特性曲線の下に位置するように変更した後、入力ビットストリーム２に置換する。
【０１００】
このような条件を満たすようにスプライスを行った場合、ビットストリーム１を復号可能なデコーダを破綻させることはない。ビットストリーム編集部１１２はスプライスをした後、最終的なビットストリームを出力する。
【０１０１】
このように、ビットストリーム中のランダムアクセスが行えるポイントのヘッダに、（Ｒ_min、Ｂ_min、Ｆ_min）といった最小ビットレート、最小バッファサイズ、最小初期遅延時間などの情報を含ませることにより、復号側において、ビットストリームの復号可能性を効率良く判断することが可能となり、また、スプライシングなどのビットストリームの編集を容易に、かつ、復号側のバッファを破綻させることなく復号が常に行えるようにすることが可能となる。
【０１０２】
図１０は、汎用のパーソナルコンピュータの内部構成例を示す図である。パーソナルコンピュータのＣＰＵ（Central Processing Unit）２１１は、ＲＯＭ（Read Only Memory）２１２に記憶されているプログラムに従って各種の処理を実行する。ＲＡＭ（Random Access Memory）２１３には、ＣＰＵ２１１が各種の処理を実行する上において必要なデータやプログラムなどが適宜記憶される。入出力インタフェース２１５は、キーボードやマウスから構成される入力部２１６が接続され、入力部２１６に入力された信号をＣＰＵ２１１に出力する。また、入出力インタフェース２１５には、ディスプレイやスピーカなどから構成される出力部７も接続されている。
【０１０３】
さらに、入出力インタフェース２１５には、ハードディスクなどから構成される記憶部２１８、および、インターネットなどのネットワークを介して他の装置とデータの授受を行う通信部２１９も接続されている。ドライブ２２０は、磁気ディスク２３１、光ディスク２３２、光磁気ディスク２３３、半導体メモリ２３４などの記録媒体からデータを読み出したり、データを書き込んだりするときに用いられる。
【０１０４】
記録媒体は、図１０に示すように、パーソナルコンピュータとは別に、ユーザにプログラムを提供するために配布される、プログラムが記録されている磁気ディスク２３１（フレキシブルディスクを含む）、光ディスク２３２（CD-ROM（Compact Disc-Read Only Memory），DVD（Digital Versatile Disc）を含む）、光磁気ディスク２３３（MD（Mini-Disc）（登録商標）を含む）、若しくは半導体メモリ２３４などよりなるパッケージメディアにより構成されるだけでなく、コンピュータに予め組み込まれた状態でユーザに提供される、プログラムが記憶されているＲＯＭ２１２や記憶部２１８が含まれるハードディスクなどで構成される。
【０１０５】
なお、本明細書において、媒体により提供されるプログラムを記述するステップは、記載された順序に従って、時系列的に行われる処理は勿論、必ずしも時系列的に処理されなくとも、並列的あるいは個別に実行される処理をも含むものである。
【０１０６】
また、本明細書において、システムとは、複数の装置により構成される装置全体を表すものである。
【０１０９】
【発明の効果】
本発明の編集装置および方法によれば、スプライスなどの編集にかかる処理を軽減させ、容易に編集可能であるか否かを判断することが可能となる。
【図面の簡単な説明】
【図１】従来の画像情報符号化装置の一例の構成を示す図である。
【図２】従来の画像情報復号装置の一例の構成を示す図である。
【図３】バッファ量について説明する図である。
【図４】ビットレートとバッファ量の関係について説明する図である。
【図５】本発明を適用した符号化装置の一実施の形態の構成を示す図である。
【図６】バッファ量について説明する図である。
【図７】本発明を適用した復号装置の一実施の形態の構成を示す図である。
【図８】本発明を適用した編集装置の一実施の形態の構成を示す図である。
【図９】ビットレートとバッファ量の関係について説明する図である。
【図１０】媒体を説明する図である。
【符号の説明】
７０ 符号化装置， ７１ バッファ， ７２ ビットストリーム解析部， ７３ バッファ情報付加部， ９０ 復号装置， ９１ ビットストリーム解析部， ９２ 復号可能性判定部， １１１ ビットストリーム解析部， １１２ビットストリーム編集部[0001]
BACKGROUND OF THE INVENTION
  The present inventionEditing apparatus and methodIn particular, when transmitting / receiving image information (bitstream) compressed by orthogonal transform and motion compensation such as discrete cosine transform or Karhunen-Labe transform via network media such as satellite broadcasting, cable television broadcasting, and the Internet. Or suitable for processing on a storage medium such as an optical disk, magnetic disk, or flash memory.Editing apparatus and methodAbout.
[0002]
[Prior art]
In recent years, image information is handled as digital data, and MPEG (compressed by orthogonal transform such as discrete cosine transform and motion compensation is used for the purpose of efficient transmission and storage of information. A device conforming to a scheme such as Moving Picture Expert Group) is becoming popular for both information distribution in broadcasting stations and information reception in general households.
[0003]
In particular, MPEG2 (ISO / IEC 13818-2) is a standard defined as a general-purpose image compression method, which covers both interlaced scanning images and progressive scanning images, as well as standard resolution images and high-definition images. As represented by the DVD (Digital Versatile Disk) standard, it is widely used in a wide range of applications for professional use and consumer use.
[0004]
By using this MPEG2 compression method, for example, 4 to 8 Mbps for a standard resolution interlaced scanning image having 720 × 480 pixels, and 18 to 22 Mbps for a high resolution interlaced scanning image having 1920 × 1088 pixels, for example. By assigning a code amount (bit rate), it is possible to realize a high compression rate and good image quality.
[0005]
MPEG2 was mainly intended for high-quality encoding suitable for broadcasting, but since it did not support encoding systems with higher compression rates, the MPEG4 encoding standard was standardized. Regarding the image coding system, the standard was approved as an international standard as ISO / IEC 14496-2 in December 1998.
[0006]
Furthermore, in recent years, H.26L (ITU-T Q6 / 16) by the ITU-T (International Telecommunication Union-Telecommunication Standardization Sector), which is the telecommunications standardization department of the International Telecommunication Union, was originally intended for video coding for video conferencing. VCEG) is being standardized. H. It is known that H.26L achieves higher encoding efficiency compared to encoding methods such as MPEG2 and MPEG4, although a larger amount of calculation is required for encoding and decoding.
[0007]
In addition, as part of MPEG4 activities, this H.264 The standardization of coding technology that realizes higher coding efficiency based on H.26L is being carried out jointly with ITU-T as JVT (Joint Video Team).
[0008]
Here, image compression by orthogonal transform such as discrete cosine transform or Karhunen-Loeve transform and motion compensation will be described. FIG. 1 is a diagram illustrating a configuration of an example of a conventional image information encoding device.
[0009]
In the image information encoding apparatus 10 shown in FIG. 1, image information composed of an analog signal input from the input terminal 11 is converted into a digital signal by the A / D conversion unit 12. The screen rearrangement buffer 13 rearranges the frames according to the GOP (Group of Pictures) structure of the image information supplied from the A / D conversion unit 12.
[0010]
Here, the screen rearrangement buffer 13 supplies image information of the entire frame to the orthogonal transform unit 15 for an image on which intra (intra-image) encoding is performed. The orthogonal transform unit 15 performs orthogonal transform such as discrete cosine transform or Karhunen-Loeve transform on the image information, and supplies transform coefficients to the quantization unit 16. The quantization unit 16 performs a quantization process on the transform coefficient supplied from the orthogonal transform unit 15.
[0011]
The lossless encoding unit 17 determines an encoding mode from the quantized transform coefficient, quantization scale, and the like supplied from the quantization unit 16, and performs variable length encoding or arithmetic encoding on the encoding mode. The information to be inserted into the header portion of the image coding unit is formed. Then, the lossless encoding unit 17 supplies the encoded encoding mode to the accumulation buffer 18 for accumulation. The encoded encoding mode is output from the output terminal 19 as image compression information.
[0012]
The lossless encoding unit 17 performs lossless encoding such as variable length encoding or arithmetic encoding on the quantized transform coefficient, and supplies the encoded transform coefficient to the accumulation buffer 18 for accumulation. Let The encoded transform coefficient is output from the output terminal 19 as image compression information.
[0013]
The behavior of the quantization unit 16 is controlled by the rate control unit 20 based on the data amount of the transform coefficient accumulated in the accumulation buffer 18. Further, the quantization unit 20 supplies the quantized transform coefficient to the inverse quantization unit 21, and the inverse quantization unit 21 inversely quantizes the quantized transform coefficient. The inverse orthogonal transform unit 22 performs inverse orthogonal transform processing on the inversely quantized transform coefficients to generate decoded image information, and supplies the information to the frame memory 23 for accumulation.
[0014]
In addition, the screen rearrangement buffer 13 supplies image information to the motion prediction / compensation unit 24 regarding an image on which inter (inter-image) encoding is performed. The motion prediction / compensation unit 24 extracts image information that is referred to at the same time from the frame memory 23, and performs motion prediction / compensation processing to generate reference image information. The motion prediction / compensation unit 24 supplies the generated reference image information to the adder 14, and the adder 14 converts the reference image information into a difference signal from the corresponding image information. In addition, the motion prediction / compensation unit 24 supplies motion vector information to the lossless encoding unit 17 at the same time.
[0015]
The lossless encoding unit 17 determines the encoding mode from the quantized transform coefficient and quantization scale supplied from the quantization unit 16, the motion vector information supplied from the motion prediction / compensation unit 24, and the like. The coding mode is subjected to lossless coding such as variable length coding or arithmetic coding, and information to be inserted into the header portion of the image coding unit is generated. Then, the lossless encoding unit 17 supplies the encoded encoding mode to the accumulation buffer 18 for accumulation. The encoded encoding mode is output as image compression information.
[0016]
Further, the lossless encoding unit 17 performs lossless encoding processing such as variable length encoding or arithmetic encoding on the motion vector information, and generates information to be inserted into the header portion of the image encoding unit.
[0017]
In contrast to intra coding, in the case of inter coding, the image information input to the orthogonal transform unit 15 is a difference signal obtained from the adder 14. The other processing is the same as the image compression information subjected to intra coding, and therefore the description thereof is omitted.
[0018]
Next, FIG. 2 shows a configuration of an example of an image information decoding device corresponding to the image information encoding device 10 described above. In the image information decoding apparatus 40 shown in FIG. 2, the image compression information input from the input terminal 41 is temporarily stored in the accumulation buffer 42 and then transferred to the lossless decoding unit 43.
[0019]
The lossless decoding unit 43 performs processing such as variable length decoding or arithmetic decoding on the compressed image information based on the determined format of the compressed image information, acquires the encoding mode information stored in the header portion, and performs inverse quantum To the control unit 44 and the like. Similarly, the lossless decoding unit 43 acquires the quantized transform coefficient and supplies it to the inverse quantization unit 44. Furthermore, when the frame to be decoded is inter-coded, the lossless decoding unit 43 also decodes the motion vector information stored in the header portion of the image compression information, and the information is motion prediction / compensation unit. 51.
[0020]
The inverse quantization unit 44 inversely quantizes the quantized transform coefficient supplied from the lossless decoding unit 43 and supplies the transform coefficient to the inverse orthogonal transform unit 45. The inverse orthogonal transform unit 45 performs inverse orthogonal transform such as inverse discrete cosine transform or inverse Karhunen-Labe transform on the transform coefficient based on the determined format of the image compression information.
[0021]
Here, when the target frame is intra-coded, the image information subjected to the inverse orthogonal transform process is stored in the screen rearrangement buffer 47, and the D / A conversion in the D / A conversion unit 48 is performed. It is output from the output terminal 49 after processing.
[0022]
When the target frame is inter-coded, the motion prediction / compensation unit 51 refers to the motion vector information subjected to the lossless decoding process and the image information stored in the frame memory 50. An image is generated and supplied to the adder 46. The adder 46 combines the reference image and the output from the inverse orthogonal transform unit 45. The other processing is the same as that of the intra-encoded frame, and thus description thereof is omitted.
[0023]
By the way, in the encoding method (hereinafter referred to as JVT Codec) standardized by the Joint Video Team described above, various methods are being studied in order to improve the encoding efficiency of MPEG2 and MPEG4. For example, the conversion method of the discrete cosine transform uses integer coefficient conversion of 4 × 4 block size. The block size at the time of motion compensation is variable, and more optimal motion compensation can be performed. However, the basic method can be performed in the same manner as the encoding method performed in the image information encoding device 10 shown in FIG.
[0024]
Therefore, it is possible to perform decoding by basically the same method as the decoding method performed in the image information decoding device 40 shown in FIG.
[0025]
By the way, in order to maintain compatibility between different decoding apparatuses (decoders) and prevent the buffer from overflowing or underflowing, a buffer model is introduced in MPEG and ITU-T. A virtual decoder buffer model is defined as a standard, and the encoding device (encoder) encodes the virtual decoder buffer so that it does not fail, thereby preventing buffer overflow or underflow on the decoder side and maintaining compatibility. Is possible.
[0026]
A virtual buffer model in MPEG will be described with reference to FIG. In the following description, the input bit rate to the decoder buffer is R, the size of the decoder buffer is B, the buffer occupation amount when the decoder pulls out the first frame from the buffer is F, and the delay time at that time is D. Further, the bit amount of each frame at times t0, t1, t2,... Is b0, b1, b2,.
[0027]
If the frame rate is M,
t_{i + 1}-T_i= 1 / M holds.
[0028]
B_iAt time t_iBit amount b of frame_iThe following equation (1) is established if the buffer occupancy immediately before is extracted.
B₀= F
B_{i + 1}= Min (B, B_i―B_i+ R (t_{i + 1}-T_i)) (1)
[0029]
Here, in the case of the constant bit rate encoding method in MPEG2, the encoder must encode so as to satisfy the condition of the following equation (2).
Bi ≦ B
Bi-bi ≧ 0 (2)
While such a condition is satisfied, the encoder does not perform encoding that causes a buffer overflow or underflow.
[0030]
In the case of the variable bit rate encoding method in MPEG2, the input bit rate R is the maximum bit rate defined by the profile and the level, and F = B. Therefore, the equation (1) can be rewritten as the following equation (3).
B₀= B
B_{i + 1}= Min (B, B_i―B_i+ R_max(T_{i + 1}-T_i)) (3)
[0031]
At this time, the encoder must execute encoding so as to satisfy the condition expressed by the following equation (4).
B_i―B_i≧ 0 (4)
When this condition is satisfied, the encoder performs encoding so that no buffer underflow occurs on the decoder side. When the decoder buffer is full, it means that the encoder buffer is empty and no encoded bit stream has been generated. Thus, the encoder need not monitor the decoder for buffer overflow.
[0032]
In MPEG, encoding is performed so as to observe the above-described buffer restrictions based on the buffer size and bit rate defined by each profile and level. A decoder conforming to each profile and level can decode the bitstream without breaking it.
[0033]
[Problems to be solved by the invention]
However, there are cases where the bit stream can be decoded even when the buffer size and the bit rate specified in the profile and level are not actually used.
[0034]
For example, a bit stream encoded with a bit rate R, a buffer B, and an initial delay time F (R, B, F) can be decoded even by a decoder having a larger buffer size B ′ (B ′> B). . It is also possible to decode at a higher bit rate R ′ (R ′> R).
[0035]
For example, even when the decoding bit rate of the decoder is lower than the encoding bit rate, it is possible to decode the decoder having a sufficiently large buffer size.
[0036]
Thus, given a given bitstream, at each bitrate, the minimum buffer size B required to decode that bitstream_minExists. Such a relationship is shown in FIG.
[0037]
JVT Codec is being standardized so that it can be decoded not only by a fixed bit rate and buffer size for each profile and level, but also by a decoder having the conditions shown in FIG. The purpose is to be able to perform decoding even if the encoding bit rate and buffer size of the encoder are not the same as the decoding bit rate and buffer size of the decoder. By achieving this object, for example, a decoder having a high decoding bit rate can reduce the buffer size.
[0038]
However, such information varies in time in the bitstream. For this reason, there is a problem that even if decoding is possible under a predetermined condition, decoding may not be possible under another condition because restrictions for decoder compatibility are relaxed. For example, when such (R, B) characteristics fluctuate with time, there is a problem that even if decoding is possible at a predetermined time, decoding may not be possible at another time.
[0039]
There has been a problem that even when moving to another scene or another channel due to random access or the like, decoding is not always possible. In addition, when editing at the bitstream level such as splicing, there is a problem that it is impossible to guarantee the decodeability.
[0040]
The present invention has been made in view of such a situation, and an object of the present invention is to efficiently determine the decodability of a bitstream and to easily edit a bitstream such as splicing.
[0053]
[Means for Solving the Problems]
  The editing apparatus according to the present invention decodes a bitstream included in a header inserted for each predetermined section that can be randomly accessed in the input bitstream.The buffer to store the bitstream inBuffer size andThe input bit rate to the buffer andWhen determining whether a bitstream is decodable according to the combination ofAs a criterion forRead the buffer characteristic information to be used, and based on the read information,BitstreamThe bitstream connected at the edit point decodes without breaking the decoder bufferA judgment means for judging whether or not it is possible and a judgment meansDecryptionBitstream if determined to be possibleConnect at edit pointsEditing means for performing editing, and the determining means includes the buffer characteristic information included in the header of the first bitstream.The first variable with the indicated buffer size and input bit rate as variablesA characteristic curve is added to the buffer characteristic information contained in the header of the second bitstream.A second with the indicated buffer size and input bit rate as variablesFirst bit stream and second bit stream if they are always above or identical to the characteristic curveAnd the bit stream connected at the edit point decodes without destroying the decoder bufferJudge that it is possible.
[0054]
  The editing method according to the present invention decodes a bitstream included in a header inserted for each predetermined section that can be accessed randomly in the input bitstream.A buffer to store the bitstream inBuffer size andThe input bit rate to the buffer andWhen determining whether a bitstream is decodable according to the combination ofAs a criterion forRead the buffer characteristic information to be used, and based on the read information,BitstreamThe bitstream connected at the edit point decodes without breaking the decoder bufferThe decision step to determine whether or not it is possible, and the processing of the decision stepDecryptionBitstream if determined to be possibleConnect at edit pointsAn editing step for performing editing, and the processing of the determining step is performed on the buffer characteristic information included in the header of the first bitstream.The first variable with the indicated buffer size and input bit rate as variablesA characteristic curve is added to the buffer characteristic information contained in the header of the second bitstream.A second with the indicated buffer size and input bit rate as variablesFirst bit stream and second bit stream if they are always above or identical to the characteristic curveAnd the bit stream connected at the edit point decodes without destroying the decoder bufferJudge that it is possible.
[0059]
  In the editing apparatus and method of the present invention, included in the header inserted for each predetermined section that can be randomly accessed in the input bitstream,BitstreamWhen decryptingA buffer to store the bitstream inBuffer size andThe input bit rate to the buffer andWhen determining whether a bitstream is decodable according to the combination ofAs a criterion forThe buffer characteristic information to be used is read, and based on the read information, the buffer characteristic information included in the header of the bitstream isThe first variable with the indicated buffer size and input bit rate as variablesA characteristic curve is added to the buffer characteristic information contained in the header of the second bitstream.A second with the indicated buffer size and input bit rate as variablesFirst bit stream and second bit stream if they are always above or identical to the characteristic curveAnd the bit stream connected at the edit point decodes without destroying the decoder bufferBitstream if determined to be possibleConnect at edit pointsEditing is done.
[0060]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the present invention will be described below with reference to the drawings. FIG. 5 is a diagram showing a configuration of an embodiment of an encoding apparatus to which the present invention is applied. The encoding device 70 shown in FIG. 5 is configured to include the image information encoding device 10 shown in FIG. Here, since the configuration of the image information encoding apparatus 10 has already been described, the description thereof will be omitted as appropriate.
[0061]
The image information input to the image information encoding device 10 is encoded and output to the buffer 71 and the bitstream analysis unit 72 as image compression information (BS: bitstream). The buffer 71 temporarily stores the input bit stream and outputs it to the buffer information adding unit 73 as necessary. The bit stream analysis unit 72 checks the buffer occupancy state between predetermined intervals in the bit stream, for example, GOPs or random access points, and supplies the information to the buffer information addition unit 73 as buffer information BH. Here, the random access point refers to a predetermined section in the JVT standard that can be accessed randomly in the bitstream. Similarly, the GOP refers to a predetermined section that can be accessed randomly in the MPEG2 / MPEG4 standard.
[0062]
The buffer information adding unit 73 adds the input buffer information BH to the input bit stream and outputs the same.
[0063]
Here, as an example of analysis performed by the bitstream analysis unit 72, a buffer occupancy state is checked between each random access point, and the bit occupancy state information is encoded as header information in each random access point to form a bitstream Will be described as an example. Here, such a description will be given, but it may be calculated in GOP units, or may be determined in other arbitrary units. In the case where other units are used as the units described below, However, it goes without saying that the present invention can be applied.
[0064]
With reference to FIG._min, B_min) Will be described. Where R_minIndicates the minimum value of the input bit rate R to the buffer, and B_minIs the minimum buffer size B.
[0065]
When a bit rate R of a predetermined bit stream is given, a minimum buffer size B that can be decoded by a decoding device (for example, having the configuration shown in FIG. 7) that decodes the bit stream at the decoding bit rate R_minIs determined as follows, for example.
[0066]
Let N be the number of frames between predetermined access points. The generated bit amount of each frame is b (i) (i = 1, N), the buffer occupancy immediately before extracting data of each frame from the buffer is B (i), and the buffer occupancy immediately after extraction is B2 (i). And If the buffer amount of the encoding device is B,
B2 (i) = B (i) -b (i)
B (i + 1) = B2 (i) + R / (Frame Rate) (5)
However, if (B (i + 1)> B) B (i + 1) = B, and the maximum value of B (i) is B. The delay amount F is F = B.
[0067]
At this time, B_minIs obtained by the following equation (6).
B_min= B-min (B2 (i)) (6)
R at this time is R_minIf the above method (R_min, B_min) Can be determined.
[0068]
Next, (R_min, B_min, F_min) Will be described. B = B_min, R = R_minAnd Similar to Expression (5), the following Expression (7) is established.
B2 (i) = B (i) -b (i)
B (i + 1) = B2 (i) + R / (Frame Rate) (7)
It becomes. However, underflow is monitored based on the following conditions.
if (B2 (i) <0) {
F_min= F_min+ (0−B2 (i));
B2 (i) = 0;}
[0069]
F_minIs initialized to 0 at the beginning of each random access point. Similarly, monitoring for overflow is performed based on the following conditions.
if (B (i + 1)> B) B (i + 1) = B
By performing the above check on all frames between random access points, (R_min, B_min, F_min) Is determined.
[0070]
(R_min, B_min, F_min) May be inspected by a predetermined number, or only independent combinations may be defined. The characteristics obtained as described above are as shown in FIG. Linear interpolation is performed between the points. (R) obtained as described above._min, B_min, F_min) And buffer information BH are inserted into a predetermined position in the bitstream by the buffer information adding unit 73, encoded and output.
[0071]
The bit stream analysis unit 72 (R) between each random access as described above._min, B_min, F_min) At the same time, the same analysis is performed on the entire bitstream, and the characteristics for the entire bitstream are expressed as_min, B_min, F_min) Global is determined, and this value is supplied to the buffer information adding unit 73 as buffer information BH.
[0072]
The bit stream BS output from the image information encoding device 10 is delayed by a predetermined time in the buffer 71 and then input to the buffer information adding unit 73. The buffer information adding unit 73 inserts the buffer information BH supplied from the bit stream analyzing unit 72 at a predetermined position in the bit stream, and outputs the final output bit stream BS.
[0073]
Here, the buffer information BH (or buffer characteristic information) is, for example, (R_min, B_min, F_min) And (R_min, B_min, F_min) Global. The buffer information adding unit 73 inserts the information at a predetermined position in the bit stream BS. Here, an example of syntax is shown and described below.
[0074]

[0075]
(R between random access points_min, B_min, F_min) Is recorded in the random access point header immediately before, for example, like the syntax described above. RAP_startcode is a code indicating the start of the RAP header.
[0076]
closed_GOP is a flag indicating whether all pictures in the GOP are independent without referring to pictures of other GOPs, or whether there is a dependency relationship referring to pictures of other GOPs. The broken_link is a flag indicating whether or not a prediction reference image exists when bitstream replacement is performed before and after the GOP by editing or the like.
[0077]
NumBuffer_Param is the property set (R_min, B_min, F_min).
Rate [i], Buffer [i], F [i]_min, B_min, F_minShow. Here, for example, R_minAre recorded in order from the smallest.
[0078]
(R for the entire bitstream_min, B_min, F_min) Global is recorded, for example, in the following sequence header in the first sequence header of the bitstream.
[0079]

[0080]
Here, NumBuffer_Param is the characteristic set (R_min, B_min, F_min) Indicates the number of global. Rate [i], Buffer [i], F [i]_min, B_min, F_minIndicates. Here, for example, R_minAre recorded in order from the smallest.
[0081]
In the buffer information adding unit 73, after the buffer information BH is added, the final output bit stream BS is output.
[0082]
In the embodiment of the invention, the buffer information BH is described so that all of the minimum bit rate Rmin, the minimum buffer size Bmin, and the minimum delay amount Fmin are added to the bit stream. However, the present invention is not limited to this example, and at least one of the minimum bit rate Rmin, the minimum buffer size Bmin, and the minimum delay amount Fmin may be added to the bitstream. For example, a combination of the minimum bit rate Rmin and the minimum buffer size Bmin may be added to the bitstream.
[0083]
FIG. 7 shows the configuration of an embodiment of a decoding apparatus to which the present invention is applied. The decoding device 90 shown in FIG. 7 corresponds to the encoding device 70 shown in FIG. 5, and includes the image information decoding device 40 shown in FIG. 2 inside. The bit stream BS input to the decoding device 90 is supplied to the bit stream analysis unit 91 and the decoding possibility determination unit 92.
[0084]
The bit stream analysis unit 91 decodes the buffer information BH in the bit stream and outputs it to the decoding possibility determination unit 92. The bitstream analysis unit 91 parses the bitstream and records it in the sequence header (R_min, B_min, F_min) Decrypt global. Also, (R) recorded in each random access point header._min, B_min, F_min). These pieces of information are output to the decoding possibility determination unit 92.
[0085]
Based on the buffer information BH and the decoder information DI supplied from the image information decoding device 40, the decodability determination unit 92 determines whether the input bitstream can be decoded without causing the buffer to fail. The decoder information DI is, for example, a decoder buffer size and a decoding bit rate.
[0086]
The decryptability determination unit 92 (R_min, B_min, F_min) From global, create a characteristic curve as shown in FIG. Linear interpolation is performed between the points. At this time, the buffer and decoding bit rate of the decoder (decoding device 90) are (R_min, B_min, F_min) If it is above the characteristic curve created by global, it can be determined that the input bitstream is decodable. Therefore, in such a case, the decoding possibility determination unit 92 determines that decoding is possible and supplies the bit stream to the image information decoding device 40.
[0087]
The image information decoding apparatus 40 has a configuration basically similar to that of the image information decoding apparatus 40 shown in FIG. 2, executes similar processing, decodes the input bit stream, and displays image information on a television not shown. Output to John receiver.
[0088]
Whether or not the entire bitstream can be decoded is determined by (R_min, B_min, F_min) It can be determined by examining the global characteristic curve, decoder buffer size, and decoding bit rate.
[0089]
Further, when it is desired to decode only a specific section from a predetermined random access point by random access or the like, the decoding possibility determination unit 92 similarly (R_min, B_min, F_min) To create a characteristic curve as shown in FIG. Linear interpolation is performed between the points. At this time, the decoder buffer and the decoding bit rate are (R_min, B_min, F_minThe bitstream is decodable if it lies above the characteristic curve created by Therefore, in such a case, the decoding possibility determination unit 92 determines that decoding is possible and supplies the bit stream to the image information decoding device 40.
[0090]
Next, a description will be given of editing a bitstream. FIG. 8 is a diagram showing a configuration of an embodiment of an editing apparatus 110 that edits a bitstream to which the present invention is applied. As an example of editing performed by the editing apparatus 110, a case where a splice is performed in which a part of the input bitstream 1 is replaced with another input bitstream 2 will be described as an example.
[0091]
Here, the splice will be briefly described. The splice is to perform editing by replacing a predetermined bit stream with another bit stream at a random access point. Such a splice is, for example, when a commercial broadcast is inserted into a television broadcast program. In this case, the input bit stream 1 is a television broadcast program bit stream, and the input bit stream 2 is a commercial bit stream.
[0092]
The input bitstream 1 is input to the bitstream analysis unit 111-1, and the input bitstream 2 is input to the bitstream analysis unit 111-2. The bit stream analysis units 111-1 and 111-2 decode the buffer information BH 1 and 2 included in the input bit streams 1 and 2, respectively, and output them to the bit stream editing unit 112.
[0093]
Based on the buffer information BH1 and 2, the bitstream editing unit 112 determines whether or not the input bitstream 2 can be inserted into the input bitstream 1 at a predetermined editing point. At this time, in order for the edited bit stream to be decodable without damaging the buffer of the decoder (decoding device 90), the condition is that the random access point and the buffer occupancy value immediately before it are the same. is necessary.
[0094]
Decoders using the MPEG2 and 4 schemes were supposed to operate at specific bit rates and buffer sizes. However, decoders using the JVT scheme have other bit rates and buffer sizes as shown in FIG. (R_min, B_min, F_min), The constraints on the buffer are relaxed so that it can be decoded.
[0095]
In order to prevent the possibility of decoding from changing before and after editing by editing the bitstream, (R_min, B_min, F_min) Are the same. Therefore, the bitstream editing unit 112 (R) in the random access point header located in the editing section._min, B_min, F_min) A characteristic is created for the input bitstreams 1 and 2, and when these values match, the section is replaced with the bitstream 2. If they do not match, padding bits are inserted into bitstream 1 or 2 and (R_min, B_min, F_min) Are matched, and then replaced with the input bitstream 2.
[0096]
In JVT, restrictions on the buffer are relaxed, but if this is utilized, it becomes possible to relax the adaptability condition of the buffer in the splice. In JVT, the buffer size and decoding bit rate of the decoder are (R_min, B_min, F_min), It can be seen that decoding is possible. Therefore, (R) of a predetermined editing section of the original input bitstream 1_min, B_min, F_min) In the predetermined editing section of the input bitstream 2 to be inserted (R)_min, B_min, F_min) Is always below, a decoder capable of decoding the input bitstream 1 can be decoded even if the section is replaced with the bitstream 2.
[0097]
FIG. 9 illustrates the relationship. Curve 1 is (R) in the editing section of input bitstream 1._min, B_min, F_min) Show characteristics. Curve 2 is (R in the editing section of input bitstream 2._min, B_min, F_min) Show characteristics. Since the decoder's buffer, the decoding bit rate, can be decoded if it is above this curve, it is guaranteed that it can be decoded when curve 2 is always below curve 1, as shown in FIG. The
[0098]
Therefore, the bitstream editing unit 112 (R) in the random access point header located in the editing section._min, B_min, F_min) Create a characteristic for the bitstreams 1 and 2, and if the characteristic curve of the bitstream 2 comes under the characteristic curve of the bitstream 1, replace that section with the bitstream 2.
[0099]
Conversely, if they do not match, padding bits are inserted into bitstream 1 or 2, and (R_min, B_min, F_min) After the characteristic curve is changed to be located below the characteristic curve of the bit stream 1, the input bit stream 2 is replaced.
[0100]
When splicing is performed so as to satisfy such a condition, a decoder capable of decoding the bitstream 1 is not broken. After performing the splicing, the bit stream editing unit 112 outputs a final bit stream.
[0101]
Thus, (R_min, B_min, F_min) Including the minimum bit rate, minimum buffer size, minimum initial delay time, and the like, the decoding side can efficiently determine the decoding possibility of the bit stream, and the bit stream such as splicing It is possible to make it possible to always perform decoding without compromising the buffer on the decoding side.
[0102]
FIG. 10 is a diagram illustrating an internal configuration example of a general-purpose personal computer. A CPU (Central Processing Unit) 211 of the personal computer executes various processes according to a program stored in a ROM (Read Only Memory) 212. A RAM (Random Access Memory) 213 appropriately stores data and programs necessary for the CPU 211 to execute various processes. The input / output interface 215 is connected to an input unit 216 including a keyboard and a mouse, and outputs a signal input to the input unit 216 to the CPU 211. The input / output interface 215 is also connected to an output unit 7 including a display and a speaker.
[0103]
Further, a storage unit 218 constituted by a hard disk or the like, and a communication unit 219 for exchanging data with other devices via a network such as the Internet are connected to the input / output interface 215. The drive 220 is used when data is read from or written to a recording medium such as the magnetic disk 231, the optical disk 232, the magneto-optical disk 233, and the semiconductor memory 234.
[0104]
As shown in FIG. 10, the recording medium is distributed to provide a program to the user separately from the personal computer, and a magnetic disk 231 (including a flexible disk) on which the program is recorded, an optical disk 232 (CD- Consists of package media including ROM (Compact Disc-Read Only Memory), DVD (including Digital Versatile Disc), magneto-optical disk 233 (including MD (Mini-Disc) (registered trademark)), or semiconductor memory 234 In addition, it is configured by a hard disk including a ROM 212 storing a program and a storage unit 218 provided to a user in a state of being pre-installed in a computer.
[0105]
In this specification, the steps for describing the program provided by the medium are performed in parallel or individually in accordance with the described order, as well as the processing performed in time series, not necessarily in time series. The process to be executed is also included.
[0106]
Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.
[0109]
【The invention's effect】
  The present inventionAccording to this editing apparatus and method, it is possible to reduce processing for editing such as splicing and to determine whether or not editing is easy.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating a configuration of an example of a conventional image information encoding device.
FIG. 2 is a diagram illustrating a configuration of an example of a conventional image information decoding device.
FIG. 3 is a diagram illustrating a buffer amount.
FIG. 4 is a diagram illustrating a relationship between a bit rate and a buffer amount.
FIG. 5 is a diagram showing a configuration of an embodiment of an encoding apparatus to which the present invention is applied.
FIG. 6 is a diagram illustrating a buffer amount.
FIG. 7 is a diagram illustrating the configuration of an embodiment of a decoding device to which the present invention has been applied.
FIG. 8 is a diagram showing a configuration of an embodiment of an editing apparatus to which the present invention is applied.
FIG. 9 is a diagram illustrating a relationship between a bit rate and a buffer amount.
FIG. 10 is a diagram illustrating a medium.
[Explanation of symbols]
70 encoding device, 71 buffer, 72 bit stream analyzing unit, 73 buffer information adding unit, 90 decoding device, 91 bit stream analyzing unit, 92 decoding possibility determining unit, 111 bit stream analyzing unit, 112 bit stream editing unit

Claims (2)

Random access in the input bit stream included in the header is inserted into a predetermined interval for each possible input to the buffer size of the buffer for storing the bit stream buffer when decoding the bit stream bits read buffer characteristic information used as a criterion in determining whether the whether the bit stream can be decoded in accordance with the combination of the bit rate, based on the readout information, and connecting the bit stream at the editing point Determining means for determining whether the stream is decodable without breaking the decoder buffer ;
Editing means for connecting the bitstream at an editing point when it is determined by the determining means that decoding is possible, and
The determination means has a first characteristic curve having the buffer size and the input bit rate indicated in the buffer characteristic information included in the header of the first bit stream as variables, and the second characteristic of the bit stream. The first bit stream if it is always above or identical to the second characteristic curve having the buffer size and the input bit rate indicated by the buffer characteristic information included in the header as variables; And an editing apparatus that determines that the bit stream connected to the second bit stream at the editing point can be decoded without breaking a decoder buffer . Random access in the input bit stream included in the header is inserted into a predetermined interval for each possible input to the buffer size of the buffer for storing the bit stream buffer when decoding the bit stream bits read buffer characteristic information used as a criterion in determining whether the whether the bit stream can be decoded in accordance with the combination of the bit rate, based on the readout information, and connecting the bit stream at the editing point A determination step of determining whether the stream is decodable without damaging the decoder buffer ;
An edit step for performing an edit by connecting the bit stream at an edit point when it is determined that decoding is possible in the process of the determination step;
In the process of the determining step, a first characteristic curve having the buffer size and the input bit rate indicated in the buffer characteristic information included in the header of the first bit stream as variables is set to the second bit. If it is always above or the same as the second characteristic curve with the buffer size and the input bit rate indicated in the buffer characteristic information included in the header of the stream as variables , the first An editing method for determining that a bit stream and a bit stream connected to the second bit stream at an editing point can be decoded without causing a decoder buffer to fail .

Images