RU2812616C1 - Image coding method, image decoding method, image coding device, image decoding device and image coding and decoding device - Google Patents

Abstract

FIELD: image encoding.

SUBSTANCE: image to be encoded is divided into a plurality of blocks to form target blocks. The predicted block is formed. The predicted block is subtracted from the target block to form a residual block. The residual block is converted and quantized to form a quantized block. Inverse quantization and inverse transformation of the quantized block are performed to form a local reconstructed residual block. A reconstructed block is formed by adding the reconstructed residual block and the predicted block. A SAO type parameter is determined for the SAO value of the reconstructed block, wherein the SAO type parameter indicates whether the SAO value is a boundary offset value or a range offset value, and the SAO value is to be added to the pixel value of the reconstructed image. A set of bins is found that represents an integer indicating the magnitude of the SAO value.

EFFECT: improved efficiency of image encoding.

1 cl, 38 dwg

Description

[Technical field]

[0001] The present invention relates to an image encoding method, an image encoding apparatus, an image decoding method, an image decoding apparatus, and an image encoding and decoding apparatus, and in particular to an image encoding method, an image encoding apparatus, an image decoding method, an image decoding apparatus, and an image encoding and decoding device for performing arithmetic encoding or arithmetic decoding on the sample adaptive offset (SAO) parameter.

[Prior Art]

[0002] Recently, there has been significant technological development in digital video devices, and increasing capabilities for encoding video (multiple images organized in a time series) with compression and recording the video onto recording media such as DVDs and hard drives, or distributing the video on the Internet . For example, H.264/AVC (MPEG-4 AVC) is one of the standards for compressed video encoding. Additionally, the High Efficiency Video Coding (HEVC) standard is currently being considered as a next-generation standard.

[0003] The current HEVC standard (NPL 1) contains an offset process called SAO. The SAO process is a process for adding an offset value to each pixel of an image decoded from a bitstream (coded bitstream). Performing the SAO process makes it possible to reduce image degradation resulting from encoding and make the decoded image more similar to the original image. According to the current HEVC standard, the SAO process performs arithmetic encoding on the offset value and adds the offset value to the bit stream.

[List of bibliographic references]

[Non-patent literature]

[0004] [NPL 1] Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 9th Meeting: Geneva, CH, 27 April-7 May 2012, JCTVC- I0602_CDTexts_r2.doc, BoG report on integrated text of SAO adoptions on top of JCTVC-I0030, http://phenix.it-sudparis.eu/jct/doc_end_user/documents/9_Geneva/wg11/JCTVC-I0602-v3.zip

[Essence of the invention]

[Technical challenge]

[0005] In the traditional technique, when encoding and decoding an image, faster processing is desired.

[0006] Here, the present invention provides an image encoding method and an image decoding method in which processing can be accelerated.

[The solution of the problem]

[0007] An image encoding method according to an aspect of the present invention is an image encoding method for encoding an input image and includes: converting into a bin string an offset value used in an offset process to be applied to a pixel value of a reconstructed image corresponding to the input image ; and performing bypass arithmetic encoding on the string of bins using constant probability.

[0008] In addition, an image decoding method according to another aspect of the present invention is an image decoding method for decoding an encoded image and includes: performing bypass arithmetic decoding on a string of bins using a constant probability, wherein the string of bins is encoded and indicates an offset value, used in the process offset, which must be applied to the pixel value of the reconstructed image corresponding to the encoded image; and converting to an offset value the string of bins on which bypass arithmetic decoding is performed.

[0009] These general or specific aspects of the present invention may be implemented by a system, device, integrated circuit, computer program, computer-readable recording medium, such as a CD-ROM, or any combination of a system, device, integrated circuit, computer program, and recording medium.

[Useful results of the invention]

[0010] By using the image encoding method, image encoding apparatus, image decoding method, image decoding apparatus, and image encoding and decoding apparatus according to the present invention, processing can be accelerated.

[Brief description of drawings]

[0011] FIG. 1 is a block diagram illustrating an exemplary configuration of an image encoding apparatus according to Embodiment 1.

Fig. 2 is a block diagram illustrating an exemplary configuration of an SAO parameter encoding unit.

Fig. 3 is a block diagram illustrating an exemplary configuration of a sao_offset encoding node.

Fig. 4 is a flowchart indicating exemplary processes in an image encoding apparatus according to Embodiment 1.

Fig. 5 is a flowchart indicating exemplary processes in the SAO parameter encoding unit in accordance with Embodiment 1.

Fig. 6A is a flowchart indicating exemplary processes in the sao_offset encoding unit in accordance with Embodiment 1.

Fig. 6B is a flowchart indicating exemplary processes in the sao_offset encoding unit in accordance with Flat 2 of Embodiment 1.

Fig. 7 is a table indicating the approximate correspondence between non-binary signals and bin strings.

Fig. 8 is a table indicating the correspondence between the bins included in binIdx and the arithmetic encoding that will be applied.

Fig. 9 is a table showing the result of an experiment on encoding efficiency in accordance with Embodiment 1 and Flavor 1.

Fig. 10 is a block diagram illustrating an exemplary configuration of an image decoding apparatus according to Embodiment 2.

Fig. 11 is a block diagram illustrating an exemplary configuration of an SAO parameter decoding section.

Fig. 12 is a block diagram illustrating an exemplary configuration of a sao_offset decoding unit.

Fig. 13 is a flowchart indicating exemplary processes in an image decoding apparatus according to Embodiment 2.

Fig. 14 is a flowchart indicating exemplary processes in the SAO parameter decoding section in accordance with Embodiment 2.

Fig. 15A is a flowchart indicating exemplary processes in the sao_offset decoding unit in accordance with Embodiment 2.

Fig. 15B is a flowchart indicating exemplary processes in the sao_offset decoding unit in accordance with Type 2 of Embodiment 2.

Fig. 16 illustrates a general configuration of a content delivery system for implementing content distribution services.

Fig. 17 illustrates a general configuration of a digital broadcast system.

Fig. 18 illustrates a block diagram of an example configuration of a TV.

Fig. 19 is a block diagram illustrating an example configuration of an information reproducing/recording unit that reads and writes information from or onto a recording medium that is an optical disc.

Fig. 20 illustrates a configuration example of a recording medium that is an optical disc.

Fig. 21A illustrates an example of a cell phone.

Fig. 21B illustrates a block diagram of an example configuration of a cellular phone.

Fig. 22 illustrates the structure of multiplexed data.

Fig. 23 schematically illustrates how each stream is multiplexed into multiplexed data.

Fig. 24 illustrates in more detail how the video stream is stored in the PES packet stream.

Fig. 25 illustrates the structure of TS packets and source packets in multiplexed data.

Fig. 26 illustrates the PMT data structure.

Fig. 27 illustrates the internal structure of multiplexed data information.

Fig. 28 illustrates the internal structure of stream attribute information.

Fig. 29 illustrates steps for identifying video data.

Fig. 30 is a block diagram illustrating an example of an integrated circuit configuration for implementing a movie encoding method and a movie decoding method according to each of the Embodiments.

Fig. 31 illustrates a configuration for switching between driving frequencies.

Fig. 32 illustrates steps for identifying video data and switching between excitation frequencies.

Fig. 33 illustrates an example of a lookup table in which video data standards are associated with excitation frequencies.

Fig. 34A illustrates an example configuration for sharing a module in a signal processing unit.

Fig. 34B illustrates an example configuration for sharing a module in a signal processing unit.

[Description of embodiments]

[0012] (Information on which the present invention is based)

The inventors have discovered that the image encoding method and others described in the "Prior Art" have the following problems.

[0013] As described above, according to the current HEVC standard, in the SAO process, which is an offset process, arithmetic encoding is performed on the offset value, and the offset value is added to the bitstream. More precisely, in HEVC arithmetic encoding, the signal to be encoded (for example, the SAO offset value) is converted from a non-binary signal to a string of bins, and arithmetic encoding is performed on the string of bins.

[0014] Arithmetic coding has two types: (1) context arithmetic coding, in which arithmetic coding is performed using an adaptive symbol occurrence probability; and (2) bypass arithmetic encoding, in which arithmetic encoding is performed by setting the probability of occurrence of a symbol to, for example, 50%.

[0015] (1) In context arithmetic encoding, a context is selected for each signal to be encoded. The probability of a symbol appearing is determined according to that context. Moreover, in context arithmetic encoding, a selected context is loaded, and arithmetic encoding is performed on the signal to be encoded based on the probability of occurrence of a symbol corresponding to the loaded context. Moreover, in context arithmetic encoding, a context updating process for updating the probability of occurrence of a symbol (eg, the probability of occurrence of 1) of the context is performed in accordance with the value of the signal to be encoded.

[0016] (2) In contrast, in bypass arithmetic encoding, arithmetic encoding is performed on the signal to be encoded by setting the symbol occurrence probability to, for example, 50% without using any context. Thus, bypass arithmetic encoding does not require loading and updating the context.

[0017] Traditionally, an image encoding apparatus performs context arithmetic encoding on the SAO offset value. The reason is that the character probability of the bin string indicating the SAO offset value most likely has some deviation according to the HEVC standard (NPL 1). When the probability of occurrence of a symbol has a deviation, the decrease in coding efficiency can be stopped using context arithmetic coding.

[0018] However, context arithmetic encoding requires processing such as loading and updating the context. In addition, when a context identical to the context selected in context arithmetic encoding over a predetermined SAO offset value is selected in context arithmetic encoding over another SAO offset value, context arithmetic encoding cannot begin over the next SAO offset value until the context for the offset value is updated The SAO on which arithmetic encoding is performed first. Thus, there are problems with sequential processes and reduced throughput.

[0019] To solve such problems, an image encoding method according to an aspect of the present invention is an image encoding method for encoding an input image and includes: converting into a bin string the offset value used in the offset process to be applied to the pixel value of the reconstructed image corresponding to the input image; and performing bypass arithmetic encoding on the string of bins using constant probability.

[0020] Since in the image encoding method, arithmetic encoding is performed on the SAO offset value using bypass arithmetic encoding, processing can be accelerated.

[0021] The inventors have discovered that the character probabilities of bin strings indicating the SAO offset value have slight variations, which will be described later. Thus, the encoding efficiency is likely not reduced even when bypass arithmetic encoding is performed. Thus, the inventors have confirmed by experiment that bypass arithmetic coding as arithmetic coding over the SAO offset value does not reduce the coding efficiency.

[0022] Thus, the method of encoding images with this configuration makes it possible to speed up processing without reducing encoding efficiency.

[0023] In addition, an image decoding method according to another aspect of the present invention is an image decoding method for decoding an encoded image and includes: performing bypass arithmetic decoding on a string of bins using a constant probability, wherein the string of bins is encoded and indicates an offset value, used in the process offset, which must be applied to the pixel value of the reconstructed image corresponding to the encoded image; and converting to an offset value the string of bins on which bypass arithmetic decoding is performed.

[0024] In addition, an image encoding device in accordance with another aspect of the present invention is an image encoding device that encodes an input image and includes a control circuit and a memory electrically connected to the control circuit, wherein the control circuit performs: conversion to a bin string an offset value used in the offset process to be applied to the pixel value of the reconstructed image corresponding to the input image; and performing bypass arithmetic encoding on the string of bins using constant probability.

[0025] In addition, an image decoding device according to another aspect of the present invention is an image decoding device that decodes an encoded image and includes a control circuit and a memory electrically connected to the control circuit, wherein the control circuit performs: performing bypass arithmetic decoding over a row of bins using a constant probability, wherein the row of bins is encoded and indicates an offset value used in the offset process to be applied to a pixel value of the reconstructed image corresponding to the encoded image; and converting to an offset value the string of bins on which bypass arithmetic decoding is performed.

[0026] In addition, an image encoding and decoding apparatus according to another aspect of the present invention includes an image encoding apparatus and an image decoding apparatus.

[0027] These general or specific aspects of the present invention may be implemented by a system, method, integrated circuit, computer program, computer-readable recording medium, such as a CD-ROM, or any combination of a system, method, integrated circuit, computer program, and recording medium.

[0028] Embodiments in accordance with the present invention will be described with reference to the drawings.

[0029] The embodiments described below indicate specific or general examples of the present invention. The values, shapes, materials, constituent elements, positions and connections of constituent elements, steps and orders of steps shown in the Embodiments are examples and do not limit the present invention. The constituent elements in the Embodiments that are not described in the independent claims that describe the most general idea of the present invention are described as arbitrary constituent elements.

[0030] (Embodiment 1)

An image encoding apparatus and an image encoding method according to Embodiment 1 will be described with reference to FIG. 1-9.

[0031]<1-1. Device Configuration>

An image encoding apparatus according to Embodiment 1 will be described with reference to FIG. 1. Fig. 1 is a block diagram illustrating the configuration of an image encoding apparatus 100 in accordance with the HEVC standard in accordance with Embodiment 1.

[0032] The image encoding device 100 is a device that encodes an image to be input (input image) block by block to generate a bit stream (coded bit stream).

[0033] As illustrated in FIG. 1, the image encoding device 100 includes a block partitioning unit 101, a subtraction unit 102, a transform unit 103, a variable length encoding unit 104 (entropy encoding unit), an inverse transform unit 105, an addition unit 106, an SAO processing unit 107, and a unit 110. SAO parameter encoding, frame memory 108 and prediction unit 109.

[0034] The block partitioning unit 101 partitions the input image into blocks to be predicted, which include pixels. The blocks that will be predicted are variable size blocks. For example, the minimum block size is 4x4 pixels, and the maximum block size is 32x32 pixels.

[0035] The subtraction unit 102 subtracts the prediction block predicted by the prediction unit 109, which will be described later, from each of the blocks to be predicted in order to calculate the difference block. The conversion unit 103 converts the difference block from the image domain to the frequency domain to output frequency coefficients, and quantizes these frequency coefficients. Quantized frequency coefficients will henceforth be called quantized coefficients.

[0036] The variable length encoding unit 104 performs entropy encoding on the quantized coefficients output from the transform unit 103 and various information required for decoding (partitioning information for blocks to be predicted, prediction type, motion vector, prediction mode (intra mode) prediction), quantized parameter, etc.) to generate a stream of binary signals.

[0037] The inverse transform unit 105 performs inverse quantization to convert the quantized coefficients output from the transform unit 103 into frequency coefficients, and performs inverse transform to convert the frequency coefficients from the frequency domain to the image domain to output a reconstructed difference block. The addition unit 106 adds the prediction block predicted by the prediction unit 109, which will be described later, with the reconstructed difference block output by the inverse transform unit 105 to form a reconstructed block.

[0038] The SAO processing unit 107 performs an SAO process (offset process) on the reconstructed block to add to the pixel value of each pixel included in the reconstructed block an SAO offset value corresponding to the category (class) to which the pixel belongs, which will be described later. Since the process performed by the transformation unit 103 is an irreversible process, an error occurs between each of the blocks to be predicted in the input image and the reconstructed block. The SAO offset value is an error correction value and is set per category.

[0039] The SAO parameter encoding section 110 performs variable length encoding on various parameters to be used in the SAO process, and adds the parameters to the bitstream output from the variable length encoding section 104. Embodiment 1 provides sao_offset indicating the SAO offset value and sao_type_idx indicating the category of the SAO offset value as the SAO parameters.

[0040] FIG. 2 is a block diagram illustrating an exemplary configuration of the SAO parameter encoding unit 110. As illustrated in FIG. 2, the SAO parameter encoding unit 110 includes a sao_type_idx encoding unit 111 and a sao_offset encoding unit 112.

[0041] The sao_type_idx encoding unit 111 performs arithmetic encoding on the sao_type_idx indicating the category of the sao_offset. Here sao_offset is classified into two categories: the edge offset value determined according to the state of the surrounding pixels; and a range offset value determined according to the region to which the pixel value of the pixel belongs.

[0042] The sao_offset encoding unit 112 performs arithmetic encoding on the sao_offset. Fig. 3 is a block diagram illustrating an exemplary configuration of sao_offset encoding unit 112.

[0043] As illustrated in FIG. 3, the sao_offset encoding unit 112 includes a sao_offset binary conversion unit 113, an arithmetic encoding switching unit 114, a context 0 arithmetic encoding unit 115, a context 1 arithmetic encoding unit 116, and a bypass arithmetic encoding unit 117.

[0044] The sao_offset to binary conversion unit 113 converts sao_offset, which is a non-binary signal, into binary form (converts sao_offset to a signal representing 0 and 1). The arithmetic encoding switching unit 114 switches the arithmetic encoding method to be applied to sao_offset. Context 0 arithmetic encoding unit 115 performs arithmetic encoding of context 0 using the first context (context 0). Context 1 arithmetic encoding unit 116 performs arithmetic encoding of context 1 using a second context (context 1). The bypass arithmetic encoding unit 117 performs bypass arithmetic encoding using a constant probability.

[0045] The frame memory 108 includes some storage device, such as random access memory (RAM) and read only memory (ROM).

[0046] The prediction unit 109 generates a prediction block by inter-prediction or intra-prediction. In external prediction, the prediction unit 109 outputs a motion vector using the block to be predicted, and generates a prediction block using the output motion vector and the reconstructed block obtained in the SAO process and stored in the frame memory 108. In intra prediction, the prediction unit 109 generates a prediction block using the reconstructed block obtained in the SAO process and stored in the frame memory 108, in accordance with the prediction mode.

[0047]<1-2 General operations>

Next, with reference to FIG. 4, operations of the image encoding apparatus will be described. Fig. 4 is a flowchart indicating a procedure in the image encoding apparatus 100.

[0048] In the image encoding apparatus 100, the block partitioning unit 101 first divides the input image into blocks to be predicted, and sequentially outputs these blocks to the subtraction unit 102 and the prediction unit 109 (step S101). The blocks to be predicted have a variable size in accordance with Embodiment 1. The input image is split using image features. The minimum size of blocks to be predicted is 4 pixels horizontally (4 pixels vertically according to Embodiment 1. Additionally, the maximum size of blocks to be predicted is 32 pixels horizontally (32 pixels vertically according to Embodiment 1). 1 implementation.

[0049] The prediction unit 109 in the image encoding device 100 generates a prediction block using the blocks to be predicted and the reconstructed block stored in the frame memory 108 (step S102).

[0050] The subtraction unit 102 in the image encoding device 100 subtracts a prediction block from each of the blocks to be predicted to generate a difference block (step S103).

[0051] The conversion unit 103 in the image encoding device 100 converts the difference block into frequency coefficients, and converts the frequency coefficients into quantized coefficients (step S104).

[0052] The inverse transform unit 105 in the image encoding apparatus 100 inversely quantizes the quantized coefficients to output frequency coefficients, and converts the frequency coefficients from the frequency domain to the image domain to form a reconstructed difference block (step S105).

[0053] The adding unit 106 in the image encoding device 100 adds the prediction block with the reconstructed difference block to form a reconstructed block (step S106).

[0054] The SAO processing unit 107 in the image encoding device 100 executes the SAO process (step S107). The SAO processing unit 107 first determines the SAO parameters to be used for the SAO process. In Embodiment 1, the SAO parameters are sao_offset indicating the SAO offset value, and sao_type_idx indicating the category of the SAO offset value. The SAO processing unit 107 adds the SAO offset value to the pixel value of each pixel included in the reconstructed block, and stores in the 108 frame memory the reconstructed block to which the SAO offset values are added. In the SAO process, each of the pixels included in the reconstructed block is classified into categories, and for each of those categories, an SAO offset value is output. Methods for classifying pixels are numerous. Therefore, the sao_type_idx parameter is used to indicate the category of the SAO offset value.

[0055] The SAO parameter encoding section 110 in the image encoding device 100 performs variable length encoding on the SAO parameter and adds the parameter to the bit stream output from the variable length encoding section 104 (step S108). Details of the SAO parameter encoding unit 110 will be described later.

[0056] The variable length encoding unit 104 in the image encoding device 100 performs variable length encoding on the quantized coefficients output from the transform unit 103 to output a bitstream (step S109).

[0057] The image encoding apparatus 100 repeatedly executes Steps S102 to S109 until encoding of all input image blocks to be predicted is completed (Step S110).

[0058]<1-2-1. SAO Parameter Encoding Node Operations>

Operations of the SAO parameter encoding unit 110 according to Embodiment 1 will be described with reference to FIG. 5.

[0059] FIG. 5 is a flowchart indicating exemplary processes of the SAO parameter encoding unit 110, and details Step S108 of FIG. 4.

[0060] The sao_type_idx encoding unit 111 in the SAO parameter encoding unit 110 encodes the sao_type_idx (step S111).

[0061] The sao_offset encoding unit 112 in the SAO parameter encoding unit 110 encodes the sao_offset (step S112).

[0062]<1-2-2. Encoding node operations sao_offset>

Operations of the sao_offset encoding unit 112 according to Embodiment 1 will be described with reference to FIG. 6A. Fig. 6A is a flowchart indicating exemplary processes in sao_offset encoding unit 112.

[0063] The sao_offset to binary conversion unit 113 in the sao_offset encoding unit 112 converts the sao_offset from a non-binary signal to a string of bins (step S121).

[0064] Embodiment 1 describes a case where sao_offset takes values from 0 to 7. FIG. 7 is a table indicating the correspondence between non-binary signals and bin strings. As can be seen from Fig. 7, the number of consecutive "1s" from the beginning of each line of bins is equal to the value indicated by the non-binary signal. Also, "0" is added to the end when the non-binary signal is between 0 and 6. When the non-binary signal is 7, which is the maximum number, no "0" is added to the end. Here binIdx is the index of the bin (one bit) included in the bin string. The first value of binIdx is 0, and subsequent values are incremented by 1. A string of bins is output to the arithmetic encoding switching unit 114 sequentially from the bin whose binIdx is 0.

[0065] The arithmetic encoding switching unit 114 in the sao_offset encoding unit 112 determines the value of binIdx (steps S122, S124 and S126).

[0066] When the value is 0 (Yes in step S122), the arithmetic encoding switching unit 114 outputs a bin to the context 0 arithmetic encoding unit 115. The context 0 arithmetic encoding unit 115 performs context arithmetic encoding using context 0 (step S123).

[0067] When the value is 1 (No in step S122 and Yes in step S124), the arithmetic encoding switching unit 114 outputs a bin to the context 1 arithmetic encoding unit 116. The context 1 arithmetic encoding unit 116 performs context arithmetic encoding using context 1 (step S125).

[0068] When the value is neither 0 nor 1 (No in step S122 and No in step S124), the arithmetic encoding switching unit 114 outputs the bin to the bypass arithmetic encoding unit 117. The bypass arithmetic encoding unit 117 performs bypass arithmetic encoding (step S126).

[0069] Since the bypass arithmetic encoding unit 117 does not use any context, the sao_offset encoding unit 112 generally uses two types of contexts in accordance with Embodiment 1.

[0070] The sao_offset encoding unit 112 repeats the processes from Step 122 to Step 127 until processing of all bins included in the bin string is completed (Step S127).

[0071]<1-3. Advantages, varieties, etc.>

The image encoding apparatus 100 according to Embodiment 1 can speed up processing by performing bypass arithmetic encoding on the bins in the second half of sao_offset. More precisely, according to Embodiment 1, not context arithmetic encoding, but bypass arithmetic encoding is performed on bins whose binIdx is 2 or higher. As described above, bypass arithmetic encoding does not require loading or updating any context. In this way, arithmetic encoding on the SAO offset value of a block to be predicted can be started without waiting for completion of arithmetic encoding on another SAO offset value, and processing can be accelerated.

[0072] In addition, the character occurrence probabilities (occurrence probabilities 1) of binIdx 1 to 6 according to the HEVC standard (NPTL 1) that is currently being studied are almost the same, but not equal to 50% and have a deviation. In other words, when there is a binIdx after binIdx1 (when sao_offset is 1 or higher), the cases are often divided into a case where binIdx1 is 0 without any binIdx2 and after (sao_offset is 1), and a case where all binIdx 1 to 6 are equal to 1 (sao_offset is equal to 7).

[0073] However, the inventors found that, as a result of the experiment, for encoding binIdx 1 to 6 with a constant symbol occurrence probability of 50%, the encoding efficiency is almost not degraded. In particular, the experiment showed that sao_offset often indicates an intermediate value (from 2 to 6), and the probabilities of binIdx 1 to 6 appearing are close to 50%. Thus, processing for suppressing reduction in encoding efficiency can be accelerated by using not context arithmetic encoding, but bypass arithmetic encoding on binIdx after binIdx2.

[0074]<1-3-1. Varieties>

Although bypass arithmetic encoding is performed on bins after binIdx2 in accordance with Embodiment 1, the processing is not limited to this.

[0075] FIG. 8 is a table indicating the correspondence between the bins included in binIdx and the arithmetic encoding that will be applied. In fig. 8 "Embodiment 1" indicates the correspondence between the bins described in Embodiment 1 and the arithmetic encoding to be applied. In addition, "Context 0" indicates context arithmetic encoding using context 0. "Context 1" indicates context arithmetic encoding using context 1. "Bypass" indicates bypass arithmetic encoding.

[0076] "Change 1" in FIG. 8 indicates Flavor 1. According to Flavor 1, context arithmetic encoding is performed on the bin at binIdx0, and bypass arithmetic encoding is performed on bins after binIdx1. Moreover, according to Type 1, the sao_offset encoding node 112 in FIG. 2 should only include sao_offset binary conversion unit 113, arithmetic encoding switching unit 114, context 0 arithmetic encoding unit 115, and arithmetic encoding bypass unit 117. In other words, the sao_offset encoding node 112 does not need to include the context 1 arithmetic encoding node 116.

[0077] "Change 2" in FIG. 8 indicates Flavor 2. In Flavor 2, bypass arithmetic encoding is performed on each of the bins in binIdx, as shown in FIG. 8.

[0078] FIG. 6B is a flowchart indicating exemplary processes (image encoding method) in the sao_offset encoding unit 112 according to Type 2. FIG. 6B sao_offset is converted to binary form (step S121 to convert the offset value into a string of bins), and bypass arithmetic encoding is performed on all bins included in sao_offset (steps S126 and S127 to perform bypass arithmetic encoding). Because bypass arithmetic encoding is performed on all bins, processing can be fastest in Type 2.

[0079] In addition, according to Type 2, sao_offset encoding unit 112 in FIG. 2 should only include a sao_offset binary conversion node 113 and a bypass arithmetic encoding node 117. In other words, the sao_offset encoding unit 112 need not include the context 0 arithmetic encoding unit 115 and the context 1 arithmetic encoding unit 116.

[0080] Although not illustrated, it is acceptable to perform context arithmetic encoding on binIdx 0 to 2 and bypass arithmetic encoding on binIdx after binIdx3.

[0081]<1-3-2. Coding efficiency experiment result>

Fig. 9 is a table showing the result of an experiment on encoding efficiency in accordance with Embodiment 1 and Flavor 1.

[0082] FIG. 9 shows the comparison result between the encoding efficiency obtained by performing arithmetic encoding on the SAO offset value in accordance with the current HEVC standard (NPL 1), and the encoding efficiency in accordance with Embodiment 1 and Variation 1. The experimental conditions adhere to the general experimental conditions in the structure of the standard. HEVC.

[0083] The larger the value, the lower the coding efficiency. A negative value indicates an increase in encoding efficiency.

[0084] As shown in "Embodiment 1" of FIG. 9, the values indicating the comparison result range between -0.1% and 0.1%. This indicates that the encoding efficiency for binIdx2 - binIdx6 encoding at the SAO offset value is almost the same between context arithmetic encoding and bypass arithmetic encoding.

[0085] Although the coding efficiency of Variation 1 is lower than that of Embodiment 1, as seen in "Variation 1" in FIG. 9, the results range between 0 and 0.1%, and the coding efficiency is almost the same.

[0086] Although the coding efficiency of Variation 2 is also lower than that of Embodiment 1, as seen in "Variation 2" in FIG. 9, the results range between 0 and 0.1%, and the coding efficiency is almost the same.

[0087] As a result of the encoding efficiency experiment, using bypass arithmetic encoding when performing arithmetic encoding on the SAO offset value makes it possible to speed up arithmetic encoding while the encoding efficiency is less likely to degrade.

[0088] (Embodiment 2)

An image decoding apparatus and an image decoding method according to Embodiment 2 will be described with reference to FIG. 10-15A.

[0089]<2-1. Device Configuration>

An image decoding apparatus according to Embodiment 2 will be described with reference to FIG. 10. Fig. 10 is a block diagram illustrating the configuration of an image decoding apparatus 200 in accordance with the HEVC standard in accordance with Embodiment 2.

[0090] The image decoding device 200 is a device that decodes an input bit stream (coded bit stream) to generate a decoded image ("Image" in FIG. 10).

[0091] As illustrated in FIG. 10, the image decoding apparatus 200 includes a variable length decoding unit 201, an inverse transform unit 202, an SAO parameter decoding unit 210, an SAO processing unit 204, a decoded block combining unit 205, a frame memory 206, and a prediction unit 207.

[0092] The variable length decoding section 201 performs entropy decoding (variable length decoding) on the binary signal stream input to the image decoding apparatus 200 to obtain quantized coefficients and various information required for decoding (partitioning information for blocks to be predicted, prediction type, motion vector, prediction mode (intra prediction mode), quantized parameter, etc.).

[0093] The inverse transform unit 202 performs inverse quantization to convert the quantized coefficients output from the variable length decoding unit 201 into frequency coefficients, and performs inverse transform to convert the frequency coefficients from the frequency domain to the image domain to output a difference block.

[0094] The addition unit 203 adds the prediction block output from the prediction unit 207 with the difference block output from the inverse transform unit 202 to form a decoded block.

[0095] The SAO parameter decoding section 210 decodes various parameters to be used in the SAO process from the bit stream. Embodiment 2 provides sao_offset indicating the SAO offset value and sao_type_idx indicating the category of the SAO offset value as the SAO parameters.

[0096] FIG. 11 is a block diagram illustrating an exemplary configuration of the SAO parameter decoding unit 210. As illustrated in FIG. 11, the SAO parameter decoding unit 210 includes a sao_type_idx decoding unit 211, which performs arithmetic decoding on sao_type_idx, and a sao_offset decoding unit 212, which performs arithmetic decoding on sao_offset.

[0097] FIG. 12 is a block diagram illustrating an exemplary configuration of sao_offset decoding unit 212. As illustrated in FIG. 12, the sao_offset decoding unit 212 includes an arithmetic decoding switching unit 213, a context 0 arithmetic decoding unit 214, a context 1 arithmetic decoding unit 215, a bypass arithmetic decoding unit 216, and a sao_offset decoding unit 217 from binary form.

[0098] The arithmetic decoding switching unit 213 switches the arithmetic decoding method to be applied to the binary signal stream. Context 0 arithmetic decoding section 214 performs arithmetic decoding of context 0 using the first context (context 0). Context 1 arithmetic decoding unit 215 performs arithmetic decoding of context 1 using a second context (context 1). Bypass arithmetic decoding unit 216 performs bypass arithmetic decoding using a constant probability. The sao_offset binary deconversion node 217 debinarizes the string of sao_offset bins output from each of the context 0 arithmetic decoding node 214, the context 1 arithmetic decoding node 215, and the bypass arithmetic decoding node 216 to output a non-binary sao_offset value.

[0099] The SAO processing unit 204 performs an SAO process (offset process) on the decoded block output from the adding unit 203 to add an SAO offset value to the pixel value of each pixel included in the decoded block, which will be described in detail later.

[0100] The decoded block combining unit 205 combines the decoded blocks obtained in the SAO process to restore the decoded image. In addition, the decoded block combining unit 205 stores the decoded image in the frame memory 206.

[0101] The frame memory 206 includes some storage device, such as random access memory (RAM) and read only memory (ROM).

[0102] The prediction unit 207 generates a prediction block by inter-prediction or intra-prediction. The prediction method to be applied by the prediction unit 207 is the same as in the prediction unit 109.

[0103]<2-2 General operations>

Next, with reference to FIG. 13, operations of the image decoding apparatus 200 will be described. Fig. 13 is a flowchart indicating a procedure in the image decoding apparatus 200.

[0104] The SAO parameter decoding section 210 in the image decoding apparatus 200 performs variable length decoding on the SAO parameter (step S201).

[0105] The variable length decoding section 201 in the image decoding apparatus 200 performs variable length decoding on the bitstream to obtain quantized coefficients, and outputs the quantized coefficients to the inverse transform section 202 (step S202).

[0106] The inverse transform unit 202 in the image decoding apparatus 200 performs inverse quantization on the quantized coefficients to obtain frequency coefficients, and performs inverse transform on the obtained frequency coefficients to generate a difference block (step S203).

[0107] The prediction unit 207 in the image decoding device 200 generates a prediction block from the decoded image stored in the frame memory 206. In addition, the adding unit 203 in the image decoding apparatus 200 adds the prediction block output from the prediction unit 207 with the difference block output from the inverse transform unit 202 to form a decoded block (step S204).

[0108] The SAO processing unit 204 in the image decoding apparatus 200 performs the SAO process (step S205). In the SAO process, the SAO processing unit 204 classifies each pixel of the decoded block into categories according to the SAO parameter and adds an offset value corresponding to each of the categories to the pixel. SAO parameters include the sao_type_idx parameter, which specifies the category, and the sao_offset parameter, which specifies the offset value.

[0109] The image decoding apparatus 200 repeats the processes from Step 201 to Step 205 until decoding of all blocks included in the encoded image is completed (Step S206).

[0110] The decoded block combining unit 205 in the image decoding apparatus 200 combines the decoded blocks to form a decoded image, and stores the decoded image in the frame memory 206.

[0111]<2-2-1. Variable Length Parameter Decoding Node Operations SAO>

Operations of the SAO parameter decoding section 210 according to Embodiment 2 will be described with reference to FIG. 14.

[0112] FIG. 14 is a flowchart indicating exemplary processes of the SAO parameter decoding unit 210, and details Step S205 of FIG. 13.

[0113] The sao_type_idx decoding section 211 in the SAO parameter decoding section 210 decodes the sao_type_idx (step S211).

[0114] The sao_offset decoding section 212 in the SAO parameter decoding section 210 decodes the sao_offset (step S212).

[0115]<2-2-2. Operations of decoding node 212 sao_offset>

Operations of the sao_offset decoding unit 212 according to Embodiment 2 will be described with reference to FIG. 15A. Fig. 15A is a flowchart indicating exemplary processes in sao_offset decoding unit 212.

[0116] The arithmetic decoding switching unit 213 in the sao_offset decoding unit 212 determines the binIdx value of the bin to be decoded (steps S221, S223 and S225).

[0117] When the value is 0 (Yes in step S221), the arithmetic decoding switching section 213 outputs the bin to be decoded to the context 0 arithmetic decoding section 214. The context 0 arithmetic decoding section 214 performs context arithmetic decoding using context 0 (step S222).

[0118] When the value is 1 (No in step S221 and Yes in step S223), the arithmetic decoding switching unit 213 outputs the bin to be decoded to the context 1 arithmetic decoding unit 215. The context 1 arithmetic decoding unit 215 performs context arithmetic decoding with using context 1 (step S224).

[0119] When the value is neither 0 nor 1 (No in step S221 and No in step S223), the arithmetic decoding switching section 213 outputs the bin to be decoded to the bypass arithmetic decoding section 216. The bypass arithmetic decoding unit 216 performs bypass arithmetic decoding.

[0120] The sao_offset decoding section 212 repeats the processes from Step 221 to Step 226 until the value of the bin obtained in the arithmetic decoding is not 0, or a bin whose binIdx is 6 is decoded (Step S226).

[0121] The sao_offset decoding section 212 converts the string of bins including the bins decoded by the sao_offset decoding section 217 from binary form into a non-binary signal (step S227). The correspondence between non-binary signals and bin strings is the same as shown in FIG. 7 in accordance with Embodiment 1.

[0122]<2-3. Benefits, etc.>

The image decoding apparatus 200 according to Embodiment 2 can speed up processing by performing bypass arithmetic decoding on bins in the second half of sao_offset.

[0123]<2-3-1. Varieties>

Although bypass arithmetic decoding is performed on bins after binIdx2 in accordance with Embodiment 2, the processing is not limited to this. In the same manner as in Embodiment 1, bypass arithmetic decoding can be performed on bins after binIdx1 (Variation 1) and on all bins (Variation 2).

[0124] In Type 1 according to Embodiment 2, context arithmetic decoding is performed on the bin in binIdx0, and bypass arithmetic decoding is performed on bins after binIdx1. This process is a decoding process corresponding to the encoding process of Type 1 of Embodiment 1 in FIG. 8. Since the sao_offset decoding unit 212 in FIG. 12, according to Flavor 1, performs bypass arithmetic decoding on the bins after binIdx1, it shall only include an arithmetic decoding switch node 213, a context 0 arithmetic decoding node 214, a bypass arithmetic decoding node 216, and a sao_offset from binary form decoding node 217. In other words, the sao_offset decoding unit 212 according to Type 1 of Embodiment 2 does not need to include the context 1 arithmetic decoding unit 215.

[0125] According to Type 2 of Embodiment 2, the sao_offset decoding node 212 performs bypass arithmetic decoding on each of the bins in binIdx. This process is a decoding process corresponding to the encoding process of Type 2 of Embodiment 1 in FIG. 8.

[0126] FIG. 15B is a flowchart indicating exemplary processes (image decoding method) in the sao_offset decoding unit 212 according to Type 2 of Embodiment 2. In fig. 15B, bypass arithmetic decoding is performed on all bins included in sao_offset (steps S225 and S226 for performing bypass arithmetic decoding), and the decoded sao_offset is debinned back from binary form (step S227 for converting a string of bins to an offset value). Because bypass arithmetic decoding is performed on all bins, processing can be fastest in Type 2.

[0127] Since the sao_offset decoding section 212 in FIG. 12, in accordance with Flavor 2 of this document, performs bypass arithmetic decoding on all bins, it shall only include a bypass arithmetic decoding node 216 and a sao_offset from binary form decoding node 217. In other words, the sao_offset decoding unit 212 according to Type 2 of Embodiment 2 does not need to include the arithmetic decoding switching unit 213, the context 0 arithmetic decoding unit 214, and the context 1 arithmetic decoding unit 215.

[0128] Although not illustrated, it is acceptable to perform context arithmetic decoding on binIdx 0 to 2 and bypass arithmetic decoding on binIdx after binIdx3 as in Embodiment 1.

[0129] (Other variations of Embodiments 1 and 2)

(1) Although Embodiments 1 and 2 (including Variants) use sao_type_idx indicating the category of the SAO offset value and sao_offset indicating the SAO offset value as the SAO parameters, the SAO parameters are not limited to this. The SAO parameters may include, for example, a parameter that is intended to be auxiliary information for pixel classification.

[0130] (2) In addition, although according to Embodiment 1 (including the Varieties), the SAO parameter is encoded on each block to be predicted, the encoding is not limited to this. The SAO parameter may be encoded not for each block that must be predicted, but for one less than the block that must be predicted, or vice versa, for one obtained by combining the blocks that must be predicted. In addition, the SAO parameter is not encoded in the block that is to be predicted, but instead the value of another block can be copied and used.

[0131] Here, the image decoding apparatus 200 according to Embodiment 2 (including Varieties) decodes the encoded SAO parameter according to the encoding size of the SAO parameter in the image encoding apparatus 100.

[0132] (3) Although sao_offset takes values from 0 to 7 in accordance with Embodiments 1 and 2 (including Varieties), sao_offset is not limited to this. It can take values of 8 or higher. Additionally, sao_offset may take a negative value, in which case another parameter sao_offset_sign representing the sign bit may be provided.

[0133] (4) In addition, although Embodiment 1 (including Varieties) describes sao_offset encoding, the processing is not limited to this. The method for encoding the SAO parameter in accordance with Embodiment 1 (including Flavors) may be used to encode other syntax to be added to the bit stream. In addition, when the method for encoding an SAO parameter according to Embodiment 1 and Variation 1 thereof is applied to encoding syntaxes, and the use of context (the number of bins and binIdx that use the context) is unified, the same variable length encoding node is available when encoding syntaxes. Accordingly, the device configuration can be simplified.

[0134] Examples of syntaxes to which the method for encoding the SAO parameter is applied in accordance with Embodiment 1 and Variants 1 and 2 thereof include ref_idx indicating a reference image index, merge_idx indicating a motion vector copy source and others, and mpm_idx or intra_chroma_pred_mode indicating the intra prediction mode, according to the current HEVC standard (NPL 1).

[0135] Here, the image decoding apparatus 200 according to Embodiment 2 (including Varieties) also decodes the syntax encoded using the method for encoding the SAO parameter using the method for decoding the SAO parameter.

[0136] (5) In addition, although the size of a block to be predicted according to Embodiments 1 and 2 (including Varieties) is a maximum of 32×32 pixels and a minimum of 4×4 pixels, the size is not limited to this. In addition, although Embodiments 1 and 2 (including Variants) describe blocks to be predicted that have a variable size, the size of the blocks may be constant.

[0137] (6) Moreover, processing in accordance with Embodiments 1 and 2 (including Varieties) may be implemented by software. The software may be distributed through, for example, downloading. In addition, the software may be distributed by recording it on a recording medium such as a CD-ROM. This is also true for other embodiments in the description.

[0138] (7) In Embodiments 1 and 2 (including Varieties), each of the functional blocks can be implemented by, for example, an MPU and a memory device in most cases. In addition, generally, processing by each of the functional blocks may be implemented by software (program), and such software is recorded on a recording medium such as a ROM. In addition, such software can be distributed, for example, by downloading and burning onto recording media such as CD-ROM. Each of the functional blocks can be implemented using hardware (specialized circuitry).

[0139] (8) The processing described in Embodiments 1 and 2 (including Varieties) may be performed as centralized processing by a single device (system) or as decentralized processing by multiple devices. Here the program can be executed by one or more computers. In other words, any of centralized processing and decentralized processing can be performed.

[0140] (9) In addition, each of the constituent elements according to Embodiments 1 and 2 (including Varieties) may be implemented by specialized hardware or by executing a computer program suitable for the constituent element. Each of the constituent elements may be implemented by a program execution unit, such as a central processing unit (CPU) and a processor that reads and executes a computer program recorded on a recording medium such as a hard disk or a semiconductor storage device.

[0141] Specifically, each of the image encoding device 100 and the image decoding device 200 includes a control circuit and a memory electrically connected to the control circuit (accessible from the control circuit). The control circuit includes at least one of specialized hardware and a program execution unit. When the control circuit includes a program execution unit, the storage device stores a computer program executed by the program execution unit.

[0142] The present invention is not limited to Embodiments 1 and 2 (including Varieties), and various modifications are possible and are included within the scope of the present invention.

[0143] (Embodiment 3)

An independent computer system can easily perform the processing described in each of the Embodiments by recording on a recording medium a program for implementing the structure of a movie encoding method (image encoding method) or a movie decoding method (image decoding method) according to the Embodiment. The recording medium may be anything as long as the program can be recorded thereon, such as a magnetic disk, an optical disk, a magneto-optical disk, an IC card, and a semiconductor storage device.

[0144] Applications of a movie encoding method (image encoding method) and a movie decoding method (image decoding method) according to each of the Embodiments, and a system using such applications, will be described below. Features of the system include an image encoding device using an image encoding method, and an image encoding and decoding device including an image decoding device using an image decoding method. Other system configurations can be changed as appropriate depending on the case.

[0145] FIG. 16 illustrates a general configuration of a content delivery system ex100 for implementing content distribution services. The area for providing communication services is divided into cells of the required size, and each cell houses base stations ex106-ex110, which are fixed wireless stations.

[0146] The content delivery system ex100 connects to devices, such as a computer ex111, a personal digital assistant (PDA) ex112, a camera ex113, a cell phone ex114, and a gaming device ex115, via the Internet ex101, an Internet service provider ex102, a telephone network ex104, and basic stations ex106-ex110.

[0147] However, the configuration of the content providing system ex100 is not limited to the configuration shown in FIG. 16, and a combination in which any elements are connected is acceptable. In addition, each of the devices can be connected to the ex104 telephone network directly, and not through ex106-ex110 base stations, which are fixed wireless stations. In addition, devices can be interconnected with each other through near field communication and others.

[0148] The camera ex113, such as a digital video camera, is capable of capturing moving images. The ex116 camera, such as a digital video camera, can capture both still images and moving images. In addition, the ex114 cellular telephone may be a telephone that complies with any of the standards, for example, Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (W-CDMA), Long Range development (LTE) and High Speed Packet Access (HSPA). Alternatively, the ex114 cellular telephone may be a Personal Handheld System (PHS).

[0149] In the content delivery system ex100, the streaming server ex103 is connected to the camera ex113 and others through the telephone network ex104 and the base station ex109, allowing distribution of live broadcasts and others. For such distribution, content (eg, video of a live music show) captured by a user using camera ex113 is encoded as described above in each of the Embodiments, and the encoded content is transmitted to streaming server ex103. On the other hand, the streaming server ex103 streams received content data to clients based on their requests. Clients include computer ex111, PDA ex112, camera ex113, cell phone ex114, and game device ex115, which are capable of decoding the above-mentioned encoded data. Each of the devices that has received the distributed data decodes and reproduces the encoded data (ie, functions as an image decoding device in accordance with an aspect of the present invention).

[0150] The captured data may be encoded by camera ex113 or a streaming server ex103 that transmits the data, or the encoding processes may be shared between camera ex113 and streaming server ex103. Likewise, the distributed data may be decoded by clients or the streaming server ex103, or the decoding processes may be shared between the clients and the streaming server ex103. In addition, still image data and moving images captured not only by the ex113 but also by the ex116 can be transmitted to the streaming server ex103 via the ex111 computer. Encoding processes can be performed by or shared between the ex116 camera, ex111 computer, or ex103 streaming server.

[0151] In addition, typically a computer ex111 and an LSI ex500 included in each of the devices perform such encoding and decoding processes. The LSI ex500 can be configured as a single chip or multiple chips. The movie encoding and decoding software may be built into some type of recording medium (eg, CD-ROM, floppy disk, hard disk) that is read by computer ex111 and others, and the encoding and decoding processes may be performed using this software. In addition, when the ex114 cell phone is equipped with a camera, video data captured by the camera can be transmitted. The video data is LSI ex500 encoded data included in the ex114 cell phone.

[0152] In addition, the streaming server ex103 may be composed of servers and computers and may decentralize data and process decentralized data, record or distribute data.

[0153] As described above, clients can receive and reproduce encoded data in the content delivery system ex100. In other words, clients can receive and decode information transmitted by the user, and play back the decoded data in real time in the content provision system ex100, so that a user who does not have any specific right and equipment can realize personal broadcasting.

[0154] The present invention is not limited to the above-mentioned content providing system ex100, and at least either a movie encoding device (image encoding device) or a movie decoding device (image decoding device) described in each of Embodiments may be included in the digital broadcast, which is shown in FIG. 17. More precisely, the broadcast station ex201 reports or transmits via radio waves to the broadcast satellite ex202 multiplexed data obtained by multiplexing video data with audio data. The video data is data encoded in accordance with the movie encoding method described in each of the Embodiments (ie, data encoded by an image encoding apparatus according to an aspect of the present invention). When receiving video data, the ex202 broadcast satellite transmits radio waves for broadcast. Then the ex204 household antenna, capable of receiving satellite broadcasts, receives radio waves. A device, such as a television ex300 (receiver) and set top box ex217 (STB), decodes the received multiplexed data and reproduces the data (ie, functions as an image decoding device in accordance with an aspect of the present invention).

[0155] In addition, a reader/writer ex218 that (i) reads and decodes multiplexed data recorded on recording media ex215, such as DVD and BD, or (ii) encodes video signals on recording media ex215 and in some cases writes data, obtained by multiplexing an audio signal onto encoded data may include a movie decoding device or a movie encoding device, which are shown in each of the Embodiments. In this case, the playback video signals are displayed on the monitor ex219, and another device or system can play back the video signals using the recording medium ex215 on which the multiplexed data is recorded. It is also possible to implement a movie decoding device in a set top box ex217 connected to a cable ex203 for cable television or to an antenna ex204 for satellite and/or terrestrial broadcasting to display video signals on a monitor ex219 of an ex300 television. The movie decoding device can be included not in the set-top box, but in the ex300 TV.

[0156] FIG. 18 illustrates a TV ex300 that uses the movie encoding method and the movie decoding method described in each of the Embodiments. The ex300 TV includes: ex301 tuner, which receives or provides multiplexed data obtained by multiplexing audio data and video data through antenna ex204 or cable ex203, etc., which receives broadcast; a modulation/demodulation unit ex302 that demodulates the received multiplexed data or modulates the data into multiplexed data to supply it to the outside of the unit; and a multiplexing/demultiplexing unit ex303 that demultiplexes the modulated multiplexed data into video data and audio data, or multiplexes the video data and audio data encoded by the signal processing unit ex306 into data.

[0157] In addition, the TV ex300 further includes: a signal processing unit ex306, including an audio signal processing unit ex304 and a video signal processing unit ex305, which decode audio data and video data and encode audio data and video data, respectively (which function as an image encoding device or an image decoding device in accordance with an aspect of the present invention); loudspeaker ex307, which provides decoded audio signal; and an output portion ex309 including a display portion ex308 that displays the decoded video signal, such as a display. In addition, the TV ex300 includes an interface unit ex317 including an operation input unit ex312 that receives user action input. In addition, the television ex300 includes a control unit ex310, which generally controls each component element of the television ex300, and a power circuit ex311, which supplies power to each of the elements. In addition to the operation input node ex312, the interface node ex317 may include: a bridge ex313, which connects to an external device, such as a reader/writer ex218; ex314 connector for connecting ex216 recording media, such as an SD card; ex315 drive, which needs to be connected to an external recording medium, such as a hard drive; and an ex316 modem that needs to be connected to the telephone network. Here, the recording medium ex216 can electrically record information using a non-volatile/volatile semiconductor storage memory element. The components of the ex300 TV are connected to each other via a synchronous bus.

[0158] First, a configuration in which the TV ex300 decodes multiplexed data received externally through antenna ex204 and others and reproduces the decoded data will be described. In the TV ex300, as a result of receiving user action from the remote control ex220 and others, the multiplexing/demultiplexing unit ex303 demultiplexes the multiplexed data demodulated by the modulation/demodulation unit ex302 under the control of the control unit ex310 including the CPU. In addition, in the TV ex300, the audio signal processing unit ex304 decodes the demultiplexed audio data, and the video signal processing unit ex305 decodes the demultiplexed video data using the decoding method described in each of the Embodiments. The output node ex309 provides the decoded video signal and audio signal to the outside. When the output node ex309 provides a video signal and an audio signal, the signals can be temporarily stored in buffers ex318 and ex319 and others so that the signals are played back in sync with each other. In addition, the ex300 TV can read multiplexed data not through broadcasting etc., but from ex215 and ex216 recording media such as magnetic disk, optical disk and SD card. Next, a configuration in which the ex300 TV encodes an audio signal and a video signal and transmits the data outside or writes the data to a recording medium will be described. In the TV ex300, as a result of receiving the user's action from the remote control ex220 and others, the audio signal processing unit ex304 encodes the audio signal, and the video signal processing unit ex305 encodes the video signal, under the control of the control unit ex310 using the encoding method that is described in each of the Embodiments. The multiplexing/demultiplexing unit ex303 multiplexes the encoded video signal and the audio signal and provides the resulting signal to the outside. When the multiplexing/demultiplexing unit ex303 multiplexes a video signal and an audio signal, the signals can be temporarily stored in buffers ex320 and ex321 and others so that the signals are reproduced in sync with each other. Here, buffers ex318-ex321 may be multiple as illustrated, or at least one buffer may be shared in TV ex300. In addition, data may be stored in a buffer other than buffers ex318-ex321 so that system overflows and underflows, for example, between the modulation/demodulation node ex302 and the multiplexing/demultiplexing node ex303 can be avoided.

[0159] In addition, the ex300 television may include a configuration for receiving audiovisual input data from a microphone or camera, in addition to a configuration for receiving audio data and video data from a broadcast or recording medium, and may encode the received data. Although the ex300 TV is described as being able to encode, multiplex and provide data to the outside, it may not allow all processes to be carried out, but only allow one of receiving, decoding and providing data to the outside.

[0160] In addition, when the reader/writer ex218 reads or writes multiplexed data from or to the recording medium, one of the TV ex300 and the reader/writer ex218 can decode or encode the multiplexed data, and the TV ex300 and the reader/writer ex218 can distribute decoding or encoding.

[0161] As an example, FIG. 19 illustrates the configuration of the information reproducing/recording portion ex400 when data is read or written from or to the optical disc. The information reproducing/recording portion ex400 includes components ex401 to ex407, which will be described below. The optical head ex401 emits a laser spot onto a recording surface of the recording medium ex215, which is an optical disk, for recording information, and detects reflected light from the recording surface of the recording medium ex215 for reading information. The modulation recording unit ex402 electrically controls the semiconductor laser included in the optical head ex401 and modulates the laser light in accordance with the recorded data. The playback demodulation section ex403 amplifies the playback signal obtained by electrically detecting reflected light from the recording surface using a photodetector included in the optical head ex401, and demodulates the playback signal by separating the signal component recorded on the recording medium ex215 to play back the required information. Buffer ex404 temporarily stores information to be recorded on recording medium ex215 and information reproduced from recording medium ex215. The disk motor ex405 rotates the recording media ex215. The servo control unit ex406 moves the optical head ex401 to a preset information track while controlling the rotation of the disk motor ex405 to follow the laser spot. The system control portion ex407 controls the information reproducing/recording portion ex400 as a whole. The reading and writing processes can be implemented by a system control unit ex407 that uses various information stored in the buffer ex404 and generates and adds new information as needed, and a modulation recording unit ex402, a playback demodulation unit ex403, and a servo control unit ex406 that record and playback information via the optical head ex401, while being controlled in a consistent manner. The system control portion ex407 includes, for example, a microprocessor and performs processing by causing a computer to execute a program for reading and writing.

[0162] Although the optical head ex401 is described as emitting a laser spot, it can perform high-density recording using near-field light.

[0163] FIG. 20 schematically illustrates a recording medium ex215, which is an optical disc. On the recording surface of the recording medium ex215, guide grooves are formed in a spiral pattern, and the information track ex230 pre-records address information indicating an absolute position on the disk according to a change in the shape of the guide grooves. The address information includes information for determining positions of recording blocks ex231, which are a unit for recording data. The device that records and reproduces data reproduces the information track ex230 and reads the address information to determine the positions of the recording blocks. In addition, the recording medium ex215 includes a data recording area ex233, an inner circumference area ex232, and an outer circumference area ex234. The data recording area ex233 is an area to be used when writing user data. The inner circle area ex232 and the outer circle area ex234, which are inside and outside the data recording area ex233, respectively, are intended for special uses other than user data recording. The information reproducing/recording portion ex400 reads and writes encoded audio data, encoded video data, or multiplexed data obtained by multiplexing the encoded audio data and encoded video data from and to the data recording area ex233 in the recording medium ex215.

[0164] Although the description describes as an example an optical disc containing a layer such as DVD and BD, the optical disc is not limited to such and may be an optical disc having a multi-layer structure and capable of being recorded on portions other than the surface. In addition, the optical disc may be structured for multi-dimensional recording/reproduction, such as recording information using light with colors of different wavelengths in the same part of the optical disc and recording information having different layers from different angles.

[0165] In addition, in the digital broadcast system ex200, the vehicle ex210 having the antenna ex205 can receive data from the satellite ex202 and others and display the video image on a display device, such as a car navigation system ex211 installed in the vehicle ex210. Here, the configuration of the car navigation system ex211 will be a configuration, for example, including a GPS receiving unit in the configuration illustrated in FIG. 18. The same will be true for the configuration of the ex111 computer, ex114 cell phone and others.

[0166] FIG. 21A illustrates a cellular phone ex114 that uses the movie encoding method and the movie decoding method described in each of the Embodiments. The ex114 cell phone includes: an ex350 antenna for transmitting and receiving radio waves via the ex110 base station; ex365 camera, capable of capturing moving and still images; and a display unit ex358, such as a liquid crystal display for displaying data, such as a decoded video image captured by the camera ex365 or received by the antenna ex350. The ex114 cell phone further includes: a housing including a set of ex366 control buttons; an audio output unit ex357, such as a speaker for audio output; an audio input unit ex356, such as a microphone for audio input; ex367 storage device for storing captured video or still images, recorded audio, encoded or decoded data of received video, still images, emails, or the like; and an ex364 connector, which is an interface node for a recording medium that stores data in the same way as an ex367 storage device.

[0167] Next, a configuration example of the cellular phone ex114 will be described with reference to FIG. 21B. In the cellular phone ex114, the main control node ex360, configured to control the whole of each housing node, including the display node ex358, as well as the control buttons ex366, is mutually connected via a synchronous bus ex370 to the power circuit ex361, the operation input control node ex362, the processing node ex355 video signal, camera interface node ex363, liquid crystal display (LCD) control node ex359, modulation/demodulation node ex352, multiplexing/demultiplexing node ex353, audio signal processing node ex354, connector ex364 and memory ex367.

[0168] When the end call key or the power key is pressed as a result of user action, the power circuit ex361 supplies the corresponding nodes with power from the battery to activate the cell phone ex114.

[0169] In the cellular phone ex114, the audio processing unit ex354 converts the audio signals collected by the audio input unit ex356 in the speech conversion mode into digital audio signals under the control of the main control unit ex360 including the CPU, ROM and RAM. Modulation/demodulation unit ex352 then performs spread spectrum processing on the digital audio signals, and transmit/receive unit ex351 performs digital-to-analog conversion and frequency conversion on the data to transmit the resulting data through antenna ex350. The modulation/demodulation section ex352 then performs despreading processing on the data, and the audio signal processing section ex354 converts them into analog audio signals to output them through the audio output section ex357.

[0170] In addition, when email is transmitted in the data transfer mode, email text data input by operating control buttons ex366 and others in the housing is sent to the main control node ex360 via the operation input control node ex362. The main control node ex360 causes the modulation/demodulation node ex352 to perform spread spectrum processing on the text data, and the transmit and receive node ex351 performs digital-to-analog conversion and frequency conversion on the result data to transmit the data to the base station ex110 through the antenna ex350. When email is received, processing that is approximately the reverse of that for email transmission is performed on the received data, and the resulting data is provided to display node ex358.

[0171] When video, still images, or video and audio are transmitted in a data transfer mode, the video processing unit ex355 (functioning as an image encoding device according to an aspect of the present invention) compresses and encodes the video signals received from the camera ex365 using a movie encoding method, shown in each of the Embodiments, and transmits the encoded video data to the multiplexing/demultiplexing node ex353. In contrast, while camera ex365 captures video images, still images and others, the audio signal processing section ex354 encodes the audio signals collected by the audio input section ex356 and transmits the encoded audio data to the multiplexing/demultiplexing section ex353.

[0172] The multiplexing/demultiplexing unit ex353 multiplexes encoded video data received from the video processing unit ex355 and encoded audio data received from the audio processing unit ex354 using a predetermined method. Next, the modulation/demodulation unit ex352 performs spread spectrum processing on the multiplexed data, and the transmitting and receiving unit ex351 performs digital-to-analog conversion and frequency conversion on the data to transmit the resulting data through antenna ex350.

[0173] When receiving video file data that is associated with a web page and others in data mode, or when receiving email with video and/or audio attached, to decode the multiplexed data received through antenna ex350, the multiplexing/demultiplexing node ex353 demultiplexes multiplexes the data into a video bit stream and an audio bit stream, and supplies the video processing unit ex355 with encoded video data and the audio processing unit ex354 with encoded audio data via synchronous bus ex370. The video signal processing unit ex355 (functioning as an image decoding device according to an aspect of the present invention) decodes the video signal using a movie decoding method corresponding to the movie encoding method shown in each of the Embodiments, and then the display unit ex358 displays, for example, video images and still images. images included in a video file associated with a web page by an LCD control node ex359. In addition, the audio signal processing unit ex354 decodes the audio signal, and the audio output unit ex357 provides the audio.

[0174] In addition, similar to the TV ex300, the terminal, such as the cellular phone ex114, may have 3 types of implementation configurations, including not only (i) a transmitting and receiving terminal including an encoding device and a decoding device, but also (ii) a transmitting terminal including only an encoding device, and (iii) a receiving terminal including only a decoding device. Although the digital broadcast system ex200 is described as receiving and transmitting multiplexed data obtained by multiplexing video data with audio data, the multiplexed data may be data obtained by multiplexing video data not with audio data but with character data related to a video image, and may not be multiplexed data but itself video data.

[0175] As such, the movie encoding method or the movie decoding method in each of the Embodiments can be used on any of the described devices and systems. Thus, the advantages described in each of the Embodiments can be obtained.

[0176] Moreover, the present invention is not limited to the Embodiments, and various modifications and changes are possible without deviating from the scope of the present invention.

[0177] (Embodiment 4)

Video data may be generated by switching, as necessary, between (i) a movie encoding method or movie encoding apparatus shown in each of the Embodiments, and (ii) a movie encoding method or movie encoding apparatus in accordance with another standard, such as MPEG-2, MPEG4-AVC and VC-1.

[0178] Here, when a plurality of video data that correspond to different standards is generated and then decoded, it is necessary to select decoding methods that correspond to different standards. However, since it cannot be determined which standard each of the plurality of video data to be decoded corresponds to, there is a problem that a suitable decoding method cannot be selected.

[0179] To solve this problem, multiplexed data obtained by multiplexing video data with audio data and others has a structure including identification information indicating which standard the video data conforms to. Below, a specific structure of multiplexed data including video data generated by the movie encoding method or the movie encoding apparatus shown in each of the Embodiments will be described. The multiplexed data is a digital stream in MPEG-2 transport stream format.

[0180] FIG. 22 illustrates the structure of multiplexed data. As illustrated in FIG. 22, multiplexed data can be obtained by multiplexing at least one of a video stream, an audio stream, a demonstration graphics (PG) stream, and an interactive graphics stream. The video stream represents the primary video and secondary video of the movie, the audio stream represents the primary audio and the secondary audio to be mixed with the primary audio, and the demo graphics stream represents the subtitles of the movie. Here, the primary video image is a normal video image shown on the screen, and the secondary video image is a video image to be shown on a smaller window in the main video image. In addition, the interactive graphics stream represents the interactive screen that must be formed by placing GUI components on the screen. The video stream is encoded by a movie encoding method or movie encoder shown in each of the Embodiments, or by a movie encoding method or movie encoder in accordance with a conventional standard such as MPEG-2, MPEG4-AVC and VC-1. The audio stream is encoded according to a standard such as Dolby-AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD and Linear PCM.

[0181] Each stream included in the multiplexed data is identified by a PID. For example, 0x1011 is assigned to the video stream that will be used for movie video, 0x1100-0x111F are assigned to audio streams, 0x1200-0x121F are assigned to demo graphics streams, 0x1400-0x141F are assigned to interactive graphics streams, 0 ×1B00-0×1B1F are assigned to the video streams that will be used for the secondary video of the movie, and 0×1A00-0×1A1F are assigned to the audio streams that will be used for the secondary video to be mixed with the primary audio.

[0182] FIG. 23 schematically illustrates how data is multiplexed. First, video stream ex235 consisting of video frames and audio stream ex238 consisting of audio frames are converted into PES packet stream ex236 and PES packet stream ex239, and further into TS packets ex237 and TS packets ex240, respectively. Likewise, data from demo graphics stream ex241 and data from interactive graphics stream ex244 are converted to PES packet stream ex242 and PES packet stream ex245, and further to TS packets ex243 and TS packets ex246, respectively. These TS packets are multiplexed into a stream to produce ex247 multiplexed data.

[0183] FIG. 24 illustrates in more detail how the video stream is stored in the PES packet stream. The first stripe in Fig. 24 shows a stream of video frames in a video stream. The second bar shows the flow of PES packets. As indicated by the arrows labeled yy1, yy2, yy3 and yy4 in FIG. 24, the video stream is divided into images in the form of I-pictures, B-pictures and P-pictures, each of which is a video presentation unit, and the pictures are stored in the payload of each of the PES packets. Each of the PES packets has a PES header, and the PES header stores a Playback Time Stamp (PTS) indicating the display time of the picture, and a Decoding Time Stamp (DTS) indicating the decoding time of the picture.

[0184] FIG. 25 illustrates the format of TS packets that are ultimately to be written to multiplexed data. Each of the TS packets is a 188-byte fixed-length packet, including a 4-byte TS header containing information such as a PID to identify the flow, and a 184-byte TS payload for data storage. PES packets are split and stored in TS payloads. When a BD ROM is used, each of the TS packets is given a 4-byte TP_Extra_Header, resulting in 192-byte original packets. The original packets are written into the multiplexed data. TP_Extra_Header stores information such as Arrival_Time_Stamp (ATS). The ATS indicates the transmission start time at which each of the TS packets needs to be transmitted to the PID filter. The numbers increasing from the beginning of the multiplexed data are called source packet numbers (SPNs), as shown at the bottom of FIG. 25.

[0185] Each of the TS packets included in the multiplexed data includes not only audio, video, subtitle and other streams, but also a Program Association Table (PAT), a Program Mapping Table (PMT), and a Program Reference Signal (PCR). The PAT indicates that the PID in the PMT used in the multiplexed data indicates, and the PID of the PAT itself is registered as zero. The PMT stores the PID of video, audio, subtitle and other streams included in the multiplexed data, and information about the attributes of the streams corresponding to the PID. The PMT also contains various descriptors related to the multiplexed data. The descriptors have information, such as copy control information, indicating whether copying of the multiplexed data is permitted. The PCR stores STC timing information corresponding to the ATS indicating when a PCR packet is transmitted to the decoder to achieve synchronization between the Arrival Timer (ATC), which is the time axis of the ATS, and the System Timer (STC), which is the time axis of the PTS and DTS.

[0186] FIG. 26 illustrates the PMT data structure in detail. The PMT header is located at the beginning of the PMT. The PMT header describes the length of data included in the PMT, etc. Following the PMT header are a number of descriptors related to the multiplexed data. Descriptors describe information, such as copy control information. Following the descriptors are multiple pieces of flow information related to the flows included in the multiplexed data. Each piece of stream information includes stream descriptors describing information such as stream type to identify the stream's compression codec, stream PID, and stream attribute information (eg, frame rate or aspect ratio). Stream descriptors are equal in number to the number of streams in the multiplexed data.

[0187] When multiplexed data is recorded on a recording medium and others, it is recorded together with multiplexed data information files.

[0188] Each of the multiplexed data information files is multiplexed data management information, as shown in FIG. 27. The multiplexed data information files are in one-to-one correspondence with the multiplexed data, and each of the files includes multiplexed data information, stream attribute information, and an input map.

[0189] As illustrated in FIG. 27, information about the multiplexed data includes a system speed, a playback start time, and a playback end time. The system rate indicates the maximum bit rate at which the system's final decoder, which will be described later, transmits the multiplexed data to the PID filter. The ATS intervals included in the multiplexed data are set to no higher than the system rate. The playback start time indicates the PTS in the video frame at the beginning of the multiplexed data. A one-frame interval is added to the PTS in the video frame at the end of the multiplexed data, and the PTS is set at the time the playback ends.

[0190] As shown in FIG. 28, a piece of attribute information is registered in the stream attribute information for each PID of each stream included in the multiplexed data. Each piece of attribute information contains different information depending on whether the corresponding stream is a video stream, an audio stream, a demo graphics stream, or an interactive graphics stream. Each piece of video stream attribute information carries information including what kind of compression codec is used to compress the video stream, and the resolution, aspect ratio, and frame rate of the pieces of image data that are included in the video stream. Each piece of audio stream attribute information carries information including what kind of compression codec is used to compress the audio stream, how many channels are included in the audio stream, what language the audio stream supports, and what the sampling rate is. Video stream attribute information and audio stream attribute information are used to initialize the decoder before the player plays the information.

[0191] In Embodiment 4, the multiplexed data to be used belongs to a stream type included in the PMT. In addition, when multiplexed data is recorded on a recording medium, video stream attribute information included in the multiplexed data information is used. More precisely, the movie encoding method or the movie encoding device described in each of the Embodiments includes a step or unit for assigning unique information indicating video data generated by the movie encoding method or the movie encoding device in each of the Embodiments to a type stream included in the PMT, or video stream attribute information. With this structure, video data generated by the movie encoding method or the movie encoding apparatus described in each of the Embodiments can be distinguished from video data that conforms to another standard.

[0192] Additionally, FIG. 29 illustrates steps of a movie decoding method according to Embodiment 4. In step exS100, the stream type included in the PMT or video stream attribute information is obtained from the multiplexed data. Next, in step exS101, it is determined whether the stream type or the video stream attribute information indicates that multiplexed data is generated by a movie encoding method or a movie encoding apparatus in each of the Embodiments. When it is determined that the stream type or video stream attribute information indicates that the multiplexed data is generated by the movie encoding method or the movie encoding apparatus in each of Embodiments, then in step exS102, the stream type or video stream attribute information is decoded by the movie decoding method in each of the Options. In addition, when the stream type or video stream attribute information indicates compliance with traditional standards such as MPEG-2, MPEG4-AVC and VC-1, in step exS103, the stream type or video stream attribute information is decoded according to a movie decoding method in accordance with traditional standards. .

[0193] As such, assigning a new unique value to the stream type or video stream attribute information makes it possible to determine whether the movie decoding method or the movie decoding apparatus that is described in each of the Embodiments can perform decoding. Even when inputting multiplexed data that conforms to a different standard, you can select the appropriate decoding method or device. Thus, it becomes possible to decode the information without any error. In addition, the movie encoding method or apparatus or the movie decoding method or apparatus in Embodiment 4 can be used in the devices and systems described above.

[0194] (Embodiment 5)

Each of the movie encoding method, movie encoding device, movie decoding method, and movie decoding device in each of the Embodiments is generally implemented as an integrated circuit or Large Scale Integrated Circuit (LSI). As an example of the LSI of FIG. 30 illustrates an LSI ex500 configuration that is made into a single chip. LSI ex500 includes elements ex501, ex502, ex503, ex504, ex505, ex506, ex507, ex508 and ex509, which will be described below, and these elements are connected to each other via bus ex510. The power circuit ex505 is activated by supplying power to each of the elements when the power circuit ex505 is turned on.

[0195] For example, when encoding is performed, LSI ex500 receives an audio-visual signal from microphone ex117, camera ex113 and others via audio-visual I/O ex509 under control of control node ex501 including CPU ex502, memory controller ex503, flow controller ex504 and node ex512 excitation frequency adjustment. The received audio-visual signal is temporarily stored in an external storage device ex511, such as SDRAM. Under the control of the control node ex501, the stored data is segmented into data pieces according to the volume and processing speed for transmission to the signal processing node ex507. Then, the signal processing unit ex507 encodes the audio signal and/or video signal. Here, the encoding of the video signal is the encoding described in each of the Embodiments. In addition, the signal processing unit ex507 sometimes multiplexes the encoded audio data and the encoded video data, and the streaming I/O ex506 provides the multiplexed data to the outside. The provided multiplexed data is transmitted to the base station ex107 or recorded on recording media ex215. When data sets are multiplexed, the data sets should be temporarily stored in buffer ex508 so that the data sets are synchronized with each other.

[0196] Although storage device ex511 is an element outside of LSI ex500, it may be included in LSI ex500. The ex508 buffer is not limited to one buffer, but can consist of buffers. In addition, the LSI ex500 can be converted into a single chip or multiple chips.

[0197] In addition, although the control unit ex501 includes a CPU ex502, a memory controller ex503, a flow controller ex504, a drive frequency adjustment unit ex512, the configuration of the control unit ex501 is not limited to such. For example, the signal processing unit ex507 may further include a CPU. Including another CPU in the signal processing node ex507 can improve processing speed. Moreover, as another example, CPU ex502 may include a signal processing unit ex507 or an audio signal processing unit that is part of the signal processing unit ex507. In such a case, the control unit ex501 includes a signal processing unit ex507 or a CPU ex502 including a portion of the signal processing unit ex507.

[0198] The name used here is LSI, but it may also be called system LSI, super-LSI, or ultra-LSI depending on the degree of integration.

[0199] In addition, methods for achieving integration are not limited to LSI, and a special circuit or a general-purpose processor, etc. can also provide integration. A field programmable gate array (FPGA), which can be programmed after LSI production, or a reconfigurable processor, which allows interconnect reconfiguration or LSI configuration, can be used for the same purpose. The programmable logic device generally can execute the movie encoding method and the movie decoding method in accordance with the Embodiments and Variants by loading or reading from a storage device a program included in software or firmware.

[0200] In the future, as semiconductor technology advances, a completely new technology may replace LSI. Functional blocks can be integrated using such technology. It is likely that the present invention is applicable to biotechnology.

[0201] (Embodiment 6)

When video data generated by the movie encoding method or the movie encoding apparatus described in each of the Embodiments is decoded, compared with the case of decoding video data that conforms to the traditional standard such as MPEG-2, MPEG4-AVC and VC-1, the amount of computation is likely to increase. Therefore, the LSI ex500 must be set to a higher drive frequency than the CPU ex502, which must be used when video data is decoded according to the traditional standard. However, when the driving frequency is set higher, there is a problem that power consumption increases.

[0202] To solve this problem, a movie decoding device, such as the TV ex300 and LSI ex500, is configured to determine which standard the video data conforms to and switch between drive frequencies in accordance with the determined standard. Fig. 31 illustrates the configuration of ex800 in Embodiment 6. The driving frequency switching section ex803 sets the driving frequency to a higher driving frequency when video data is generated by the movie encoding method or the movie encoding apparatus described in each of the Embodiments. Then, the drive frequency switching portion ex803 instructs the decoding processing portion ex801, which executes the movie decoding method described in each of the Embodiments, to decode the video data. When the video data conforms to the conventional standard, the driving frequency switching section ex803 sets the driving frequency to a lower driving frequency than that of the video data generated by the movie encoding method or the movie encoding apparatus described in each of the Embodiments. Then, the drive frequency switching portion ex803 instructs the decoding processing portion ex802, which conforms to the conventional standard, to decode the video data.

[0203] More precisely, the driving frequency switching unit ex803 includes the CPU ex502 and the driving frequency adjusting unit ex512 in FIG. 30. Here, each of the decoding processing unit ex801, which performs the movie decoding method described in each of the Embodiments, and the decoding processing unit ex802, which conforms to the conventional standard, corresponds to the signal processing unit ex507 in FIG. 30. CPU ex502 determines which standard the video data conforms to. Then, the drive frequency adjustment portion ex512 determines the drive frequency based on the signal from CPU ex502. In addition, the signal processing unit ex507 decodes video data based on the signal from CPU ex502. For example, the identification information described in Embodiment 4 is likely used to identify video data. The identification information is not limited to that described in Embodiment 4, but may be any information as long as the information indicates which standard the video data conforms to. For example, when it is possible to determine which standard the video data conforms to based on an external signal to determine that video data is used for television or disc, etc., the determination can be made based on such an external signal. In addition, CPU ex502 selects the drive frequency based on, for example, a lookup table in which video data standards are associated with drive frequencies, as shown in FIG. 33. The driving frequency can be selected by storing a lookup table in the ex508 buffer and LSI internal memory, and by accessing the lookup table using the ex502 CPU.

[0204] FIG. 32 illustrates steps for performing the method in Embodiment 6. First, in step exS200, the signal processing unit ex507 obtains identification information from the multiplexed data. Next, in step exS201, CPU ex502 determines whether video data is generated based on identification information using the encoding method and encoding apparatus described in each of the Embodiments. When video data is generated by the encoding method or encoding apparatus described in each of the Embodiments, in step exS202, CPU ex502 transmits a signal to the drive frequency adjusting unit ex512 to set the drive frequency to a higher drive frequency. Then, the drive frequency adjusting portion ex512 sets the drive frequency to a higher drive frequency. On the other hand, when the identification information indicates that the video data conforms to a traditional standard such as MPEG-2, MPEG4-AVC and VC-1, in step exS203, the CPU ex502 transmits a signal to the drive frequency adjustment unit ex512 to set the drive frequency to a lower drive frequency. Then, the driving frequency adjusting section ex512 sets the driving frequency to a lower driving frequency than in the case where video data is generated by the encoding method or encoding apparatus described in each of the Embodiments.

[0205] In addition, along with switching driving frequencies, the power saving effect can be enhanced by changing the voltage to be supplied to the LSI ex500 or a device including the LSI ex500. For example, when the driving frequency is set lower, the voltage that must be supplied to the LSI ex500 or a device including the LSI ex500 is likely set to a voltage lower than in the case where the driving frequency is set higher.

[0206] In addition, as a method for setting the driving frequency, when the amount of calculation for decoding is larger, the driving frequency can be set higher, and when the amount of calculation for decoding is smaller, the driving frequency can be set lower. Therefore, the installation method is not limited to the methods described above. For example, when the amount of calculation for decoding video data in accordance with MPEG4-AVC is greater than the amount of calculation for decoding video data generated by the movie encoding method or the movie encoding apparatus described in each of the Embodiments, the driving frequency is most likely set in the reverse order to that described above installation.

[0207] Moreover, the method for setting the driving frequency is not limited to the method for setting the driving frequency lower. For example, when the identification information indicates that video data is generated by the movie encoding method or the movie encoding apparatus described in each of the Embodiments, the voltage to be supplied to the LSI ex500 or a device including the LSI ex500 is likely set higher. When the identification information indicates that the video data conforms to a traditional standard such as MPEG-2, MPEG4-AVC and VC-1, the voltage to be supplied to the LSI ex500 or a device including the LSI ex500 is likely set lower. As another example, when the identification information indicates that the video data is generated by the movie encoding method or the video encoding apparatus described in each of the Embodiments, driving of the CPU ex502 most likely does not need to be suspended. When the identification information indicates that the video data conforms to the traditional standard, such as MPEG-2, MPEG4-AVC and VC-1, the operation of the CPU ex502 is likely to be suspended at a given time because the CPU ex502 has excess processing capacity. Even when the identification information indicates that the video data is generated by the movie encoding method or by the movie encoding apparatus described in each of the Embodiments, in the case where the CPU ex502 has excess processing capacity, the operation of the CPU ex502 is likely to be suspended at a given time. In such a case, the pause time is likely to be set shorter than in the case where the identification information indicates that the video data conforms to a traditional standard such as MPEG-2, MPEG4-AVC and VC-1.

[0208] Accordingly, it is possible to enhance the power saving effect by switching between driving frequencies in accordance with the standard to which the video data conforms. In addition, when the LSI ex500 or a device including the LSI ex500 is driven using a battery, the operating time of the batteries can be extended along with an energy saving effect.

[0209](Embodiment 7)

There are cases where a variety of video data that conform to different standards are provided to devices and systems such as televisions and cell phones. To be able to decode multiple video data that conform to different standards, the ex507 signal processing node in the LSI ex500 must comply with different standards. However, there are problems with increasing the scale of the LSI ex500 circuit and increasing the cost when separately using ex507 signal processing nodes that comply with the relevant standards.

[0210] To solve these problems, a configuration is provided in which a decoding processing unit for implementing the movie decoding method described in each of the Embodiments, and a decoding processing unit that conforms to a conventional standard such as MPEG-2, MPEG4-AVC and VC- 1, are partially used together. Ex900 in fig. 34A shows an example of this configuration. For example, the movie decoding method described in each of the Embodiments and the movie decoding method that complies with MPEG4-AVC share some of the same processing details, such as entropy encoding, inverse quantization, deblocking filtering, and motion compensation. The processing details that must be shared most likely include the use of a decoding processing unit ex902, which corresponds to MPEG4-AVC. In contrast, the dedicated decoding processing unit ex901 may be used for other processing that is unique to the present invention and not compliant with MPEG-4 AVC. A decoding processing unit for implementing the movie decoding method described in each of the Embodiments may be shared for processing to be distributed, and a dedicated decoding processing unit may be used for processing unique to MPEG4-AVC processing.

[0211] Additionally, ex1000 in FIG. 34B shows another example in which processing is partially distributed. This example uses a configuration including a dedicated decoding processing unit ex1001 that supports processing unique to an aspect of the present invention, a specialized decoding processing unit ex1002 that supports processing unique to another legacy standard, and a decoding processing unit ex1003 that supports processing that is desired allocated between the movie decoding method in the aspect of the present invention and the conventional movie decoding method. Here, the specialized decoding processing nodes ex1001 and ex1002 are not necessarily specialized for processing according to the aspect of the present invention and conventional standard processing, but may be nodes capable of implementing general processing. In addition, the configuration of Embodiment 7 can be implemented using LSI ex500.

[0212] As such, reducing the scale of the LSI circuit and reducing costs is possible by sharing a decoding processing unit for processing to be shared between the movie decoding method of the present invention and the movie decoding method according to the conventional standard.

[Industrial applicability]

[0213] The present invention has the advantage of speeding up processing with almost no reduction in coding efficiency and is applicable to various uses such as data storage, transmission and exchange. The present invention is applicable, for example, to information display devices and image capturing devices such as televisions, digital video recorders, car navigation systems, mobile phones, digital cameras and digital video cameras, and is very suitable for practical use.

[List of designators]

[0214] 100 Image encoder

101 Blocking node

102 Subtraction node

103 Conversion node

104 Variable Length Coding Unit

105, 202 Inverse conversion node

106, 203 Addition unit

107, 204 SAO processing unit

108, 206 Frame memory

109, 207 Prediction node

110 Parameter coding unit SAO

111 Coding node sao_type_idx

112 Coding node sao_offset

113 Sao_offset to binary conversion node

114 Arithmetic coding switching unit

115 Context arithmetic encoding node 0

116 Context arithmetic encoding node 1

117 Bypass arithmetic coding unit

200 Image decoding device

201 Variable Length Decoding Unit

205 Unit for combining decoded blocks

210 SAO parameter decoding node

211 Decoding node sao_type_idx

212 Decoding node sao_offset

213 Arithmetic decoding switching node

214 Context arithmetic decoding node 0

215 Context 1 arithmetic decoding node

216 Bypass arithmetic decoding unit

217 Inverse conversion node sao_offset from binary form

Claims (10)

A method of encoding an adaptive sample offset (SAO) value using arithmetic encoding into a bitstream, the encoding method comprising the steps of: dividing the image to be encoded into a plurality of blocks (S101) to form target blocks; generating a predicted block (S102); subtracting the predicted block from the target block (S104) to form a residual block; converting and quantizing the residual block to form a quantized block (S104); performing inverse quantization and inverse transformation of the quantized block to form a local reconstructed residual block (S106); generating a reconstructed block by adding the reconstructed residual block and the predicted block; determining a SAO type parameter for a SAO value for the reconstructed block, the SAO type parameter indicating whether the SAO value is a boundary offset value or a range offset value, and the SAO value is to be added to a pixel value of the reconstructed image; defining a plurality of bins, the plurality of bins representing an integer indicating the magnitude of the SAO value; And encode a SAO type parameter and a plurality of bins into the bit stream using constant probability bypass arithmetic encoding.