RU2789394C2 - Image encoding method, image encoding device, image decoding method, image decoding device, and image encoding and decoding device - Google Patents

Abstract

FIELD: image encoding/decoding.

SUBSTANCE: parameter of a sample adaptive offset (hereinafter – SAO) type is decoded from a bit stream. A SAO type is determined from the parameter of the SAO type for a SAO value to be added to a value of a restored image pixel. A set of beans with variable length encoding is decoded from the bit stream, using bypass arithmetic encoding with a constant probability, wherein the set of beans is an integer indicating the SAO value. The integer indicating the SAO value is determined based on the set of beans decoded using bypass arithmetic encoding with the constant probability. The SAO value is determined using the mentioned determined integer.

EFFECT: provision of determination of a SAO value.

14 cl, 38 dwg

Description

[Technical field]

[0001] The present invention relates to an image encoding method, an image encoding apparatus, an image decoding method, an image decoding apparatus, and an image encoding and decoding apparatus, and in particular, an image encoding method, an image encoding apparatus, an image decoding method, an image decoding apparatus, and to an image encoding and decoding device to perform arithmetic encoding or arithmetic decoding on a sample adaptive offset (SAO) parameter.

[prior art]

[0002] Recently, there has been a significant technical development in digital video devices, and increasing possibilities for encoding a video signal (multiple images organized in a time series) with compression and recording the video signal on recording media, such as DVDs and hard drives, or distributing the video signal on the Internet . For example, H.264/AVC (MPEG-4 AVC) is one of the standards for compressed video coding. In addition, the High Efficiency Video Coding (HEVC) standard is currently being considered as the next generation standard.

[0003] The current HEVC standard (NPL 1) contains an offset process called SAO. The SAO process is a process for adding an offset value to each pixel of an image decoded from a bitstream (encoded bitstream). Performing the SAO process makes it possible to reduce image deterioration due to encoding and make the decoded image more similar to the original image. According to the current HEVC standard, the SAO process performs arithmetic encoding on the offset value and adds the offset value to the bit stream.

[List of bibliographic references]

[Non-Patent Literature]

[0004] [NPL 1] Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 9th Meeting: Geneva, CH, 27 April-7 May 2012, JCTVC- I0602_CDTexts_r2.doc, BoG report on integrated text of SAO adoptions on top of JCTVC-I0030, http://phenix.it-sudparis.eu/jct/doc_end_user/documents/9_Geneva/wg11/JCTVC-I0602-v3.zip

[Summary of the Invention]

[Technical issue]

[0005] In the conventional technique, faster processing is desirable when encoding and decoding an image.

[0006] Here, the present invention provides an image encoding method and an image decoding method in which processing can be accelerated.

[The solution of the problem]

[0007] An image encoding method according to an aspect of the present invention is an image encoding method for encoding an input image, and includes: converting to a bin string an offset value used in the offset process to be applied to a pixel value of a reconstructed image corresponding to an input image ; and performing bypass arithmetic encoding on the bin string using a constant probability.

[0008] Further, an image decoding method according to another aspect of the present invention is an image decoding method for decoding an encoded image, and includes: performing bypass arithmetic decoding on a bin string using a constant probability, the bin string being encoded and indicating an offset value, an offset used in the process to be applied to the pixel value of the reconstructed image corresponding to the encoded image; and converting to an offset value of the bin string on which bypass arithmetic decoding is performed.

[0009] These general or specific aspects of the present invention may be implemented by a system, device, integrated circuit, computer program, computer-readable recording medium such as a CD-ROM, or any combination of system, device, integrated circuit, computer program, and recording medium.

[Useful results of the invention]

[0010] With the image encoding method, image encoding apparatus, image decoding method, image decoding apparatus, and image encoding and decoding apparatus according to the present invention, processing can be accelerated.

[Brief Description of Drawings]

[0011] FIG. 1 is a block diagram illustrating an exemplary configuration of an image encoding apparatus according to Embodiment 1.

Fig. 2 is a block diagram illustrating an exemplary configuration of an SAO parameter encoding unit.

Fig. 3 is a block diagram illustrating an exemplary configuration of the sao_offset encoding node.

Fig. 4 is a flowchart indicating exemplary processes in an image encoding apparatus according to Embodiment 1.

Fig. 5 is a flowchart indicating exemplary processes in the SAO parameter encoding section according to Embodiment 1.

Fig. 6A is a flowchart indicating exemplary processes in the sao_offset encoding node according to Embodiment 1.

Fig. 6B is a flowchart indicating exemplary processes in the sao_offset encoding node according to Variation 2 of Embodiment 1.

Fig. 7 is a table indicating an approximate correspondence between non-binary signals and bin strings.

Fig. 8 is a table indicating the correspondence between the bins included in binIdx and the arithmetic encoding to be applied.

Fig. 9 is a table showing the result of an experiment on coding efficiency according to Embodiment 1 and Embodiment 1.

Fig. 10 is a block diagram illustrating an exemplary configuration of an image decoding apparatus according to Embodiment 2.

Fig. 11 is a block diagram illustrating an exemplary configuration of an SAO parameter decoding portion.

Fig. 12 is a block diagram illustrating an exemplary configuration of a sao_offset decoding node.

Fig. 13 is a flowchart indicating exemplary processes in the image decoding apparatus according to Embodiment 2.

Fig. 14 is a flowchart indicating exemplary processes in the SAO parameter decoding section according to Embodiment 2.

Fig. 15A is a flowchart indicating exemplary processes in the sao_offset decoding node according to Embodiment 2.

Fig. 15B is a flowchart indicating exemplary processes in the sao_offset decoding node according to Embodiment 2 of Embodiment 2.

Fig. 16 illustrates the general configuration of a content provision system for realizing content distribution services.

Fig. 17 illustrates the overall configuration of a digital broadcast system.

Fig. 18 is a block diagram of a TV configuration example.

Fig. 19 is a block diagram illustrating a configuration example of an information playback/recording unit that reads and writes information from or to a recording medium that is an optical disc.

Fig. 20 illustrates a configuration example of a recording medium that is an optical disc.

Fig. 21A illustrates an example of a cell phone.

Fig. 21B illustrates a block diagram of a cell phone configuration example.

Fig. 22 illustrates the structure of the multiplexed data.

Fig. 23 schematically illustrates how each stream is multiplexed into multiplexed data.

Fig. 24 illustrates in more detail how the video stream is stored in the PES packet stream.

Fig. 25 illustrates the structure of TS packets and source packets in multiplexed data.

Fig. 26 illustrates the PMT data structure.

Fig. 27 illustrates the internal structure of multiplexed data information.

Fig. 28 illustrates the internal structure of the stream attribute information.

Fig. 29 illustrates steps for identifying video data.

Fig. 30 is a block diagram illustrating an example of an integrated circuit configuration for implementing a movie encoding method and a movie decoding method according to each of the Embodiments.

Fig. 31 illustrates a configuration for switching between driving frequencies.

Fig. 32 illustrates steps for identifying video data and switching between drive frequencies.

Fig. 33 illustrates an example lookup table in which video data standards are associated with drive frequencies.

Fig. 34A illustrates a configuration example for sharing a module in a signal processing node.

Fig. 34B illustrates a configuration example for sharing a module in a signal processing node.

[Description of Embodiments]

[0012] (Information on which the present invention is based)

The inventors have found that the image encoding method and others described in the "Prior Art" have the following problems.

[0013] As described above, according to the current HEVC standard, in the SAO process, which is an offset process, arithmetic encoding is performed on the offset value, and the offset value is added to the bit stream. More specifically, in HEVC arithmetic coding, a signal to be encoded (eg, an SAO offset value) is converted from a non-binary signal to a bin string, and arithmetic encoding is performed on the bin string.

[0014] Arithmetic coding has two types: (1) context arithmetic coding, in which arithmetic coding is performed using an adaptive symbol occurrence probability; and (2) bypass arithmetic coding, in which arithmetic coding is performed by setting the probability of occurrence of a symbol to, for example, 50%.

[0015] (1) In context arithmetic coding, a context is selected per signal to be encoded. The probability of a character appearing is determined according to that context. In addition, in context arithmetic coding, a selected context is loaded, and arithmetic coding is performed on a signal to be encoded based on a symbol occurrence probability corresponding to the loaded context. Further, in context arithmetic coding, a context update process to update a symbol occurrence probability (eg, occurrence probabilities of 1) of a context is performed according to a signal value to be encoded.

[0016] (2) In contrast, in bypass arithmetic coding, arithmetic coding is performed on the signal to be encoded by setting the occurrence probability of a character to, for example, 50% without using any context. Thus, bypass arithmetic coding does not require the context to be loaded and updated.

[0017] Conventionally, an image encoding device performs context arithmetic encoding on the SAO offset value. The reason is that the probability of a character appearing in the bin string indicating the SAO offset value is likely to have some deviation according to the HEVC (NPL 1) standard. When the probability of occurrence of a symbol has a deviation, the decrease in coding efficiency can be stopped using context arithmetic coding.

[0018] However, context arithmetic coding requires processing such as context loading and updating. In addition, when a context identical to the context selected in context arithmetic coding over a preset SAO offset value is selected in context arithmetic coding over another SAO offset value, context arithmetic coding cannot be started over the next SAO offset value until the context for the offset value is updated. The SAO on which the arithmetic coding is performed first. Thus, there are problems with sequential processes and reduced throughput.

[0019] In order to solve such problems, an image encoding method according to an aspect of the present invention is an image encoding method for encoding an input image, and includes: converting to a bin string an offset value used in the offset process to be applied to a pixel value of the reconstructed an image corresponding to the input image; and performing bypass arithmetic encoding on the bin string using a constant probability.

[0020] Since, in the image encoding method, arithmetic encoding is performed on the SAO offset value using bypass arithmetic encoding, processing can be accelerated.

[0021] The inventors have found that the probabilities of occurrence of characters in the bin strings indicating the SAO offset value have small deviations, which will be described later. Thus, the coding efficiency is not likely to decrease even when bypass arithmetic coding is performed. Thus, the inventors have confirmed by experiment that bypass arithmetic coding as arithmetic coding over the SAO offset value does not reduce the coding efficiency.

[0022] Thus, the image encoding method with this configuration enables processing to be accelerated without degrading encoding efficiency.

[0023] Further, an image decoding method according to another aspect of the present invention is an image decoding method for decoding an encoded image, and includes: performing bypass arithmetic decoding on a bin string using a constant probability, the bin string being encoded and indicating an offset value, an offset used in the process to be applied to the pixel value of the reconstructed image corresponding to the encoded image; and converting to an offset value of the bin string on which bypass arithmetic decoding is performed.

[0024] In addition, the image encoding device according to another aspect of the present invention is an image encoding device that encodes an input image and includes a control circuit and a memory electrically connected to the control circuit, the control circuit performing: bin string conversion an offset value used in the offset process to be applied to the pixel value of the reconstructed image corresponding to the input image; and performing bypass arithmetic encoding on the bin string using a constant probability.

[0025] In addition, the image decoding apparatus according to another aspect of the present invention is an image decoding apparatus that decodes an encoded image and includes a control circuit and a memory electrically connected to the control circuit, the control circuit executing: performing bypass arithmetic decoding above the bin row using a constant probability, the bin row being encoded and indicating an offset value used in the offset process to be applied to a pixel value of the reconstructed image corresponding to the encoded image; and converting to an offset value of the bin string on which bypass arithmetic decoding is performed.

[0026] In addition, the image encoding and decoding apparatus according to another aspect of the present invention includes an image encoding apparatus and an image decoding apparatus.

[0027] These general or specific aspects of the present invention may be implemented with a system, method, integrated circuit, computer program, computer-readable recording medium such as a CD-ROM, or any combination of system, method, integrated circuit, computer program, and recording medium.

[0028] Embodiments in accordance with the present invention will be described with reference to the drawings.

[0029] The embodiments described below indicate specific or general examples of the present invention. The meanings, shapes, materials, constituent elements, positions and connections of constituent elements, steps and orders of steps indicated in the Embodiments are examples and do not limit the present invention. The constituent elements in the Embodiments that are not described in independent claims that describe the most general idea of the present invention are described as arbitrary constituent elements.

[0030] (Embodiment 1)

An image encoding apparatus and an image encoding method according to Embodiment 1 will be described with reference to FIG. 1-9.

[0031] <1-1. Device Configuration>

An image encoding apparatus according to Embodiment 1 will be described with reference to FIG. 1. FIG. 1 is a block diagram illustrating the configuration of the HEVC standard image encoding device 100 according to Embodiment 1.

[0032] The image encoding apparatus 100 is an apparatus that encodes an image to be input (input image) block by block to generate a bitstream (coded bitstream).

[0033] As illustrated in FIG. 1, the image encoding apparatus 100 includes a blocking unit 101, a subtracting unit 102, a transformation unit 103, a variable length encoding unit 104 (entropy encoding unit), an inverse transformation unit 105, an addition unit 106, an SAO processing unit 107, a unit 110 SAO parameter coding, frame memory 108, and prediction node 109 .

[0034] The blocking unit 101 breaks the input image into blocks to be predicted and includes pixels. The blocks to be predicted are variable size blocks. For example, the minimum block size is 4x4 pixels and the maximum block size is 32x32 pixels.

[0035] The subtraction node 102 subtracts the prediction block predicted by the prediction node 109, which will be described later, from each of the blocks to be predicted to calculate the difference block. The transform section 103 converts the difference block from the image domain to the frequency domain to derive frequency coefficients, and quantizes these frequency coefficients. The quantized frequency coefficients will henceforth be referred to as quantized coefficients.

[0036] The variable length encoding section 104 performs entropy encoding on the quantized coefficients output from the transformation section 103 and various information necessary for decoding (partition information for blocks to be predicted, prediction type, motion vector, prediction mode (internal prediction), a quantized parameter, and the like) to generate a bit stream.

[0037] The inverse transform section 105 performs inverse quantization to convert the quantized coefficients output from the transform section 103 into frequency coefficients, and performs an inverse transform to transform the frequency coefficients from the frequency domain to the image domain to output the reconstructed difference block. The addition node 106 adds the prediction block predicted by the prediction node 109, which will be described later, with the reconstructed difference block output by the inverse transform node 105 to form a reconstructed block.

[0038] The SAO processing unit 107 performs an SAO process (offset process) on the reconstructed block to add to the pixel value of each pixel included in the reconstructed block an SAO offset value corresponding to the category (class) to which the pixel belongs, which will be described later. Because the process performed by the transform node 103 is an irreversible process, an error occurs between each of the blocks to be predicted in the input image and the reconstructed block. The SAO offset value is an error correction value and is set per category.

[0039] The SAO parameter encoding section 110 performs variable length encoding on various parameters to be used in the SAO process, and adds the parameters to the bit stream output from the variable length encoding section 104 . Embodiment 1 sets as SAO parameters sao_offset indicating the SAO offset value and sao_type_idx indicating the category of the SAO offset value.

[0040] FIG. 2 is a block diagram illustrating an exemplary configuration of the SAO parameter encoding portion 110. As illustrated in FIG. 2, the SAO parameter encoding section 110 includes a sao_type_idx encoding section 111 and a sao_offset encoding section 112.

[0041] Node 111 encoding sao_type_idx performs arithmetic encoding on sao_type_idx indicating the category of sao_offset. Here, sao_offset is classified into two categories: a border offset value determined according to the state of the surrounding pixels; and a range offset value determined according to the region to which the pixel value of the pixel belongs.

[0042] The sao_offset encoding unit 112 performs arithmetic encoding on sao_offset. Fig. 3 is a block diagram illustrating an exemplary configuration of the sao_offset encoding node 112.

[0043] As illustrated in FIG. 3, the sao_offset encoding section 112 includes a sao_offset binarization section 113, an arithmetic encoding switching section 114, a context 0 arithmetic encoding section 115, a context 1 arithmetic encoding section 116, and a bypass arithmetic encoding section 117.

[0044] The sao_offset binarization section 113 converts sao_offset, which is a non-binary signal, into binary form (converts sao_offset to a signal representing 0 and 1). The arithmetic coding switching section 114 switches the arithmetic coding method to be applied to sao_offset. The context 0 arithmetic encoding section 115 performs context 0 arithmetic encoding using the first context (context 0). The context 1 arithmetic encoding node 116 performs context 1 arithmetic encoding using the second context (context 1). The bypass arithmetic coding unit 117 performs bypass arithmetic coding using a constant probability.

[0045] The frame memory 108 includes a storage device such as random access memory (RAM) and read only memory (ROM).

[0046] The prediction node 109 generates a prediction block by inter prediction or intra prediction. In inter-prediction, the prediction node 109 outputs a motion vector using a block to be predicted, and generates a prediction block using the derived motion vector and the reconstructed block obtained in the SAO process and stored in the frame memory 108. In intra prediction, the prediction node 109 generates a prediction block using the reconstructed block obtained in the SAO process and stored in the frame memory 108 in accordance with the prediction mode.

[0047] <1-2 General Operations>

Next, with reference to FIG. 4, operations of the image encoding apparatus will be described. Fig. 4 is a flowchart indicating a procedure in the image encoding apparatus 100.

[0048] In the image encoding apparatus 100, the block splitter 101 first splits the input image into blocks to be predicted, and sequentially outputs these blocks to the subtraction node 102 and the prediction node 109 (step S101). The blocks to be predicted have a variable size according to Embodiment 1. The input image is split using image features. The minimum block size to be predicted is 4 horizontal pixels × 4 vertical pixels according to Embodiment 1. In addition, the maximum block size to be predicted is 32 horizontal pixels × 32 vertical pixels according to Embodiment 1.

[0049] The prediction section 109 in the image encoding apparatus 100 generates a prediction block using the blocks to be predicted and the reconstructed block stored in the frame memory 108 (step S102).

[0050] The subtraction section 102 in the image encoding apparatus 100 subtracts a prediction block from each of the blocks to be predicted to form a difference block (step S103).

[0051] The transform section 103 in the image encoding apparatus 100 converts the difference block into frequency coefficients, and converts the frequency coefficients into quantized coefficients (step S104).

[0052] The inverse transform section 105 in the image encoding apparatus 100 inversely quantizes the quantized coefficients to derive frequency coefficients, and converts the frequency coefficients from the frequency domain to the image domain to generate a reconstructed difference block (step S105).

[0053] The addition section 106 in the image encoding apparatus 100 adds the prediction block to the reconstructed difference block to form the reconstructed block (step S106).

[0054] The SAO processing section 107 in the image encoding apparatus 100 executes the SAO process (step S107). The SAO processing unit 107 first determines the SAO parameters to be used for the SAO process. In Embodiment 1, the SAO parameters are sao_offset indicating the SAO offset value and sao_type_idx indicating the category of the SAO offset value. The SAO processing section 107 adds the SAO offset value to the pixel value of each pixel included in the reconstructed block, and stores in the frame memory 108 the reconstructed block to which the SAO offset values are added. In the SAO process, each of the pixels included in the reconstructed block is classified into categories, and an SAO offset value is output for each of those categories. The methods for classifying pixels are numerous. Thus, the sao_type_idx parameter is used to indicate the category of the SAO offset value.

[0055] The SAO parameter encoding section 110 in the image encoding apparatus 100 performs variable length encoding on the SAO parameter and adds this parameter to the bitstream output from the variable length encoding section 104 (step S108). The details of the SAO parameter encoding node 110 will be described later.

[0056] The variable length encoding section 104 in the image encoding apparatus 100 performs variable length encoding on the quantized coefficients output from the transform section 103 to output a bit stream (step S109).

[0057] The image encoding device 100 repeatedly executes Steps S102 to S109 until the encoding of all input image blocks to be predicted is completed (step S110).

[0058] <1-2-1. SAO parameter encoding node operations>

Operations of the SAO parameter encoding section 110 according to Embodiment 1 will be described with reference to FIG. 5.

[0059] FIG. 5 is a flowchart indicating exemplary processes of the SAO parameter encoding portion 110, and describes in detail Step S108 of FIG. 4.

[0060] The sao_type_idx encoding section 111 in the SAO parameter encoding section 110 encodes the sao_type_idx (step S111).

[0061] The sao_offset encoding section 112 in the SAO parameter encoding section 110 encodes sao_offset (step S112).

[0062] <1-2-2. Encoding Node Operations sao_offset>

Operations of the sao_offset encoding section 112 according to Embodiment 1 will be described with reference to FIG. 6A. Fig. 6A is a flowchart indicating exemplary processes in the sao_offset encoding node 112.

[0063] The sao_offset to binary conversion section 113 in the sao_offset encoding section 112 converts sao_offset from a non-binary signal to a bin string (step S121).

[0064] Embodiment 1 describes the case where sao_offset takes values from 0 to 7. FIG. 7 is a table indicating the correspondence between non-binary signals and bin strings. As can be seen from FIG. 7, the number of consecutive "1s" from the beginning of each row of bins is equal to the value indicated by the non-binary signal. Also, "0" is added to the end when the non-binary signal is between 0 and 6. When the non-binary signal is 7, which is the maximum number, no "0" is added to the end. Here, binIdx is the index of a bin (one bit) included in the bin string. The first value of binIdx is 0, and subsequent values are incremented by 1. The bin string is output to the arithmetic encoding switching node 114 sequentially from the bin whose binIdx is 0.

[0065] Node 114 switch arithmetic encoding node 112 encoding sao_offset determines the value of binIdx (steps S122, S124 and S126).

[0066] When the value is 0 (Yes in step S122), the arithmetic encoding switching section 114 outputs the bin to the context 0 arithmetic encoding section 115. The context 0 arithmetic encoding section 115 performs context arithmetic encoding using context 0 (step S123).

[0067] When the value is 1 (No in step S122 and Yes in step S124), the arithmetic encoding switching section 114 outputs the bin to the context 1 arithmetic encoding section 116. The context 1 arithmetic encoding section 116 performs context arithmetic encoding using context 1 (step S125).

[0068] When the value is neither 0 nor 1 (No in step S122 and No in step S124), the arithmetic encoding switching section 114 outputs the bin to the bypass arithmetic encoding section 117 . The bypass arithmetic encoding section 117 performs bypass arithmetic encoding (step S126).

[0069] Since the bypass arithmetic encoding node 117 does not use any context, the sao_offset encoding node 112 generally uses two types of contexts in accordance with Embodiment 1.

[0070] The sao_offset encoding section 112 repeats the processes from Step 122 to Step 127 until all the bins included in the bin string have been processed (step S127).

[0071] <1-3. Benefits, varieties, etc.>

The image encoding apparatus 100 according to Embodiment 1 can speed up processing by performing bypass arithmetic encoding on the bins in the second half of sao_offset. More specifically, according to Embodiment 1, not context arithmetic encoding but bypass arithmetic encoding is performed on bins whose binIdx is 2 or higher. As described above, bypass arithmetic coding does not require any context to be loaded and updated. Thus, arithmetic encoding on an SAO offset value of a block to be predicted can be started without waiting for completion of arithmetic encoding on another SAO offset value, and processing can be accelerated.

[0072] In addition, the character occurrence probabilities (occurrence probabilities 1) of binIdx 1 to 6 according to the HEVC standard (NPTL 1) currently under study are almost the same, but not 50% and have a deviation. In other words, when there is a binIdx after binIdx1 (when sao_offset is 1 or higher), the cases are often divided into the case where binIdx1 is 0 without any binIdx2 and after (sao_offset is 1), and the case where all binIdx 1 to 6 are 1 (sao_offset is 7).

[0073] However, the inventors have found that as a result of the experiment to encode binIdx 1 to 6 with a constant symbol occurrence probability of 50%, the encoding efficiency hardly deteriorates. In particular, the experiment has shown that sao_offset often indicates an intermediate value (from 2 to 6), and the probabilities of occurrence of characters in binIdx from 1 to 6 are close to 50%. Thus, the processing for suppressing the decrease in coding efficiency can be accelerated by using not context arithmetic coding but bypass arithmetic coding over binIdx after binIdx2.

[0074] <1-3-1. Varieties>

Although bypass arithmetic encoding is performed on bins after binIdx2 according to Embodiment 1, processing is not limited to this.

[0075] FIG. 8 is a table indicating the correspondence between the bins included in binIdx and the arithmetic encoding to be applied. In FIG. 8 "Option 1" indicates the correspondence between the beans described in Embodiment 1 and the arithmetic coding to be applied. In addition, "Context 0" indicates context arithmetic encoding using context 0. "Context 1" indicates context arithmetic encoding using context 1. "Bypass" indicates bypass arithmetic encoding.

[0076] "Change 1" in FIG. 8 indicates Variant 1. According to Variant 1, context arithmetic encoding is performed on the bin at binIdx0 and bypass arithmetic encoding is performed on bins after binIdx1. In addition, according to Variation 1, the sao_offset encoding node 112 in FIG. 2 should only include a sao_offset binarization node 113, an arithmetic coding switching node 114, a context 0 arithmetic coding node 115, and a bypass arithmetic coding node 117. In other words, the sao_offset encoding node 112 should not include the context 1 arithmetic encoding node 116.

[0077] "Change 2" in FIG. 8 indicates Variant 2. In Variant 2, bypass arithmetic encoding is performed on each of the bins in binIdx, as shown in FIG. 8.

[0078] FIG. 6B is a flowchart indicating exemplary processes (image encoding method) in the sao_offset encoding section 112 according to Variation 2. FIG. 6B, sao_offset is binarized (step S121 to convert the offset value to a bin string), and bypass arithmetic encoding is performed on all bins included in sao_offset (steps S126 and S127 to perform bypass arithmetic encoding). Because bypass arithmetic coding is performed on all bins, processing can be fastest in Variation 2.

[0079] In addition, according to Variant 2, the sao_offset encoding node 112 in FIG. 2 should only include the sao_offset binarization node 113 and the bypass arithmetic coding node 117 . In other words, the sao_offset encoding node 112 should not include the context 0 arithmetic encoding node 115 and the context 1 arithmetic encoding node 116.

[0080] Although it is not illustrated, it is acceptable to perform contextual arithmetic coding over Binidx from 0 to 2 and bypass arithmetic coding over binidx after Binidx3.

[0081] <1-3-2. Coding Efficiency Experiment Result>

Fig. 9 - a table showing the result of an experiment on the effectiveness of coding in accordance with the option 1 implementation and a variety of 1.

[0082] FIG. 9 shows the comparison result between the encoding gain obtained by performing arithmetic encoding on the SAO offset value according to the current HEVC standard (NPL 1) and the encoding gain according to Embodiment 1 and Variation 1. The experimental conditions follow the general experimental conditions in the structure of the standard. HEVC.

[0083] The larger the value, the lower the encoding efficiency. A negative value indicates an increase in coding efficiency.

[0084] As shown in "Embodiment 1" of FIG. 9, the values indicating the result of the comparison range between -0.1% and 0.1%. This indicates that the encoding efficiency for encoding binIdx2 to binIdx6 of the SAO offset value is almost the same between context arithmetic coding and bypass arithmetic coding.

[0085] Although the encoding efficiency of Variant 1 is lower than that of Embodiment 1, as seen from "Change 1" in FIG. 9, the results fluctuate between 0 and 0.1%, and the coding efficiency is almost the same.

[0086] Although the coding efficiency of Variant 2 is also lower than that of Embodiment 1, as seen from "Change 2" in FIG. 9, the results fluctuate between 0 and 0.1%, and the coding efficiency is almost the same.

[0087] As a result of the encoding efficiency experiment, applying bypass arithmetic encoding when performing arithmetic encoding on the SAO offset value makes it possible to speed up the arithmetic encoding while encoding efficiency is less likely to decrease.

[0088] (Embodiment 2)

The device for decoding images and the method of decoding images in accordance with the option 2 of the implementation will be described with reference to FIG. 10-15A.

[0089] <2-1. Device Configuration>

The device for decoding images in accordance with the option 2 will be described with reference to FIG. 10. FIG. 10 - block diagram illustrating the configuration of the device 200 decoding images in accordance with the HeVC standard in accordance with the option 2 implementation.

[0090] The image decoding apparatus 200 is an apparatus that decodes an input bitstream (encoded bitstream) to generate a decoded image ("Picture" in FIG. 10).

[0091] As illustrated in FIG. 10, the image decoding apparatus 200 includes a variable length decoding section 201, an inverse transformation section 202, an SAO parameter decoding section 210, an SAO processing section 204, a decoded block combining section 205, a frame memory 206, and a prediction section 207.

[0092] The variable length decoding section 201 performs entropy decoding (variable length decoding) on the bit stream input to the image decoding apparatus 200 to obtain quantized coefficients and various information necessary for decoding (partitioning information for blocks to be predicted, prediction type, motion vector, prediction mode (intra prediction mode), quantized parameter, etc.).

[0093] The inverse transform section 202 performs inverse quantization to convert the quantized coefficients output from the variable length decoding section 201 into frequency coefficients, and performs an inverse transform to convert the frequency coefficients from the frequency domain to the image domain to output a difference block.

[0094] The addition node 203 adds the prediction block output from the prediction node 207 with the difference block output from the inverse transform node 202 to form a decoded block.

[0095] The SAO parameter decoding section 210 decodes various parameters to be used in the SAO process from the bit stream. Embodiment 2 sets as SAO parameters sao_offset indicating the SAO offset value and sao_type_idx indicating the category of the SAO offset value.

[0096] Fig. 11 is a block diagram illustrating an exemplary configuration of the SAO parameter decoding portion 210. As illustrated in FIG. 11, the SAO parameter decoding section 210 includes a sao_type_idx decoding section 211 that performs arithmetic decoding on sao_type_idx and a sao_offset decoding section 212 that performs arithmetic decoding on sao_offset.

[0097] FIG. 12 is a block diagram illustrating an exemplary configuration of the sao_offset decoding node 212. As illustrated in FIG. 12, the sao_offset decoding section 212 includes an arithmetic decoding switching section 213, a context 0 arithmetic decoding section 214, a context 1 arithmetic decoding section 215, a bypass arithmetic decoding section 216, and a debinary sao_offset decoding section 217.

[0098] The arithmetic decoding switching section 213 switches the arithmetic decoding method to be applied to the bit stream. The context 0 arithmetic decoding section 214 performs context 0 arithmetic decoding using the first context (context 0). The context 1 arithmetic decoding section 215 performs context 1 arithmetic decoding using the second context (context 1). The bypass arithmetic decoding section 216 performs bypass arithmetic decoding using a constant probability. The sao_offset debinarization node 217 debinarizes the sao_offset bin string output from each of the context 0 arithmetic decoding node 214, the context 1 arithmetic decoding node 215, the bypass arithmetic decoding node 216 to output a non-binary sao_offset value.

[0099] The SAO processing unit 204 performs an SAO process (offset process) on the decoded block output from the addition node 203 to add the SAO offset value to the pixel value of each pixel included in the decoded block, which will be described in detail later.

[0100] The decoded block merge unit 205 merges the decoded blocks obtained in the SAO process to reconstruct the decoded image. In addition, the decoded block merge node 205 stores the decoded image in the frame memory 206 .

[0101] The frame memory 206 includes a storage device such as random access memory (RAM) and read only memory (ROM).

[0102] The prediction node 207 generates a prediction block by inter prediction or intra prediction. The prediction method to be applied by the prediction node 207 is the same as in the prediction node 109 .

[0103] <2-2 General Operations>

Further with reference to FIG. 13, operations of the image decoding apparatus 200 will be described. Fig. 13 is a flowchart indicating a procedure in the image decoding apparatus 200.

[0104] The SAO parameter decoding section 210 in the image decoding apparatus 200 performs variable length decoding on the SAO parameter (step S201).

[0105] The variable length decoding section 201 in the image decoding apparatus 200 performs variable length decoding on the bit stream to obtain quantized coefficients, and outputs the quantized coefficients to the inverse transform section 202 (step S202).

[0106] The inverse transform section 202 in the image decoding apparatus 200 performs inverse quantization on the quantized coefficients to obtain frequency coefficients, and performs inverse transformation on the obtained frequency coefficients to form a difference block (step S203).

[0107] The prediction section 207 in the image decoding apparatus 200 generates a prediction block from the decoded image stored in the frame memory 206 . In addition, the adding section 203 in the image decoding apparatus 200 adds the prediction block outputted from the prediction section 207 with the difference block outputted from the inverse transform section 202 to form a decoded block (step S204).

[0108] The SAO processing section 204 in the image decoding apparatus 200 executes the SAO process (step S205). In the SAO process, the SAO processing node 204 classifies each pixel of the decoded block into categories according to the SAO parameter, and adds an offset value corresponding to each of the categories to the pixel. The SAO parameters include a sao_type_idx parameter indicating a category and a sao_offset parameter indicating an offset value.

[0109] The image decoding apparatus 200 repeats the processes of Step 201 to Step 205 until decoding of all blocks included in the encoded image is completed (step S206).

[0110] The decoded block combining unit 205 in the image decoding apparatus 200 combines the decoded blocks to form a decoded image, and stores the decoded image in the frame memory 206 .

[0111] <2-2-1. SAO parameters variable length decoding node operations

The operations of the node 210 decoding SAO parameters in accordance with the option 2 will be described with reference to FIG. 14.

[0112] FIG. 14 is a flowchart indicating exemplary processes of the SAO parameter decoding portion 210, and describes in detail Step S205 of FIG. 13.

[0113] The sao_type_idx decoding section 211 in the SAO parameter decoding section 210 decodes the sao_type_idx (step S211).

[0114] The sao_offset decoding section 212 in the SAO parameter decoding section 210 decodes the sao_offset (step S212).

[0115] <2-2-2. Operations of decoding node 212 sao_offset>

Operations of the sao_offset decoding section 212 according to Embodiment 2 will be described with reference to FIG. 15A. Fig. 15A is a flowchart indicating exemplary processes in the sao_offset decoding node 212 .

[0116] The arithmetic decoding switching node 213 in the sao_offset decoding node 212 determines the binIdx value of the bin to be decoded (steps S221, S223 and S225).

[0117] When the value is 0 (Yes in step S221), the arithmetic decoding switching section 213 outputs the bin to be decoded to the context 0 arithmetic decoding section 214. The context 0 arithmetic decoding section 214 performs context arithmetic decoding using context 0 (step S222).

[0118] When the value is 1 (No in step S221 and Yes in step S223), the arithmetic decoding switching section 213 outputs the bin to be decoded to the context 1 arithmetic decoding section 215. The context 1 arithmetic decoding section 215 performs context arithmetic decoding with using context 1 (step S224).

[0119] When the value is not equal to 0 or 1 (not at the S221 stage and is not at the S223 stage), the assembly 213 of the arithmetic decoding shifts, which should be decoded, into the assembly of 216 bypass arithmetic decoding. Knot 216 bypass arithmetic decoding performs bypass arithmetic decoding.

[0120] The sao_offset decoding section 212 repeats the processes of Step 221 to Step 226 until the value of the bin obtained in arithmetic decoding is 0, or a bin whose binIdx is 6 is decoded (step S226).

[0121] The sao_offset decoding section 212 converts the bin string including the bins decoded by the sao_offset demapping section 217 from binary form into a non-binary signal (step S227). The correspondence between non-binary signals and bin strings is the same as shown in FIG. 7 in accordance with Embodiment 1.

[0122] <2-3. Benefits, etc.>

The image decoding apparatus 200 according to Embodiment 2 can speed up processing by performing bypass arithmetic decoding on the bins in the second half of sao_offset.

[0123] <2-3-1. Varieties>

Although bypass arithmetic decoding is performed on bins after binIdx2 according to Embodiment 2, the processing is not limited to this. In the same manner as in Embodiment 1, bypass arithmetic decoding can be performed on bins after binIdx1 (Variation 1) and on all bins (Variation 2).

[0124] In Embodiment 1, according to Embodiment 2, context arithmetic decoding is performed on the bin in binIdx0, and bypass arithmetic decoding is performed on bins after binIdx1. This process is a decoding process corresponding to the encoding process of Variant 1 of Embodiment 1 in FIG. 8. Since the sao_offset decoding node 212 in FIG. 12 according to Variation 1 performs bypass arithmetic decoding on the bins after binIdx1, it should only include an arithmetic decoding switch node 213, a context 0 arithmetic decoding node 214, a bypass arithmetic decoding node 216, and a sao_offset inverse transform node 217 from binary. In other words, the sao_offset decoding node 212 according to Variant 1 of Embodiment 2 does not need to include the context 1 arithmetic decoding node 215.

[0125] In accordance with Variant 2 of Option 2, the sao_offset decoding node 212 performs bypass arithmetic decoding on each of the bins in binIdx. This process is a decoding process corresponding to the encoding process of Variant 2 of Embodiment 1 in FIG. 8.

[0126] FIG. 15B is a flowchart indicating exemplary processes (image decoding method) in the sao_offset decoding section 212 according to Embodiment 2 of Embodiment 2. In FIG. 15B, bypass arithmetic decoding is performed on all bins included in sao_offset (steps S225 and S226 to perform bypass arithmetic decoding), and the decoded sao_offset is debinarized (step S227 to convert the bin string to an offset value). Since bypass arithmetic decoding is performed on all bins, processing can be fastest in Variation 2.

[0127] Since the sao_offset decoding node 212 in FIG. 12 according to Variation 2 in this document performs arithmetic bypass decoding on all bins, it should only include arithmetic bypass node 216 and sao_offset inverse transform node 217 from binary. In other words, the sao_offset decoding node 212 according to Variation 2 of Embodiment 2 does not need to include the arithmetic decoding switch node 213, the context 0 arithmetic decoding node 214, and the context 1 arithmetic decoding node 215.

[0128] Although not illustrated, it is acceptable to perform context arithmetic decoding on binIdx 0 through 2 and bypass arithmetic decoding on binIdx after binIdx3, as in Embodiment 1.

[0129] (Other variations of Embodiments 1 and 2)

(1) Although Embodiments 1 and 2 (including Variations) use sao_type_idx indicating the SAO offset value category and sao_offset indicating the SAO offset value as SAO parameters as SAO parameters, the SAO parameters are not limited to this. The SAO parameters may include, for example, a parameter that should be ancillary information for pixel classification.

[0130] (2) In addition, although according to Embodiment 1 (including Variations), the SAO parameter is encoded per block to be predicted, encoding is not limited to this. The SAO parameter may not be encoded per block to be predicted, but one less than the block to be predicted, or vice versa per one obtained by combining the blocks to be predicted. Also, the SAO parameter is not encoded in the block to be predicted, but instead the value of another block can be copied and used.

[0131] Here, the image decoding apparatus 200 according to Embodiment 2 (including Variations) decodes the encoded SAO parameter according to the encoding size of the SAO parameter in the image encoding apparatus 100.

[0132] (3) Although sao_offset takes values from 0 to 7 in accordance with Embodiments 1 and 2 (including Variations), sao_offset is not limited to this. It can take values of 8 or higher. In addition, sao_offset may be negative, in which case another sao_offset_sign parameter representing the sign bit may be provided.

[0133] (4) In addition, although Embodiment 1 (including Variations) describes sao_offset encoding, processing is not limited to this. The method for encoding the SAO parameter according to Embodiment 1 (including Variations) may be applied to encoding another syntax to be added to the bitstream. In addition, when the method for encoding an SAO parameter according to Embodiment 1 and its Variant 1 is applied to encoding syntaxes, and the use of the context (the number of bins and binIdx that use the context) is unified, the same variable length encoding node is available when encoding syntaxes. Accordingly, the device configuration can be simplified.

[0134] Examples of syntaxes to which the method for encoding an SAO parameter according to Embodiment 1 and Variants 1 and 2 is applied include ref_idx indicating a reference picture index, merge_idx indicating a copy source of a motion vector and others, and mpm_idx or intra_chroma_pred_mode indicating the mode of intra prediction, according to the current HEVC standard (NPL 1).

[0135] Here, the image decoding apparatus 200 according to Embodiment 2 (including Variations) also decodes the syntax encoded using the SAO parameter encoding method using the SAO parameter decoding method.

[0136] (5) In addition, although the block size to be predicted according to Embodiments 1 and 2 (including Variations) is maximum 32×32 pixels and minimum 4×4 pixels, the size is not limited to this. In addition, although Embodiments 1 and 2 (including Variations) describe blocks to be predicted that are of variable size, the block size may be constant.

[0137] (6) In addition, the processing according to Embodiments 1 and 2 (including Variations) can be implemented by software. The software may be distributed through, for example, a download. In addition, the software may be distributed by writing it to recording media such as CD-ROM. This is true for other Embodiments in the description.

[0138] (7) In Embodiments 1 and 2 (including Variations), each of the functional blocks can be implemented by, for example, an MPU and a storage device in most cases. In addition, generally, processing by each of the function blocks may be implemented by software (program), and such software is recorded on a recording medium such as a ROM. In addition, such software may be distributed, for example, by downloading and writing to recording media such as CD-ROM. Each of the functional blocks can be implemented using hardware (customized circuit).

[0139] (8) The processing described in Embodiments 1 and 2 (including Variations) may be performed as centralized processing with a single device (system) or as decentralized processing with a plurality of devices. Here, the program can be executed by one or more computers. In other words, any of centralized processing and decentralized processing can be performed.

[0140] (9) In addition, each of the constituent elements according to Embodiments 1 and 2 (including Variations) can be implemented using dedicated hardware or by executing a computer program suitable for the constituent element. Each of the constituent elements may be implemented by a program execution unit such as a central processing unit (CPU) and a processor that reads and executes a computer program recorded on a recording medium such as a hard disk or a semiconductor memory device.

[0141] Specifically, each of the image encoding apparatus 100 and the image decoding apparatus 200 includes a control circuit and a memory electrically connected to the control circuit (accessible from the control circuit). The control circuit includes at least one of the specialized hardware and the program execution unit. When the control circuit includes a program execution unit, the storage device stores a computer program executable by the program execution unit.

[0142] The present invention is not limited to Embodiments 1 and 2 (including Variations), and various modifications are possible and are included within the scope of the present invention.

[0143] (Embodiment 3)

An independent computer system can easily perform the processing described in each of the Embodiments by writing to a recording medium a program to implement the structure of the movie encoding method (picture encoding method) or the movie decoding method (picture decoding method) according to the Embodiment. The recording medium may be any as long as a program can be recorded on it, such as a magnetic disk, an optical disk, a magneto-optical disk, an IC board, and a semiconductor memory device.

[0144] Applications of the motion picture encoding method (image encoding method) and the motion picture decoding method (image decoding method) according to each of the Embodiments, and a system using such applications, will be described below. Features of the system include an image encoding apparatus using an image encoding method and an image encoding and decoding apparatus including an image decoding apparatus using an image decoding method. Other system configurations can be suitably changed depending on the case.

[0145] FIG. 16 illustrates the general configuration of the content providing system ex100 for realizing content distribution services. The area for providing communication services is divided into cells of a desired size, and base stations ex106 to ex110, which are fixed wireless stations, are housed in each of the cells.

[0146] The content delivery system ex100 connects to devices such as a computer ex111, a personal digital assistant (PDA) ex112, a camera ex113, a cellular phone ex114, and a gaming device ex115 via the Internet ex101, an Internet service provider ex102, a telephone network ex104, as well as basic stations ex106-ex110.

[0147] However, the configuration of the content providing system ex100 is not limited to the configuration shown in FIG. 16, and a combination is allowed in which any elements are connected. In addition, each of the devices can be connected to the telephone network ex104 directly, and not through the base stations ex106-ex110, which are fixed wireless stations. In addition, the devices may be interconnected with each other via short range wireless communications and others.

[0148] The camera ex113, such as a digital video camera, is capable of capturing moving images. The camera ex116, such as a digital video camera, is capable of capturing both still images and moving images. In addition, the cell phone ex114 may be a phone that conforms to any of the standards such as Global System for Mobile Communications (GSM), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (W-CDMA), Long Term development (LTE) and High Speed Packet Access (HSPA). Alternatively, cell phone ex114 may refer to the Personal Handheld Phone System (PHS).

[0149] In the content providing system ex100, the streaming server ex103 is connected to the camera ex113 and others via the telephone network ex104 and the base station ex109, enabling live transmission and others to be distributed. For such distribution, the content (eg, a live music video) captured by the user using the camera ex113 is encoded as described above in each of the Embodiments, and the encoded content is transmitted to the streaming server ex103. On the other hand, the streaming server ex103 streams the received content data to the clients according to their requests. The clients include a computer ex111, a PDA ex112, a camera ex113, a cellular phone ex114, and a gaming device ex115 that are capable of decoding the above encoded data. Each of the devices that has received the broadcast data decodes and reproduces the encoded data (ie, functions as an image decoding device in accordance with an aspect of the present invention).

[0150] The captured data may be encoded by the camera ex113 or the streaming server ex103 that transmits the data, or the encoding processes may be distributed between the camera ex113 and the streaming server ex103. Similarly, the distributed data may be decoded by the clients or the streaming server ex103, or the decoding processes may be shared between the clients and the streaming server ex103. In addition, data of still images and moving images taken not only by the camera ex113 but also by the camera ex116 can be transmitted to the streaming server ex103 via the computer ex111. The encoding processes may be performed by or distributed between the camera ex116, the computer ex111, or the streaming server ex103.

[0151] Also, typically, the computer ex111 and the LSI ex500 included in each of the devices perform such encoding and decoding processes. The LSI ex500 can be configured from a single chip or multiple chips. The motion picture encoding and decoding software may be embedded in some type of recording medium (eg, CD-ROM, floppy disk, hard disk) that is readable by the computer ex111 and others, and encoding and decoding processes may be performed using the software. In addition, when the cell phone ex114 is equipped with a camera, it is possible to transmit video data acquired by the camera. The video data is data encoded by the LSI ex500 included in the cell phone ex114.

[0152] In addition, the streaming server ex103 may be composed of servers and computers, and may decentralize data and process decentralized data, record or distribute data.

[0153] As described above, clients can receive and reproduce the encoded data in the content providing system ex100. In other words, the clients can receive and decode the information transmitted by the user and play the decoded data in real time in the content providing system ex100 so that a user who does not have any specific right and equipment can realize a personal broadcast.

[0154] The present invention is not limited to the above content providing system ex100, and at least either the movie encoding device (picture encoding device) or the movie decoding device (picture decoding device) described in each of the Embodiments can be included in the digital broadcast, which is shown in Fig. 17. More specifically, the broadcast station ex201 reports or transmits via radio waves to the broadcast satellite ex202 multiplexed data obtained by multiplexing video data with audio data. Video data is data encoded according to the movie encoding method described in each of the Embodiments (ie, data encoded by an image encoding apparatus according to an aspect of the present invention). When receiving video data, the broadcast satellite ex202 transmits radio waves for broadcasting. Then, a home antenna ex204 capable of receiving satellite broadcasts receives the radio waves. A device such as a TV set ex300 (receiver) and a set top box ex217 (STB) decodes the received multiplexed data and reproduces the data (ie, functions as a picture decoding device according to an aspect of the present invention).

[0155] In addition, the reader/writer ex218, which (i) reads and decodes the multiplexed data recorded on the recording media ex215, such as DVD and BD, or (ii) encodes the video signals on the recording media ex215 and in some cases writes data, obtained by multiplexing an audio signal onto the encoded data may include a movie decoding device or a movie encoding device, which are shown in each of the Embodiments. In this case, the reproduced video signals are displayed on the monitor ex219, and another device or system can reproduce the video signals using the recording medium ex215 on which the multiplexed data is recorded. Further, it is possible to implement a movie decoding apparatus in a set top box ex217 connected to a cable ex203 for cable TV or an antenna ex204 for satellite and/or terrestrial broadcasting to display video signals on a monitor ex219 of a TV set ex300. The movie decoding device may not be included in the set-top box, but in the TV ex300.

[0156] FIG. 18 illustrates a TV (receiver) ex300 that uses the movie encoding method and the movie decoding method described in each of the Embodiments. The TV ex300 includes: a tuner ex301 that receives or provides multiplexed data obtained by multiplexing audio data and video data through an antenna ex204 or a cable ex203, etc., which receives a broadcast; a modulation/demodulation node ex302 that demodulates the received multiplexed data or modulates the data into multiplexed data to be delivered outside the node; and a multiplexing/demultiplexing section ex303 that demultiplexes the modulated multiplexed data into video data and audio data, or multiplexes the video data and audio data encoded by the signal processing section ex306 into data.

[0157] In addition, the television ex300 further includes: a signal processing portion ex306 including an audio signal processing portion ex304 and a video signal processing portion ex305 that decode audio data and video data and encode the audio data and video data, respectively (which function as an image encoding device or an image decoding device according to an aspect of the present invention); a speaker ex307 that provides a decoded audio signal; and an output portion ex309 including a display portion ex308 that displays the decoded video signal such as a display. In addition, the TV ex300 includes an interface node ex317 including an operation input node ex312 that receives an action input of a user. In addition, the TV ex300 includes a control unit ex310 that generally controls each constituent element of the TV ex300, and a power circuit ex311 that supplies power to each of the elements. In addition to the operation input portion ex312, the interface portion ex317 may include: a bridge ex313 that connects to an external device such as a reader/writer ex218; a connector ex314 for connecting a recording medium ex216 such as an SD card; a drive ex315 to be connected to an external recording medium such as a hard disk; and an ex316 modem to be connected to the telephone network. Here, the recording medium ex216 can electrically record information using a non-volatile/volatile semiconductor memory element for storage. The constituent elements of the TV ex300 are connected to each other via a synchronous bus.

[0158] First, a configuration in which the TV ex300 decodes the multiplexed data received from the outside through the antenna ex204 and others will be described, and reproduces the decoded data. In the TV ex300, as a result of receiving the user's operation from the remote control ex220 and others, the multiplexing/demultiplexing part ex303 demultiplexes the multiplexed data demodulated by the modulation/demodulation part ex302 under the control of the control part ex310 including the CPU. Further, in the TV ex300, the audio signal processing portion ex304 decodes the demultiplexed audio data, and the video signal processing portion ex305 decodes the demultiplexed video data using the decoding method described in each of the Embodiments. The output node ex309 provides the decoded video signal and audio signal to the outside. When the output node ex309 provides a video signal and an audio signal, the signals can be temporarily stored in buffers ex318 and ex319 and others so that the signals are reproduced in sync with each other. In addition, the TV ex300 can read the multiplexed data not through broadcasting and the like, but from recording media ex215 and ex216 such as a magnetic disk, an optical disk, and an SD card. Next, a configuration in which the TV ex300 encodes an audio signal and a video signal and transmits data to the outside or writes data to a recording medium will be described. In the TV ex300, as a result of receiving the user's operation from the remote control ex220 and others, the audio signal processing unit ex304 encodes the audio signal, and the video signal processing unit ex305 encodes the video signal under the control of the control unit ex310 using the coding method described in each of the Embodiments. The multiplexing/demultiplexing section ex303 multiplexes the encoded video signal and audio signal, and provides the resulting signal to the outside. When the multiplexing/demultiplexing section ex303 multiplexes the video signal and the audio signal, the signals can be temporarily stored in the buffers ex320 and ex321 and others so that the signals are reproduced in synchronism with each other. Here, the buffers ex318-ex321 may be multiple, as illustrated, or at least one buffer may be shared in the TV ex300. In addition, data may be stored in a buffer other than the buffers ex318 to ex321 so that overflow and underflow of the system can be avoided, for example, between the modulation/demodulation node ex302 and the multiplexing/demultiplexing node ex303.

[0159] In addition, the TV ex300 may include a configuration for receiving audiovisual input data from a microphone or a camera, in addition to the configuration for receiving audio data and video data from a broadcast or from a recording medium, and may encode the received data. Although in the description, the TV ex300 can encode, multiplex, and provide data to the outside, it may not allow all processes to be performed, but only one of receiving, decoding, and providing data to the outside.

[0160] In addition, when the reader/writer ex218 reads or writes the multiplexed data from or to the recording medium, one of the TV ex300 and the reader/writer ex218 can decode or encode the multiplexed data, and the TV ex300 and the reader/writer ex218 may distribute decoding or encoding.

[0161] As an example, FIG. 19 illustrates the configuration of the information playback/recording unit ex400 when data is read from or written to or from an optical disc. The information playback/recording portion ex400 includes constituent elements ex401 to ex407 to be described below. The optical pickup ex401 emits a laser spot onto the recording surface of the recording medium ex215, which is an optical disc, to record information, and detects reflected light from the recording surface of the recording medium ex215 to read information. The modulation recording unit ex402 electrically controls the semiconductor laser included in the optical pickup ex401 and modulates the laser light in accordance with the recorded data. The playback demodulation unit ex403 amplifies the playback signal obtained by electrically detecting reflected light from the recording surface using the photodetector included in the optical head ex401, and demodulates the playback signal by separating the signal component recorded on the recording medium ex215 to reproduce the necessary information. The buffer ex404 temporarily stores information to be recorded on the recording medium ex215 and information reproduced from the recording medium ex215. The disc motor ex405 rotates the recording medium ex215. The servo control unit ex406 moves the optical pickup ex401 to a predetermined information track while controlling the rotation of the disc motor ex405 to follow the laser spot. The system control section ex407 controls the information playback/recording section ex400 as a whole. The read and write processes can be implemented by the system control unit ex407 using various information stored in the buffer ex404 and generating and adding new information as needed, and the modulation recording unit ex402, the playback demodulation unit ex403, and the servo control unit ex406, which record and reproduce information via the optical pickup ex401 while being controlled in a consistent manner. The system control unit ex407 includes, for example, a microprocessor and performs processing by causing the computer to execute a program for reading and writing.

[0162] Although the optical head ex401 emits a laser spot in the description, it can perform high-density recording using near-field light.

[0163] FIG. 20 schematically illustrates the recording medium ex215, which is an optical disc. On the recording surface of the recording medium ex215, guide grooves are formed in a spiral manner, and the information track ex230 records in advance address information indicating an absolute position on the disc in accordance with a change in the shape of the guide grooves. The address information includes information for determining the positions of the recording units ex231, which are the unit for recording data. A device that records and reproduces data reproduces the information track ex230 and reads the address information to determine the positions of the recording blocks. In addition, the recording medium ex215 includes a data recording area ex233, an inner circumference area ex232, and an outer circumference area ex234. The data recording area ex233 is an area to be used when recording user data. The inner circumference area ex232 and the outer circumference area ex234, which are inside and outside the data recording area ex233, respectively, are for special use except for recording user data. The information playback/recording portion ex400 reads and writes encoded audio data, encoded video data, or multiplexed data obtained by multiplexing encoded audio data and encoded video data from and to the data recording area ex233 of the recording medium ex215.

[0164] Although the description describes an optical disc containing a layer such as a DVD and a BD as an example, the optical disc is not limited thereto and may be an optical disc having a multi-layer structure and capable of being recorded on parts other than the surface. In addition, the optical disc may have a structure for multi-dimensional recording/reproducing, such as recording information using light with colors of different wavelengths in the same part of the optical disc, and recording information having different layers from different angles.

[0165] In addition, in the digital broadcast system ex200, the vehicle ex210 having the antenna ex205 can receive data from the satellite ex202 and others, and display video on a display device such as the vehicle navigation system ex211 installed in the vehicle ex210. Here, the configuration of the car navigation system ex211 will be a configuration, for example, including a GPS receiving unit in the configuration illustrated in FIG. 18. The same will be true for the configuration of the computer ex111, cell phone ex114 and others.

[0166] FIG. 21A illustrates a cell phone ex114 that uses the movie encoding method and the movie decoding method described in each of the Embodiments. The cell phone ex114 includes: an antenna ex350 for transmitting and receiving radio waves through the base station ex110; a camera ex365 capable of capturing moving and still images; and a display unit ex358 such as a liquid crystal display for displaying data such as a decoded video captured by the camera ex365 or received by the antenna ex350. The cell phone ex114 further includes: a body including a set of control buttons ex366; an audio output unit ex357 such as a speaker for audio output; an audio input unit ex356 such as a microphone for audio input; a storage device ex367 for storing captured video or still images, recorded audio, encoded or decoded data of the received video, still images, emails, or the like; and a connector ex364 which is an interface node for the recording medium which stores data in the same manner as the storage device ex367.

[0167] Next, a configuration example of the cell phone ex114 will be described with reference to FIG. 21b. In the cellular phone ex114, the main control node ex360, configured to control each body node including the display node ex358 as well as the control buttons ex366 as a whole, is mutually connected via the synchronous bus ex370 to the power circuit ex361, the operation input control node ex362, the processing node ex355 a video signal interface unit ex363, a liquid crystal display (LCD) driving unit ex359, a modulation/demodulation unit ex352, a multiplexing/demultiplexing unit ex353, an audio processing unit ex354, a connector ex364, and a storage device ex367.

[0168] When a call end key or a power key is pressed as a result of user action, the power circuit ex361 supplies battery power to the respective nodes to wake up the cellular phone ex114.

[0169] In the cell phone ex114, the audio signal processing unit ex354 converts the audio signals collected by the audio input unit ex356 in speech conversion mode into digital audio signals under the control of the main control unit ex360 including the CPU, ROM, and RAM. Then, the modulation/demodulation section ex352 performs spreading processing on the digital audio signals, and the transmission and reception section ex351 performs digital-to-analogue and frequency conversion on the data to transmit the resulting data through the antenna ex350. Then, the modulation/demodulation section ex352 performs inverse spreading processing on the data, and the audio signal processing section ex354 converts them into analog audio signals to be output through the audio output section ex357.

[0170] In addition, when e-mail is transmitted in data transmission mode, e-mail text data entered by operating the control buttons ex366 and others in the body is sent to the main control node ex360 via the operation input control node ex362. The main control node ex360 causes the modulation/demodulation node ex352 to perform spreading processing on the text data, and the transmission and reception node ex351 performs D/A and frequency conversion on the resulting data to transmit the data to the base station ex110 via the antenna ex350. When an email is received, processing is performed on the received data that is approximately the reverse of that for sending an email, and the resulting data is provided to the display node ex358.

[0171] When video, still images, or video and audio are transmitted in data mode, the video signal processing unit ex355 (functioning as an image encoding device according to an aspect of the present invention) compresses and encodes the video signals received from the camera ex365 using a motion picture encoding method, shown in each of the Embodiments, and transmits the encoded video data to the multiplexing/demultiplexing node ex353. In contrast, while the camera ex365 is capturing video, still images, and the like, the audio processing unit ex354 encodes the audio signals collected by the audio input unit ex356 and transmits the encoded audio data to the multiplexing/demultiplexing unit ex353.

[0172] The multiplexing/demultiplexing section ex353 multiplexes the encoded video data received from the video signal processing section ex355 and the encoded audio data received from the audio signal processing section ex354 using a predetermined method. Then, the modulation/demodulation section ex352 performs spreading processing on the multiplexed data, and the transmission and reception section ex351 performs digital-to-analogue and frequency conversion on the data to transmit the resulting data through the antenna ex350.

[0173] When receiving video file data that is linked to a web page and others in a data mode, or when receiving an e-mail with an attached video image and/or audio, in order to decode the multiplexed data received through the antenna ex350, the multiplexing/demultiplexing unit ex353 demultiplexes the multiplexed data into a video bit stream and an audio bit stream, and supplies the video signal processing unit ex355 with encoded video data and the audio signal processing unit ex354 with encoded audio data via the synchronous bus ex370. The video signal processing unit ex355 (functioning as an image decoding apparatus according to an aspect of the present invention) decodes the video signal using a motion picture decoding method corresponding to the motion picture encoding method shown in each of Embodiments, and then the display portion ex358 displays, for example, a video image and still images included in the video file associated with the web page by the LCD control portion ex359. In addition, the audio signal processing portion ex354 decodes the audio signal, and the audio output portion ex357 provides audio.

[0174] In addition, similar to the TV ex300, the terminal, such as the cellular phone ex114, possibly has 3 types of execution configurations including not only (i) a transmitting and receiving terminal including an encoding device and a decoding device, but also (ii) a transmitting terminal including only an encoding device; and (iii) a receiving terminal including only a decoding device. Although the digital broadcast system ex200 receives and transmits multiplexed data obtained by multiplexing video data with audio data in the description, the multiplexed data may be data obtained by multiplexing video data not with audio data but with character data related to the video image, and may not be multiplexed data but itself. video data.

[0175] As such, the movie encoding method or movie decoding method in each of the Embodiments can be used on any of the devices and systems described. Thus, the advantages described in each of the Embodiments can be obtained.

[0176] In addition, the present invention is not limited to Embodiments, and various modifications and changes are possible without deviating from the scope of the present invention.

[0177] (Embodiment 4)

The video data may be generated by switching, as needed, between (i) a movie encoding method or a movie encoding device shown in each of the Embodiments, and (ii) a movie encoding method or a movie encoding device according to another standard such as MPEG-2, MPEG4-AVC and VC-1.

[0178] Here, when a plurality of video data that conforms to different standards is generated and then decoded, decoding methods that conform to different standards need to be selected. However, since it cannot be determined which standard each of the plurality of video data to be decoded conforms to, there is a problem that an appropriate decoding method cannot be selected.

[0179] In order to solve this problem, multiplexed data obtained by multiplexing video data with audio data and others has a structure including identification information indicating which standard the video data conforms to. The specific structure of the multiplexed data including video data generated by the movie encoding method or the movie encoding apparatus shown in each of the Embodiments will be described below. The multiplexed data is a digital stream in the MPEG-2 transport stream format.

[0180] FIG. 22 illustrates the structure of the multiplexed data. As illustrated in FIG. 22, multiplexed data can be obtained by multiplexing at least one of a video stream, an audio stream, a demo graphics (PG) stream, and an interactive graphics stream. The video stream represents the primary video and the secondary video of the movie, the audio stream represents the primary audio and the secondary audio to be mixed with the primary audio, and the demo graphics stream represents the subtitles of the movie. Here, the primary video is a normal video displayed on a screen, and the secondary video is a video to be displayed on a smaller window in the main video. In addition, the interactive graphics stream represents an interactive screen to be formed by placing GUI components on the screen. The video stream is encoded by a movie encoding method or a movie encoding device shown in each of the Embodiments, or a movie encoding method or a movie encoding device according to a conventional standard such as MPEG-2, MPEG4-AVC and VC-1. The audio stream is encoded according to a standard such as Dolby-AC-3, Dolby Digital Plus, MLP, DTS, DTS-HD and Linear PCM.

[0181] Each stream included in the multiplexed data is identified by a PID. For example, 0x1011 is assigned to the video stream that will be used for the movie's video, 0x1100-0x111F is assigned to the audio streams, 0x1200-0x121F is assigned to the demo graphics streams, 0x1400-0x141F is assigned to the interactive graphics streams, 0x1B00-0x1B1F is assigned to the video streams to be used for the movie's secondary video, and 0x1A00-0x1A1F are assigned to the audio streams that will be used for the secondary video to be mixed with the primary audio.

[0182] FIG. 23 schematically illustrates how data is multiplexed. First, the video stream EX235, consisting of video frames, and an audio operating system EX238, consisting of audio cadres, are converted into an EX236 PES PES and EX239 stream PES, and additionally into EX237 TS packages and EX240 TS packages, respectively. Similarly, the data from the EX241 stream of demonstration graphics and data from the EX244 flow of interactive graphics are converted into an EX242 PES stream and the EX245 stream of PES packages, and additionally into EX243 TS packages and EX246 TS packages, respectively. These TS packages are multiplexed in the stream to obtain multiplexed data EX247.

[0183] FIG. 24 In more detail, it illustrates how the video stream persists in the stream of PES packages. The first strip in Fig. 24 shows the flow of video frames in the video stream. The second strip shows the stream of PES packages. As indicated by arrows indicated by YY1, YY2, YY3 and YY4 in FIG. 24, the video stream is divided into I-pictures, B-pictures, and P-pictures, each of which is a video demonstration unit, and the pictures are stored in the payload of each of the PES packets. Each of the PES packets has a PES header, and the PES header stores a Playback Time Stamp (PTS) indicating the display time of the image and a Decoding Time Stamp (DTS) indicating the decoding time of the image.

[0184] FIG. 25 illustrates the format of the TS packets that are ultimately to be written to the multiplexed data. Each of the TS packets is a 188-byte fixed-length packet, including a 4-byte TS header containing information such as a PID for identifying a stream, and a 184-byte TS payload for storing data. PES packets are split and stored in TS payloads. When a BD ROM is used, each of the TS packets is given a 4-byte TP_Extra_Header, respectively resulting in 192-byte source packets. The source packets are written to the multiplexed data. TP_Extra_Header stores information such as Arrival_Time_Stamp (ATS). The ATS indicates the transmission start time at which each of the TS packets must be sent to the PID filter. The numbers incrementing from the beginning of the multiplexed data are called Source Packet Numbers (SPNs), as shown at the bottom of FIG. 25.

[0185] Each of the TS packets included in the multiplexed data includes not only audio, video, subtitle, and other streams, but also a Program Relationship Table (PAT), a Program Mapping Table (PMT), and a Program Reference Signal (PCR). The PAT shows what the PID in the PMT used in the multiplexed data indicates, and the PID of the PAT itself is registered as zero. The PMT stores the PIDs of the video, audio, subtitle and other streams included in the multiplexed data, and attribute information of the streams corresponding to the PIDs. The PMT also contains various descriptors related to the multiplexed data. The descriptors have information, such as copy control information, indicating whether copying of the multiplexed data is allowed. The PCR stores STC timing information corresponding to the ATS indicating when the PCR packet is transmitted to the decoder to achieve synchronization between the Arrival Timer (ATC) which is the time axis of the ATS and the System Timer (STC) which is the time axis of the PTS and DTS.

[0186] FIG. 26 illustrates the PMT data structure in detail. The PMT header is located at the beginning of the PMT. The PMT header describes the length of the data included in the PMT and so on. Following the PMT header is a set of descriptors related to the multiplexed data. Descriptors describe information, such as copy control information. Following the descriptors are a plurality of stream information pieces relating to the streams included in the multiplexed data. Each piece of stream information includes stream descriptors describing information such as a stream type to identify a stream compression codec, a stream PID, and stream attribute information (eg, frame rate or aspect ratio). The stream descriptors are equal in number to the number of streams in the multiplexed data.

[0187] When the multiplexed data is recorded on a recording medium and others, it is recorded along with information files of the multiplexed data.

[0188] Each of the multiplexed data information files is the multiplexed data control information as shown in FIG. 27. The information files of the multiplexed data are in one-to-one correspondence with the multiplexed data, and each of the files includes information about the multiplexed data, information about the stream attributes, and an input map.

[0189] As illustrated in FIG. 27, information about the multiplexed data includes a system rate, a playback start time, and a playback end time. The system rate indicates the maximum bit rate at which the final system decoder, which will be described later, transmits the multiplexed data to the PID filter. The intervals of the ATS included in the multiplexed data are set no higher than the system rate. The playback start time indicates the PTS in the video frame at the beginning of the multiplexed data. An interval of one frame is added to the PTS in the video frame at the end of the multiplexed data, and the PTS is set at the end of playback.

[0190] As shown in FIG. 28, a piece of attribute information is registered in the stream attribute information for each PID of each stream included in the multiplexed data. Each piece of attribute information contains different information depending on whether the corresponding stream is a video stream, an audio stream, a demo graphics stream, or an interactive graphics stream. Each piece of video stream attribute information carries information including which kind of compression codec is used to compress the video stream, and the resolution, aspect ratio, and frame rate of the pieces of image data that are included in the video stream. Each piece of audio stream attribute information carries information including what kind of compression codec is used to compress the audio stream, how many channels are included in the audio stream, which language the audio stream supports, and what is the sampling rate. The video stream attribute information and the audio stream attribute information are used to initialize the decoder before the player plays the information.

[0191] In Embodiment 4, the multiplexed data to be used belongs to the stream type included in the PMT. In addition, when the multiplexed data is recorded on the recording medium, the video stream attribute information included in the multiplexed data information is used. More precisely, the method of coding a movie or a movie coding device described in each of the options, includes a stage or a node for the appointment of unique information indicating video data formed according to the method of coding or using the coding device in each of the options, type stream included in the PMT, or video stream attribute information. With this structure, video data generated by the movie encoding method or by the movie encoding apparatus described in each of Embodiments can be distinguished from video data that conforms to another standard.

[0192] In addition, FIG. 29 illustrates the steps of the movie decoding method according to Embodiment 4. In step exS100, the stream type included in the PMT or video stream attribute information is obtained from the multiplexed data. Next, in step exS101, it is determined whether the stream type or video stream attribute information indicates that the multiplexed data is generated by the movie encoding method or by the movie encoding apparatus in each of the Embodiments. When it is determined that the stream type or the video stream attribute information indicates that the multiplexed data is generated by the movie encoding method or by the movie encoding apparatus in each of the Embodiments, in step exS102, the stream type or the video stream attribute information is decoded by the movie decoding method in each of the Embodiments. In addition, when the stream type or video stream attribute information indicates compliance with conventional standards such as MPEG-2, MPEG4-AVC, and VC-1, in step exS103, the stream type or video stream attribute information is decoded according to the conventional standard movie decoding method. .

[0193] As such, assigning a new unique value to a stream type or video stream attribute information makes it possible to determine whether the movie decoding method or movie decoding apparatus that are described in each of the Embodiments can perform decoding. Even when inputting multiplexed data that conforms to a different standard, a suitable decoding method or device can be selected. Thus, it becomes possible to decode information without any error. In addition, the movie encoding method or apparatus or the movie decoding method or apparatus in Embodiment 4 can be used in the devices and systems described above.

[0194] (Embodiment 5)

Each of the movie encoding method, the movie encoding device, the movie decoding method, and the movie decoding device in each of the Embodiments is typically implemented as an integrated circuit or a Large Scale Integrated Circuit (LSI). As an example, the LSI of FIG. 30 illustrates the LSI ex500 configuration that has been made into a single chip. The LSI ex500 includes elements ex501, ex502, ex503, ex504, ex505, ex506, ex507, ex508, and ex509 to be described below, and these elements are connected to each other via a bus ex510. The power circuit ex505 is activated by supplying power to each of the cells when the power circuit ex505 is turned on.

[0195] For example, when encoding is performed, the LSI ex500 receives an audiovisual signal from a microphone ex117, a camera ex113, and others via an audiovisual input/output ex509 under the control of a control unit ex501 including a CPU ex502, a memory controller ex503, a flow controller ex504, and a node ex512 excitation frequency adjustment. The received audiovisual signal is temporarily stored in an external storage device ex511, such as SDRAM. Under the control of the control node ex501, the stored data is segmented into pieces of data according to the processing amount and speed, to be transmitted to the signal processing node ex507. Then, the signal processing unit ex507 encodes the audio signal and/or the video signal. Here, the video encoding is the encoding described in each of Embodiments. In addition, the signal processing part ex507 sometimes multiplexes the encoded audio data and the encoded video data, and the stream I/O ex506 provides the multiplexed data to the outside. The provided multiplexed data is transmitted to the base station ex107 or recorded on the recording media ex215. When data sets are multiplexed, the data sets should be temporarily stored in the buffer ex508 so that the data sets are synchronized with each other.

[0196] Although the storage device ex511 is an element outside the LSI ex500, it may be included in the LSI ex500. The buffer ex508 is not limited to one buffer, but may be composed of buffers. In addition, the LSI ex500 can be converted into a single chip or multiple chips.

[0197] In addition, although the control unit ex501 includes a CPU ex502, a memory controller ex503, a flow controller ex504, a drive frequency adjustment unit ex512, the configuration of the control unit ex501 is not limited as such. For example, the signal processing node ex507 may further include a CPU. The inclusion of another CPU in the EX507 node of signal processing can increase the processing speed. In addition, as another example, the CPU ex502 may include a signal processing part ex507 or an audio signal processing part that is part of the signal processing part ex507. In such a case, the control portion ex501 includes a signal processing portion ex507 or a CPU ex502 including a part of the signal processing portion ex507.

[0198] The name used here is LSI, but it can also be called systemic LSI, super-LSI, or ultra-LSI, depending on the degree of integration.

[0199] In addition, methods for achieving integration are not limited to LSI, and a special circuit or general purpose processor, etc. can also provide integration. For the same purpose, a field programmable gate array (FPGA) that can be programmed after LSI production, or a reconfigurable processor that allows connection reconfiguration or LSI configuration can be used. The programmable logic device can generally execute the movie encoding method and the movie decoding method according to Embodiments and Variations by downloading or reading from a memory device a program included in software or firmware.

[0200] In the future, with the development of semiconductor technology, a completely new technology can replace LSI. Function blocks can be integrated using such technology. It is likely that the present invention is applied to biotechnology.

[0201] (Embodiment 6)

Когда декодируются видеоданные, сформированные по способу кодирования кинофильма или с помощью устройства кодирования кинофильма, описанных в каждом из Вариантов осуществления, по сравнению со случаем декодирования видеоданных, которые соответствуют традиционному стандарту, например MPEG-2, MPEG4-AVC и VC-1, объем вычислений is likely to increase. Thus, the LSI EX500 must be adjusted for the excitation frequency higher than that of the CPU EX502, which should be used when video data is decoded in accordance with the traditional standard. However, when the frequency of excitation is established above, there is a problem that energy consumption increases.

[0202] In order to solve this problem, a movie decoding device such as a TV ex300 and LSI ex500 is configured to determine which standard the video data conforms to and switch between drive frequencies according to the determined standard. Fig. 31 illustrates the configuration of ex800 in Embodiment 6. The drive frequency switching portion ex803 sets the drive frequency to a higher drive frequency when video data is generated by the movie encoding method or the movie encoding device described in each of Embodiments. Then, the driving frequency switching portion ex803 instructs the decoding processing portion ex801, which executes the movie decoding method described in each of Embodiments, to decode video data. When the video data conforms to the conventional standard, the driving frequency switching portion ex803 sets the driving frequency to a lower driving frequency than the video data generated by the movie encoding method or the movie encoding apparatus described in each of Embodiments. Then, the driving frequency switching portion ex803 instructs the decoding processing portion ex802, which conforms to the conventional standard, to decode the video data.

[0203] More specifically, the drive frequency switching portion ex803 includes the CPU ex502 and the drive frequency adjustment portion ex512 in FIG. 30. Here, each of the decoding processing unit ex801 that executes the movie decoding method described in each of the Embodiments and the decoding processing unit ex802 that conforms to the conventional standard corresponds to the signal processing unit ex507 in FIG. 30. The CPU ex502 determines which standard the video data conforms to. Then, the drive frequency adjustment portion ex512 determines the drive frequency based on the signal from the CPU ex502. In addition, the signal processing part ex507 decodes video data based on a signal from the CPU ex502. For example, the identification information described in Embodiment 4 is most likely used to identify video data. The identification information is not limited to that described in Embodiment 4, but may be any information as long as the information indicates which standard the video data conforms to. For example, when it can be determined which standard the video data conforms to, based on the external signal to determine that the video data is used for television or disc, etc., the determination can be performed based on such an external signal. In addition, the CPU ex502 selects a drive frequency based on, for example, a look-up table in which video data standards are associated with drive frequencies, as shown in FIG. 33. The drive frequency can be selected by storing a look-up table in the buffer ex508 and the LSI internal memory, and by referring to the look-up table with the CPU ex502.

[0204] FIG. 32 illustrates steps for performing the method in Embodiment 6. First, in step exS200, the signal processing unit ex507 obtains identification information from the multiplexed data. Next, in step exS201, the CPU ex502 determines whether video data is generated based on the identification information with the encoding method and encoding device described in each of the Embodiments. When video data is generated by the encoding method or the encoding apparatus described in each of the Embodiments, in step exS202, the CPU ex502 sends a signal to the drive frequency adjustment portion ex512 to set the drive frequency to a higher drive frequency. Then, the drive frequency adjusting portion ex512 sets the drive frequency to a higher drive frequency. On the other hand, when the identification information indicates that the video data conforms to the conventional standard such as MPEG-2, MPEG4-AVC and VC-1, in step exS203, the CPU ex502 sends a signal to the drive frequency adjustment portion ex512 to set the drive frequency to a lower drive frequency. Then, the driving frequency adjusting portion ex512 sets the driving frequency to a lower driving frequency than in the case where video data is generated by the encoding method or encoding device described in each of the Embodiments.

[0205] In addition, along with switching excitation frequencies, the effect of energy saving by changing the voltage, which should be supplied to the LSI EX500 or a device that includes the LSI EX500. For example, when the excitation frequency is set below, the voltage that should be supplied to the LSI EX500 or a device that includes the LSI EX500 is most likely set to voltage lower than in the case where the excitation frequency is set higher.

[0206] In addition, as a method for setting the drive frequency, when the calculation amount for decoding is larger, the drive frequency can be set higher, and when the amount of calculation for decoding is smaller, the drive frequency can be set lower. Thus, the installation method is not limited to the methods described above. For example, when the amount of computation for decoding video data according to MPEG4-AVC is larger than the amount of computation for decoding video data generated by the motion picture encoding method or the motion picture encoding apparatus described in each of the Embodiments, the driving frequency is most likely set in the reverse order to that described. above installation.

[0207] In addition, the method for setting the drive frequency is not limited to the method for setting the drive frequency lower. For example, when the identification information indicates that the video data is generated by the movie encoding method or the movie encoding device described in each of the Embodiments, the voltage to be applied to the LSI ex500 or the device including the LSI ex500 is likely to be set higher. When the identification information indicates that the video data conforms to the conventional standard such as MPEG-2, MPEG4-AVC and VC-1, the voltage to be supplied to the LSI ex500 or a device including the LSI ex500 is likely to be set lower. As another example, when the identification information indicates that the video data is generated by the movie encoding method or by the video encoding apparatus described in each of the Embodiments, the driving of the CPU ex502 does not likely need to be suspended. When the identification information indicates that the video data conforms to the conventional standard such as MPEG-2, MPEG4-AVC, and VC-1, the driving of the CPU ex502 is likely to be suspended at a given time because the CPU ex502 has excess processing capacity. Even when the identification information indicates that the video data is generated by the motion picture encoding method or the motion picture encoding apparatus described in each of the Embodiments, in the case where the CPU ex502 has excessive processing capacity, the driving of the CPU ex502 is likely to be suspended at a given time. . In such a case, the pause time is likely to be set shorter than when the identification information indicates that the video data conforms to a conventional standard such as MPEG-2, MPEG4-AVC and VC-1.

[0208] Accordingly, it is possible to enhance the power saving effect by switching between driving frequencies according to the standard to which the video data conforms. In addition, when the LSI ex500 or a device including the LSI ex500 is powered using a battery, the battery life can be extended along with a power saving effect.

[0209](Embodiment 7)

There are cases where a plurality of video data that conforms to different standards is provided to devices and systems such as a TV and a cell phone. In order to be able to decode a plurality of video data that conform to different standards, the signal processing unit ex507 in the LSI ex500 needs to conform to different standards. However, there are problems of scaling up the LSI ex500 circuit and increasing the cost by separately using signal processing units ex507 that conform to the relevant standards.

[0210] In order to solve these problems, a configuration is presented in which a decoding processing node for implementing the movie decoding method described in each of the Embodiments, and a decoding processing node that conforms to a conventional standard such as MPEG-2, MPEG4-AVC and VC- 1 are partially shared. Ex900 in FIG. 34A shows an example of this configuration. For example, the motion picture decoding method described in each of Embodiments and the motion picture decoding method that conforms to MPEG4-AVC share some processing details such as entropy encoding, inverse quantization, deblocking filtering, and motion compensation. The processing details to be shared most likely include the use of a decoding processing node ex902 that conforms to MPEG4-AVC. In contrast, the dedicated decoding processing unit ex901 may be used for other processing that is unique to the present invention and does not conform to MPEG-4 AVC. A decoding processing unit for implementing the movie decoding method described in each of Embodiments may be shared for processing to be distributed, and a dedicated decoding processing unit may be used for processing unique to MPEG4-AVC processing.

[0211] In addition, ex1000 in FIG. 34B shows another example in which processing is partially distributed. This example uses a configuration including a dedicated decoding processing node ex1001 that supports processing unique to an aspect of the present invention, a dedicated decoding processing node ex1002 that supports processing unique to another conventional standard, and a decoding processing node ex1003 that supports processing to be distributed between the movie decoding method in the aspect of the present invention and the conventional movie decoding method. Here, the dedicated decoding processing nodes ex1001 and ex1002 are not necessarily dedicated to the processing of an aspect of the present invention and conventional standard processing, but may be nodes capable of realizing common processing. Moreover, the configuration of Embodiment 7 can be implemented with the LSI ex500.

[0212] Essentially, reducing the scale of the LSI scheme and cost reduction are possible by joint use of the decoding processing unit for processing, which needs to be distributed between the method of decoding the movie in the present invention and the method of decoding the movie in accordance with the traditional standard.

[Industrial applicability]

[0213] The present invention has an advantage of accelerating processing almost without reducing the effectiveness of coding and applicable to various use, such as the accumulation, transmission and exchange of data. The present invention is applicable, for example, to information display devices and image capture devices, such as TVs, digital video recording devices, automobile navigation systems, mobile phones, digital cameras and digital video cameras, and is very suitable for practical use.

[Tag List]

[0214] 100 Image encoding device

101 Block Breaker

102 Subtraction node

103 Conversion node

104 Variable length encoding node

105, 202 Reverse conversion unit

106, 203 Addition unit

107, 204 SAO Processing Unit

108, 206 Frame memory

109, 207 Prediction Knot

110 SAO parameter coding unit

111 Encoding node sao_type_idx

112 Encoding node sao_offset

113 Node convert sao_offset to binary

114 Node switching arithmetic coding

115 Arithmetic coding node context 0

116 Node Arithmetic Coding Context 1

117 Bypass arithmetic coding unit

200 Image decoder

201 Variable length decoding node

205 Decoded block merge unit

210 SAO parameter decoding unit

211 Decoding node sao_type_idx

212 Decoding node sao_offset

213 Arithmetic decoding switching unit

214 Node arithmetic decode context 0

215 Node arithmetic decoding context 1

216 Bypass arithmetic decoding unit

217 sao_offset inverse transform node from binary form

Claims (30)

1. An image encoding method, comprising the steps of: determining an adaptive sample offset (SAO) type parameter for the SAO value to be added to the pixel value of the reconstructed image; encoding the SAO type parameter into a bitstream; determining an integer indicating the magnitude of the SAO value; encoding said specific integer using variable length coding into a plurality of bins, each bin being encoded into a bit stream using constant probability bypass arithmetic coding. 2. The method of claim 1, wherein the constant probability is a 50% probability. 3. The method of claim 1, wherein the SAO type parameter indicates that the SAO value is a boundary offset value. 4. The method of claim 1, wherein the SAO type parameter indicates that the SAO value is a range offset value. 5. An image decoding device, comprising: at least one processor; at least one memory device containing computer program code, wherein the memory device and the computer program code are configured, operating with at least one processor, to cause the device to perform at least the following: decode from the bitstream an adaptive sample offset (SAO) type parameter; determine from the SAO type parameter the SAO type for the SAO value to be added to the pixel value of the reconstructed image; decode from the bitstream using constant probability bypass arithmetic encoding a plurality of bins with variable length encoding, the plurality of bins representing an integer indicating the magnitude of the SAO value; determine the whole number indicating the value of the value of the SAO, on the basis of many beans, decoded using a bypass arithmetic coding with constant probability; And determine the value of SAO using said specific integer. 6. The apparatus of claim 5, wherein the storage device and the computer program code are configured, in operation with at least one processor, to cause the device to add an SAO value to an image pixel value. 7. The apparatus of claim 5, wherein the constant probability is a 50% probability. 8. The device according to claim 5, in which the SAO parameter indicates that the value of the SAO is the value of the displacement along the border. 9. The apparatus of claim 5, wherein the SAO type parameter indicates that the SAO value is a range offset value. 10. An image decoding method, comprising the steps of: decoding from the bitstream a type parameter of adaptive sample offset (SAO); Determine from the SAO type parameter for the SAO value to be added to the value of the pixel of the restored image; Decoded from a bit flow using a bypass arithmetic coding with constant probability of many bins with encoding variable lengths, and many bins are an integer indicating the value of the value of the SAO; determining an integer indicating the magnitude of the SAO value based on the plurality of bins decoded using constant probability bypass arithmetic coding; And determining the value of SAO using said specific integer. 11. The method of claim 10, further comprising adding an SAO value to an image pixel value. 12. The method of claim 10, wherein the constant probability is a 50% probability. 13. The method of claim 10, wherein the SAO type parameter indicates that the SAO value is a boundary offset value. 14. The method according to claim 10, in which the SAO parameter indicates that the value of the SAO is the value of the displacement according to the range.

Images