JP6299901B2 - Image processing apparatus and method, program, and recording medium - Google Patents

Description

  The present invention relates to an image processing apparatus and method, a program, and a recording medium, and in particular, an image processing apparatus, method, and program capable of improving encoding efficiency while suppressing reduction in encoding processing efficiency. And a recording medium.

  In recent years, image information is handled as digital data, and MPEG (compressed by orthogonal transform such as discrete cosine transform and motion compensation is used for the purpose of efficient transmission and storage of information. A device conforming to a system such as Moving Picture Experts Group) is becoming popular in both information distribution at broadcasting stations and information reception in general households.

  In particular, MPEG2 (ISO (International Organization for Standardization) / IEC (International Electrotechnical Commission) 13818-2) is defined as a general-purpose image coding system, which includes both interlaced scanning images and progressive scanning images, as well as standard resolution images and This standard covers high-definition images and is currently widely used in a wide range of professional and consumer applications. By using the MPEG2 compression method, for example, a standard resolution interlaced scanning image having 720 × 480 pixels is 4 to 8 Mbps, and a high resolution interlaced scanning image having 1920 × 1088 pixels is 18 to 22 Mbps. (Bit rate) can be assigned to achieve a high compression rate and good image quality.

  MPEG2 was mainly intended for high-quality encoding suitable for broadcasting, but did not support encoding methods with a lower code amount (bit rate) than MPEG1, that is, a higher compression rate. With the widespread use of mobile terminals, the need for such an encoding system is expected to increase in the future, and the MPEG4 encoding system has been standardized accordingly. Regarding the image coding system, the standard was approved as an international standard as ISO / IEC 14496-2 in December 1998.

  Furthermore, in recent years, the standardization of the standard called H.26L (ITU-T (International Telecommunication Union Telecommunication Standardization Sector) Q6 / 16 VCEG (Video Coding Expert Group)) has progressed for the purpose of image coding for the initial video conference. Yes. H.26L is known to achieve higher encoding efficiency than the conventional encoding schemes such as MPEG2 and MPEG4, although a large amount of calculation is required for encoding and decoding. Currently, as part of MPEG4 activities, standardization to achieve higher coding efficiency based on this H.26L and incorporating functions not supported by H.26L is performed as Joint Model of Enhanced-Compression Video Coding. It has been broken.

  The standardization schedule became an international standard in March 2003 under the names of H.264 and MPEG-4 Part 10 (Advanced Video Coding, hereinafter referred to as AVC).

  Furthermore, as an extension, RGB, 4: 2: 2, 4: 4: 4 encoding tools necessary for business use, 8x8DCT (Discrete Cosine Transform) and quantization matrix specified by MPEG-2 are added. FRExt (Fidelity Range Extension) standardization was completed in February 2005, and as a result, it became an encoding method that can well express film noise contained in movies using AVC. It has been used for a wide range of applications such as Blu-Ray Disc.

  However, in recent years, even higher compression ratios such as wanting to compress images of about 4000 x 2000 pixels, which is four times higher than high-definition images, or distributing high-definition images in a limited transmission capacity environment such as the Internet. There is a growing need for encoding. For this reason, in the VCEG (Video Coding Expert Group) under the ITU-T, studies on improving the coding efficiency are being continued.

  By the way, for the purpose of further improving the coding efficiency than AVC, HEVC (High Efficiency Video Coding) is being developed by JCTVC (Joint Collaboration Team-Video Coding), a joint standardization organization of ITU-T and ISO / IEC. Is being standardized (for example, see Non-Patent Document 1).

  In this HEVC encoding system, a coding unit (CU (Coding Unit)) is defined as a processing unit similar to a macroblock in AVC. The CU is not fixed to a size of 16 × 16 pixels like the AVC macroblock, and is specified in the image compression information in each sequence.

  The CU is hierarchically configured from the largest LCU (Largest Coding Unit) to the smallest SCU (Smallest Coding Unit). That is, it can be considered that the LCU generally corresponds to an AVC macroblock, and a CU in a layer lower than the LCU (a CU smaller than the LCU) corresponds to an AVC sub-macroblock.

  By the way, it has a coding mode for encoding and outputting image data, and a non-coding mode for outputting image data without encoding, and a macroblock for selecting the coding mode or the non-coding mode. There is an encoding method that can be selected in units and in which a coding mode and a non-coding mode can be mixed in one picture (see, for example, Patent Document 1). Also in the AVC encoding method, an I_PCM mode that outputs image data without encoding is supported as mb_type (see, for example, Patent Document 2). This is used to guarantee the real-time operation of the arithmetic encoding process when the quantization parameter is set to a small value such as QP = 0 and the amount of encoded data exceeds the input image. In addition, lossless encoding can be realized by using I-PCM.

  Further, a method for increasing the internal calculation has been proposed (see, for example, Non-Patent Document 2). As a result, it is possible to reduce the internal calculation error in processing such as orthogonal transform and motion compensation, and improve the coding efficiency.

  Furthermore, a method having an FIR filter in a motion compensation loop has been proposed (see, for example, Non-Patent Document 3). In the encoding device, the FIR filter coefficient is obtained by the Wiener Filter so as to minimize the error from the input image, thereby minimizing deterioration in the reference image and encoding efficiency of the image compression information to be output. It is possible to improve.

Patent No. 3992303 Japanese Patent No. 4240283

"Test Model under Consideration", JCTVC-B205, Joint Collaborative Team on Video Coding (JCT-VC) of ITU-T SG16 WP3 and ISO / IEC JTC1 / SC29 / WG112nd Meeting: Geneva, CH, 21-28 July, 2010 Takeshi Chujoh, Reiko Noda, "Internal bit depth increase except frame memory", VCEG-AF07, ITU-Telecommunications Standardization SectorSTUDY GROUP 16 Question 6Video Coding Experts Group (VCEG) 32nd Meeting: San Jose, USA, 20-21 April, 2007 Takeshi Chujoh, Goki Yasuda, Naofumi Wada, Takashi Watanabe, Tomoo Yamakage, "Block-based Adaptive Loop Filter", VCEG-AI18, ITU-Telecommunications Standardization SectorSTUDY GROUP 16 Question 6Video Coding Experts Group (VCEG) 35th Meeting: Berlin, Germany, 16-18 July, 2008

  However, in the case of an encoding method that defines a CU as in HEVC and performs various processing in units of that CU, the macroblock in AVC is considered to be equivalent to LCU, but I_PCM can only be set in LCU units. Since the maximum processing unit is 128 × 128 pixel units, unnecessary encoding processing may increase, and the efficiency of the encoding processing may be reduced. For example, it may be difficult to guarantee real-time operation of CABAC.

  In addition, the encoding schemes proposed in Non-Patent Document 2 and Non-Patent Document 3 are not included in the AVC encoding scheme, and the correspondence to the I_PCM mode has not been disclosed.

  The present invention has been made in view of such a situation, and an object of the present invention is to improve encoding efficiency while suppressing reduction in efficiency of encoding processing.

One aspect of the present technology is an uncompressed mode that is an encoding mode in which image data is not encoded but output as unencoded data for an encoding unit obtained by recursively dividing the maximum encoding unit. A coding unit set as a compression unit, a coding unit set as a compression mode, which is a coding mode for encoding the image data and outputting it as coded data, and a filter process performed after the deblocking filter process. or a filter identification information identifying the dividing, the encoded data including the identification information identifying whether uncoded data is included in the coding unit to generate image data by decoding for each of the coding units a decoding unit, wherein using filter identification information and said identification information, and controls the filter process on the image data generated by the decoding unit braking An image processing apparatus and a part.

Said decoding unit is configured to acquire from the encoded data and the filter identification information and the identification information, the control unit uses the decryption unit the filter identification information acquired from the said identification information, said decoding The filter processing for the image data generated by the unit can be controlled.

The encoded data includes, as the filter identification information, loop filter identification information for identifying whether the deblock filter processing and the filter processing are performed as loop filter processing, and the control unit includes the filter identification information and the filter identification information. by using the identification information, it is possible to control the said filtering and the deblocking filter processing of the image data generated by the decoding unit.

  The decoding unit acquires the filter identification information from the encoded data, and the control unit uses the filter identification information acquired from the decoding unit to perform the decoding on the image data generated by the decoding unit. Block filter processing and the filter processing can be controlled.

  A deblocking filter unit that performs the deblocking filter process on the image data generated by the decoding unit, and the filtering process on the image data that has been subjected to the deblocking filter process by the deblocking filter unit And a filter unit to be performed.

  The filter process may be an adaptive loop filter process using a Wiener filter.

  The coding unit may be recursively divided into the maximum coding unit according to a quadtree structure.

  The maximum coding unit may be a coding unit of the highest layer in the quadtree structure.

  The maximum coding unit may be a fixed size block in sequence units, and the coding unit may be a variable size block.

One aspect of the present technology is a non-encoding mode that outputs image data as non-encoded data without encoding image data for an encoding unit obtained by recursively dividing the maximum encoding unit. A coding unit set as a compression mode, a coding unit set as a compression mode that is a coding mode for coding the image data and outputting it as coded data, and filter processing performed after deblocking filter processing a filter identification information identifying whether is performed, the encoded data including the identification information identifying whether uncoded data is included in the coding unit, the image data by decoding for each of the coding units An image processing method for generating and controlling the filter processing on the generated image data using the filter identification information and the identification information .

Get the said filter identification information from the encoded data and the identification information, can be used with the acquired filter identification information and said identification information, and controls the filtering performed on the generated the image data .

The encoded data includes, as the filter identification information, loop filter identification information for identifying whether the deblock filter processing and the filter processing are performed as loop filter processing , and uses the filter identification information and the identification information. Thus, it is possible to control the deblocking filtering process and the filtering process for the generated image data.

  The filter identification information can be acquired from the encoded data, and the deblocking filter processing and the filter processing for the generated image data can be controlled using the acquired filter identification information.

  The deblocking filter process may be performed on the generated image data, and the filter process may be performed on the image data on which the deblocking filter process has been performed.

  The filter process may be an adaptive loop filter process using a Wiener filter.

  The coding unit may be recursively divided into the maximum coding unit according to a quadtree structure.

  The maximum coding unit may be a coding unit of the highest layer in the quadtree structure.

  The maximum coding unit may be a fixed size block in sequence units, and the coding unit may be a variable size block.

One aspect of the present technology further provides an encoding mode in which a computer outputs an unencoded data without encoding image data for an encoding unit obtained by recursively dividing a maximum encoding unit. A coding unit set as a non-compression mode, a coding unit set as a compression mode, which is a coding mode for encoding the image data and outputting it as coded data, and a block after deblocking filter processing. Coded data including filter identification information for identifying whether or not filtering processing is performed and identification information for identifying whether uncoded data is included in a coding unit is decoded for each coding unit to generate an image a decoding unit for generating data, by using said identification information and the filter identification information, said with respect to the image data generated by the decoding unit Fi A program to function as a control unit for controlling the data processing.

One aspect of the present technology further provides an encoding mode in which a computer outputs an unencoded data without encoding image data for an encoding unit obtained by recursively dividing a maximum encoding unit. A coding unit set as a non-compression mode, a coding unit set as a compression mode, which is a coding mode for encoding the image data and outputting it as coded data, and a block after deblocking filter processing. Coded data including filter identification information for identifying whether or not filtering processing is performed and identification information for identifying whether uncoded data is included in a coding unit is decoded for each coding unit to generate an image a decoding unit for generating data, by using said identification information and the filter identification information, said with respect to the image data generated by the decoding unit Fi Computer which records a program for functioning as a control unit for controlling the data processing is a recording medium readable.

In one aspect of the present technology, non-compression is an encoding mode in which image data is output as non-encoded data without encoding the encoded data obtained by recursively dividing the maximum encoding unit. A coding unit set as a mode, a coding unit set as a compression mode, which is a coding mode for coding the image data and outputting it as coded data, and a filter process performed after the deblocking filter process. a filter identification information identifying whether performed, coded data uncoded data includes identification information for identifying whether included in the coding unit, is the image data is generated and decoded for each coding unit, Filter identification information and identification information are used to control filter processing for the generated image data.

  According to the present invention, an image can be processed. In particular, the encoding efficiency can be improved while suppressing a reduction in the efficiency of the encoding process.

It is a block diagram which shows the image coding apparatus which outputs the image compression information based on an AVC encoding system. It is a block diagram which shows the image decoding apparatus which inputs the image compression information based on an AVC encoding system. It is a figure which shows the example of the kind of macroblock. It is a figure explaining the structural example of a coding unit. It is a figure explaining the method to increase bit amount in internal calculation. It is a figure explaining an adaptive loop filter. It is a block diagram which shows the main structural examples of the image coding apparatus of this Embodiment. FIG. 8 is a block diagram illustrating a main configuration example of a lossless encoding unit, a loop filter, and a PCM encoding unit of FIG. 7. It is a block diagram which shows the main structural examples of the PCM determination part of FIG. It is a flowchart explaining the example of the flow of an encoding process. It is a flowchart explaining the example of the flow of a PCM encoding control process. It is a flowchart explaining the example of the flow of a PCM encoding process. It is a flowchart explaining the example of the flow of a reference image generation process. It is a flowchart explaining the example of the flow of a loop filter process. It is a block diagram which shows the main structural examples of the image decoding apparatus of this Embodiment. FIG. 16 is a block diagram illustrating a main configuration example of a lossless decoding unit, a loop filter, and a PCM decoding unit in FIG. 15. It is a flowchart explaining the example of the flow of a decoding process. It is a flowchart following FIG. 17 explaining the example of the flow of a decoding process. It is a flowchart explaining the example of the flow of a loop filter process. It is a figure explaining the example of I_PCM information. It is a block diagram which shows the main structural examples of the personal computer of this Embodiment. It is a block diagram which shows the main structural examples of the television receiver of this Embodiment. It is a block diagram which shows the main structural examples of the mobile telephone of this Embodiment. It is a block diagram which shows the main structural examples of the hard disk recorder of this Embodiment. It is a block diagram which shows the main structural examples of the camera to which this invention is applied.

Hereinafter, modes for carrying out the invention (hereinafter referred to as embodiments) will be described. The description will be given in the following order.
1. First Embodiment (Image Encoding Device)
2. Second embodiment (image decoding apparatus)
3. Third embodiment (personal computer)
4). Fourth embodiment (television receiver)
5. Fifth embodiment (mobile phone)
6). Sixth embodiment (hard disk recorder)
7). Seventh embodiment (camera)

<1. First Embodiment>
[AVC coding image coding device]
FIG. 1 illustrates a configuration of an embodiment of an image encoding apparatus that encodes an image using an H.264 and MPEG (Moving Picture Experts Group) 4 Part 10 (AVC (Advanced Video Coding)) encoding method.

  An image encoding apparatus 100 shown in FIG. 1 is an apparatus that encodes and outputs an image using an encoding method based on the AVC standard. As illustrated in FIG. 1, the image encoding device 100 includes an A / D conversion unit 101, a screen rearrangement buffer 102, a calculation unit 103, an orthogonal transformation unit 104, a quantization unit 105, a lossless encoding unit 106, and an accumulation. A buffer 107 is provided. In addition, the image encoding device 100 includes an inverse quantization unit 108, an inverse orthogonal transform unit 109, a calculation unit 110, a deblock filter 111, a frame memory 112, a selection unit 113, an intra prediction unit 114, a motion prediction / compensation unit 115, A selection unit 116 and a rate control unit 117 are included.

  The A / D conversion unit 101 performs A / D conversion on the input image data, and outputs to the screen rearrangement buffer 102 for storage. The screen rearrangement buffer 102 rearranges the stored frame images in the display order in the order of frames for encoding in accordance with the GOP (Group of Picture) structure. The screen rearrangement buffer 102 supplies the image with the rearranged frame order to the arithmetic unit 103. The screen rearrangement buffer 102 also supplies the image in which the order of the frames is rearranged to the intra prediction unit 114 and the motion prediction / compensation unit 115.

  The calculation unit 103 subtracts the prediction image supplied from the intra prediction unit 114 or the motion prediction / compensation unit 115 via the selection unit 116 from the image read from the screen rearrangement buffer 102, and orthogonalizes the difference information. The data is output to the conversion unit 104.

  For example, in the case of an image on which intra coding is performed, the calculation unit 103 subtracts the prediction image supplied from the intra prediction unit 114 from the image read from the screen rearrangement buffer 102. For example, in the case of an image on which inter coding is performed, the arithmetic unit 103 subtracts the predicted image supplied from the motion prediction / compensation unit 115 from the image read from the screen rearrangement buffer 102.

  The orthogonal transform unit 104 performs orthogonal transform such as discrete cosine transform and Karhunen-Loeve transform on the difference information supplied from the calculation unit 103 and supplies the transform coefficient to the quantization unit 105.

  The quantization unit 105 quantizes the transform coefficient output from the orthogonal transform unit 104. The quantization unit 105 sets a quantization parameter based on information on the target value of the code amount supplied from the rate control unit 117, and performs quantization. The quantization unit 105 supplies the quantized transform coefficient to the lossless encoding unit 106.

  The lossless encoding unit 106 performs lossless encoding such as variable length encoding and arithmetic encoding on the quantized transform coefficient. Since the coefficient data is quantized under the control of the rate control unit 117, the code amount becomes a target value set by the rate control unit 117 (or approximates the target value).

  The lossless encoding unit 106 acquires information indicating intra prediction from the intra prediction unit 114, and acquires information indicating inter prediction mode, motion vector information, and the like from the motion prediction / compensation unit 115. Note that information indicating intra prediction (intra-screen prediction) is hereinafter also referred to as intra prediction mode information. In addition, information indicating an information mode indicating inter prediction (inter-screen prediction) is hereinafter also referred to as inter prediction mode information.

  The lossless encoding unit 106 encodes the quantized transform coefficient, and also converts various information such as filter coefficient, intra prediction mode information, inter prediction mode information, and quantization parameter into one piece of header information of the encoded data. Part (multiplex). The lossless encoding unit 106 supplies the encoded data obtained by encoding to the accumulation buffer 107 for accumulation.

  For example, the lossless encoding unit 106 performs lossless encoding processing such as variable length encoding or arithmetic encoding. Examples of variable length coding include H.264. CAVLC (Context-Adaptive Variable Length Coding) defined in H.264 / AVC format. Examples of arithmetic coding include CABAC (Context-Adaptive Binary Arithmetic Coding).

  The accumulation buffer 107 temporarily holds the encoded data supplied from the lossless encoding unit 106, and at a predetermined timing, the H.264 buffer stores the encoded data. As an encoded image encoded by the H.264 / AVC format, for example, it is output to a recording device or a transmission path (not shown) in the subsequent stage.

  The transform coefficient quantized by the quantization unit 105 is also supplied to the inverse quantization unit 108. The inverse quantization unit 108 inversely quantizes the quantized transform coefficient by a method corresponding to the quantization by the quantization unit 105. The inverse quantization unit 108 supplies the obtained transform coefficient to the inverse orthogonal transform unit 109.

  The inverse orthogonal transform unit 109 performs inverse orthogonal transform on the supplied transform coefficient by a method corresponding to the orthogonal transform process by the orthogonal transform unit 104. The inversely orthogonal transformed output (restored difference information) is supplied to the calculation unit 110.

  The calculation unit 110 uses the inverse prediction unit 114 or the motion prediction / compensation unit 115 via the selection unit 116 for the inverse orthogonal transformation result supplied from the inverse orthogonal transformation unit 109, that is, the restored difference information. The images are added to obtain a locally decoded image (decoded image).

  For example, when the difference information corresponds to an image on which intra coding is performed, the calculation unit 110 adds the predicted image supplied from the intra prediction unit 114 to the difference information. For example, when the difference information corresponds to an image on which inter coding is performed, the calculation unit 110 adds the predicted image supplied from the motion prediction / compensation unit 115 to the difference information.

  The addition result is supplied to the deblock filter 111 or the frame memory 112.

  The deblocking filter 111 removes block distortion of the decoded image by appropriately performing deblocking filter processing. The deblocking filter 111 supplies the filter processing result to the frame memory 112. Note that the decoded image output from the calculation unit 110 can be supplied to the frame memory 112 without passing through the deblocking filter 111. That is, the deblocking filter process of the deblocking filter 111 can be omitted.

  The frame memory 112 stores the supplied decoded image, and outputs the stored decoded image as a reference image to the intra prediction unit 114 or the motion prediction / compensation unit 115 via the selection unit 113 at a predetermined timing. .

  For example, in the case of an image on which intra coding is performed, the frame memory 112 supplies the reference image to the intra prediction unit 114 via the selection unit 113. For example, when inter coding is performed, the frame memory 112 supplies the reference image to the motion prediction / compensation unit 115 via the selection unit 113.

  When the reference image supplied from the frame memory 112 is an image on which intra coding is performed, the selection unit 113 supplies the reference image to the intra prediction unit 114. Further, when the reference image supplied from the frame memory 112 is an image to be subjected to inter coding, the selection unit 113 supplies the reference image to the motion prediction / compensation unit 115.

  The intra prediction unit 114 performs intra prediction (intra-screen prediction) that generates a predicted image using the pixel values in the processing target picture supplied from the frame memory 112 via the selection unit 113. The intra prediction unit 114 performs this intra prediction in a plurality of modes (intra prediction modes) prepared in advance.

  In the H.264 image information encoding method, for the luminance signal, an intra 4 × 4 prediction mode, an intra 8 × 8 prediction mode, and an intra 16 × 16 prediction mode are defined, and for a color difference signal, For each macroblock, it is possible to define a prediction mode independent of the luminance signal. For intra 4x4 prediction mode, one intra prediction mode must be defined for each 4x4 luminance block, and for intra 8x8 prediction mode, for each 8x8 luminance block become. For the intra 16 × 16 prediction mode and the color difference signal, one prediction mode is defined for each macroblock.

  The intra prediction unit 114 generates predicted images in all candidate intra prediction modes, evaluates the cost function value of each predicted image using the input image supplied from the screen rearrangement buffer 102, and selects the optimum mode. select. When the optimal intra prediction mode is selected, the intra prediction unit 114 supplies the prediction image generated in the optimal mode to the calculation unit 103 and the calculation unit 110 via the selection unit 116.

  Further, as described above, the intra prediction unit 114 supplies information such as intra prediction mode information indicating the adopted intra prediction mode to the lossless encoding unit 106 as appropriate.

  The motion prediction / compensation unit 115 uses the input image supplied from the screen rearrangement buffer 102 and the reference image supplied from the frame memory 112 via the selection unit 113 for the image to be inter-coded, Motion prediction (inter prediction) is performed, motion compensation processing is performed according to the detected motion vector, and a predicted image (inter predicted image information) is generated. The motion prediction / compensation unit 115 performs such inter prediction in a plurality of modes (inter prediction modes) prepared in advance.

  The motion prediction / compensation unit 115 generates a prediction image in all candidate inter prediction modes, evaluates the cost function value of each prediction image, and selects an optimal mode. The motion prediction / compensation unit 115 supplies the generated predicted image to the calculation unit 103 and the calculation unit 110 via the selection unit 116.

  In addition, the motion prediction / compensation unit 115 supplies the inter prediction mode information indicating the adopted inter prediction mode and the motion vector information indicating the calculated motion vector to the lossless encoding unit 106.

  The selection unit 116 supplies the output of the intra prediction unit 114 to the calculation unit 103 and the calculation unit 110 in the case of an image to be subjected to intra coding, and outputs the output of the motion prediction / compensation unit 115 in the case of an image to be subjected to inter coding. It supplies to the calculating part 103 and the calculating part 110.

  The rate control unit 117 controls the quantization operation rate of the quantization unit 105 based on the compressed image stored in the storage buffer 107 so that overflow or underflow does not occur.

[AVC encoding image decoding device]
FIG. 2 is a block diagram illustrating a main configuration example of an image decoding apparatus that realizes image compression by orthogonal transformation such as discrete cosine transformation or Karhunen-Labe transformation and motion compensation. An image decoding apparatus 200 shown in FIG. 2 is a decoding apparatus corresponding to the image encoding apparatus 100 of FIG.

  The encoded data encoded by the image encoding apparatus 100 is supplied to the image decoding apparatus 200 corresponding to the image encoding apparatus 100 via an arbitrary path such as a transmission path or a recording medium, and is decoded. .

  As shown in FIG. 2, the image decoding apparatus 200 includes a storage buffer 201, a lossless decoding unit 202, an inverse quantization unit 203, an inverse orthogonal transform unit 204, a calculation unit 205, a deblock filter 206, a screen rearrangement buffer 207, And a D / A converter 208. The image decoding apparatus 200 includes a frame memory 209, a selection unit 210, an intra prediction unit 211, a motion prediction / compensation unit 212, and a selection unit 213.

  The accumulation buffer 201 accumulates the transmitted encoded data. This encoded data is encoded by the image encoding device 100. The lossless decoding unit 202 decodes the encoded data read from the accumulation buffer 201 at a predetermined timing by a method corresponding to the encoding method of the lossless encoding unit 106 in FIG.

  When the frame is intra-coded, intra prediction mode information is stored in the header portion of the encoded data. The lossless decoding unit 202 also decodes the intra prediction mode information and supplies the information to the intra prediction unit 211. On the other hand, when the frame is inter-encoded, motion vector information is stored in the header portion of the encoded data. The lossless decoding unit 202 also decodes the motion vector information and supplies the information to the motion prediction / compensation unit 212.

  The inverse quantization unit 203 inversely quantizes the coefficient data (quantization coefficient) obtained by decoding by the lossless decoding unit 202 by a method corresponding to the quantization method of the quantization unit 105 in FIG. That is, the inverse quantization unit 203 performs inverse quantization of the quantization coefficient by the same method as the inverse quantization unit 108 in FIG.

  The inverse quantization unit 203 supplies the inversely quantized coefficient data, that is, the orthogonal transform coefficient, to the inverse orthogonal transform unit 204. The inverse orthogonal transform unit 204 is a method corresponding to the orthogonal transform method of the orthogonal transform unit 104 in FIG. 1 (the same method as the inverse orthogonal transform unit 109 in FIG. 1), and inverse orthogonal transforms the orthogonal transform coefficient to generate an image code The decoding apparatus 100 obtains decoded residual data corresponding to the residual data before being orthogonally transformed in the encoding apparatus 100. For example, fourth-order inverse orthogonal transform is performed.

  Decoded residual data obtained by the inverse orthogonal transform is supplied to the calculation unit 205. In addition, a prediction image is supplied to the calculation unit 205 from the intra prediction unit 211 or the motion prediction / compensation unit 212 via the selection unit 213.

  The computing unit 205 adds the decoded residual data and the predicted image, and obtains decoded image data corresponding to the image data before the predicted image is subtracted by the computing unit 103 of the image encoding device 100. The arithmetic unit 205 supplies the decoded image data to the deblock filter 206.

  The deblocking filter 206 removes block distortion of the supplied decoded image, and then supplies it to the screen rearranging buffer 207.

  The screen rearrangement buffer 207 rearranges images. That is, the order of frames rearranged for the encoding order by the screen rearrangement buffer 102 in FIG. 1 is rearranged in the original display order. The D / A conversion unit 208 D / A converts the image supplied from the screen rearrangement buffer 207, outputs it to a display (not shown), and displays it.

  The output of the deblocking filter 206 is further supplied to the frame memory 209.

  The frame memory 209, the selection unit 210, the intra prediction unit 211, the motion prediction / compensation unit 212, and the selection unit 213 are the frame memory 112, the selection unit 113, the intra prediction unit 114, and the motion prediction / compensation unit of the image encoding device 100. 115 and the selection unit 116 respectively.

  The selection unit 210 reads out the inter-processed image and the referenced image from the frame memory 209, and supplies them to the motion prediction / compensation unit 212. Further, the selection unit 210 reads an image used for intra prediction from the frame memory 209 and supplies the image to the intra prediction unit 211.

  Information indicating the intra prediction mode obtained by decoding the header information is appropriately supplied from the lossless decoding unit 202 to the intra prediction unit 211. Based on this information, the intra prediction unit 211 generates a prediction image from the reference image acquired from the frame memory 209 and supplies the generated prediction image to the selection unit 213.

  The motion prediction / compensation unit 212 acquires information (prediction mode information, motion vector information, reference frame information, flags, various parameters, and the like) obtained by decoding the header information from the lossless decoding unit 202.

  The motion prediction / compensation unit 212 generates a prediction image from the reference image acquired from the frame memory 209 based on the information supplied from the lossless decoding unit 202, and supplies the generated prediction image to the selection unit 213.

  The selection unit 213 selects the prediction image generated by the motion prediction / compensation unit 212 or the intra prediction unit 211 and supplies the selected prediction image to the calculation unit 205.

[Macro block type]
By the way, as shown in Patent Document 1, there is an encoding mode for encoding and outputting image data and a non-encoding mode for outputting image data without encoding, and whether the encoding mode is set or not. There is an encoding method in which whether to select an encoding mode is selected for each macroblock, and an encoding mode and a non-encoding mode can be mixed in one picture. As shown in Patent Document 2, also in the AVC encoding method, as shown in FIG. 3, as one of the macroblock types (mb_type), I_PCM (Intra-block) that outputs image data without encoding pulse code modulation) mode is supported.

  This is used to guarantee the real-time operation of the arithmetic encoding process when the quantization parameter is set to a small value such as QP = 0 and the amount of encoded data exceeds the input image. In addition, lossless encoding can be realized by using the I-PCM mode.

[Cost function]
By the way, in the AVC encoding method, in order to achieve higher encoding efficiency, selection of an appropriate prediction mode is important.

  An example of such a selection method is the reference software for H.264 / MPEG-4 AVC called JM (Joint Model), which is published at http://iphome.hhi.de/suehring/tml/index.htm. The methods that are implemented can be mentioned.

  In JM, it is possible to select the following two mode determination methods: High Complexity Mode and Low Complexity Mode. In both cases, a cost function value for each prediction mode Mode is calculated, and a prediction mode that minimizes the cost function value is selected as the optimum mode for the block or macroblock.

  The cost function in High Complexity Mode is as shown in the following formula (1).

Cost (Mode∈Ω) = D + λ * R
... (1)

  Here, Ω is the entire set of candidate modes for encoding the block or macroblock, and D is the difference energy between the decoded image and the input image when encoded in the prediction mode Mode. λ is a Lagrange undetermined multiplier given as a function of the quantization parameter. R is a total code amount when encoding is performed in the mode Mode, including orthogonal transform coefficients.

  In other words, in order to perform encoding in the High Complexity Mode, the parameters D and R are calculated, so it is necessary to perform temporary encoding processing once in all candidate modes (Modes), which requires a higher calculation amount. .

  The cost function in Low Complexity Mode is as shown in the following formula (2).

Cost (Mode ∈ Ω) = D + QP2Quant (QP) * HeaderBit
... (2)

  Here, unlike the case of High Complexity Mode, D is the difference energy between the predicted image and the input image. QP2Quant (QP) is given as a function of the quantization parameter QP, and HeaderBit is a code amount related to information belonging to Header, such as a motion vector and mode, which does not include an orthogonal transform coefficient.

  That is, in Low Complexity Mode, it is necessary to perform prediction processing for each candidate mode (Mode), but it is not necessary to perform decoding processing because it is not necessary to perform decoding processing. For this reason, realization with a calculation amount lower than High Complexity Mode is possible.

[Coding unit]
Next, a coding unit defined in the HEVC encoding method described in Non-Patent Document 1 will be described.

  Coding Unit (CU) is also called Coding Tree Block (CTB), and is a partial area of a picture unit image that plays the same role as a macroblock in AVC. The latter is fixed to a size of 16 × 16 pixels, whereas the size of the former is not fixed, and is specified in the image compression information in each sequence.

  In particular, a CU having the largest size is called an LCU (Largest Coding Unit), and a CU having the smallest size is called an SCU (Smallest Coding Unit). For example, in the sequence parameter set (SPS) included in the image compression information, the sizes of these areas are specified, but each is limited to a square and a size represented by a power of 2.

  FIG. 4 shows an example of a coding unit defined by HEVC. In the example of FIG. 4, the LCU size is 128 and the maximum hierarchical depth is 5. When the value of split_flag is “1”, the 2N × 2N size CU is divided into N × N size CUs that are one level below.

  Furthermore, the CU is divided into prediction units (Prediction Unit (PU)) that are regions (partial regions of images in units of pictures) that are processing units of intra or inter prediction, and are regions that are processing units of orthogonal transformation It is divided into transform units (Transform Units (TU)), which are (partial regions of images in picture units). Currently, in HEVC, it is possible to use 16 × 16 and 32 × 32 orthogonal transforms in addition to 4 × 4 and 8 × 8.

[IBDI]
By the way, Non-Patent Document 2 proposes a method (IBDI (Internal bit depth increase except frame memory)) for increasing the internal calculation as shown in FIG. In the case of this method, as shown in FIG. 5, in the quantization process, the lossless encoding process, the inverse quantization process, the filter process, the prediction process, the lossless decoding process, and the like of the encoding apparatus and the decoding apparatus, For example, the bit depth is expanded from 8 bits to 12 bits. As a result, it is possible to reduce the internal calculation error in processing such as orthogonal transform and motion compensation, and improve the coding efficiency.

[BALF]
By the way, in Non-Patent Document 3, as shown in FIG. 5, a method of having a FIR filter in a motion compensation loop and adaptively performing loop filter processing (BALF (Block-based Adaptive Loop Filter)) using the filter. Has been proposed. In the encoding device, the FIR filter coefficient is obtained by the Wiener Filter so as to minimize the error from the input image, thereby minimizing deterioration in the reference image and encoding efficiency of the image compression information to be output. It is possible to improve.

[Efficiency of encoding process]
By the way, in the case of an encoding method in which a CU is defined as in HEVC and various processes are performed in units of the CU, a macroblock in AVC can be considered to correspond to an LCU. However, since the CU has a hierarchical structure as shown in FIG. 4, the size of the LCU in the highest hierarchy is generally set larger than the AVC macroblock, for example, 128 × 128 pixels. is there.

  Therefore, even in such an encoding method, as in the case of AVC, when the I_PCM mode is set in LCU units, the processing unit is larger than in AVC, for example, 128 × 128 pixel units. turn into.

  Intra prediction and inter prediction modes are determined by calculating and comparing cost function values as described above. That is, prediction and encoding are performed in all modes, the respective cost function values are calculated, the optimal mode is selected, and encoded data is generated in the optimal mode.

  However, when the I_PCM mode is adopted, the encoded data generated in the optimal mode is discarded, and the input image (unencoded data) is directly adopted as the encoding result. Therefore, when the I_PCM mode is selected, all processes for generating encoded data of the optimal mode are unnecessary processes. That is, unnecessary processing further increases as the unit of selection control in this I_PCM mode increases. In other words, as described above, if it is selected whether to adopt the I_PCM mode for each LCU, the efficiency of the encoding process may be further reduced. Therefore, for example, it may be difficult to guarantee the real-time operation of CABAC.

  Further, the above-described techniques such as IBDI and BALF are not included in the AVC encoding system, and it is unclear how these processes are controlled when the I_PCM mode is adopted.

  Therefore, in this embodiment, the selection of the I_PCM mode can be controlled in more detail, and the encoding efficiency can be improved while suppressing the reduction of the encoding process efficiency. In addition, according to the selection of the I_PCM mode, whether or not IBDI or BALF is appropriately executed is controlled, so that reduction in the efficiency of the encoding process can be further suppressed.

[Image encoding device]
FIG. 7 is a block diagram illustrating a main configuration example of an image encoding device to which the present invention has been applied.

  An image encoding device 300 shown in FIG. 7 is basically the same device as the image encoding device 100 of FIG. 1, and encodes image data. As illustrated in FIG. 7, the image encoding device 300 includes an A / D conversion unit 301, a screen rearrangement buffer 302, an adaptive left shift unit 303, a calculation unit 304, an orthogonal transformation unit 305, a quantization unit 306, a lossless encoding. A unit 307 and a storage buffer 308. In addition, the image encoding device 300 includes an inverse quantization unit 309, an inverse orthogonal transform unit 310, a calculation unit 311, a loop filter 312, an adaptive right shift unit 313, a frame memory 314, an adaptive left shift unit 315, a selection unit 316, an intra A prediction unit 317, a motion prediction / compensation unit 318, a selection unit 319, and a rate control unit 320 are included.

  The image encoding device 300 further includes a PCM encoding unit 321.

  As in the case of the A / D conversion unit 101, the A / D conversion unit 301 performs A / D conversion on the input image data. The A / D conversion unit 301 supplies the converted image data (digital data) to the screen rearrangement buffer 302 for storage. As with the screen rearrangement buffer 102, the screen rearrangement buffer 302 arranges the images of the frames in the stored display order in the order of the frames for encoding according to the GOP (Group of Picture) structure. Change. The screen rearrangement buffer 302 supplies the image with the rearranged frame order to the adaptive left shift unit 303.

  The screen rearrangement buffer 302 also supplies the image in which the frame order is rearranged to the lossless encoding unit 307 and the PCM encoding unit 321.

  The adaptive left shift unit 303 is controlled by the PCM encoding unit 321 to shift the image data read from the screen rearrangement buffer 302 in the left direction and increase the bit depth by a predetermined number of bits (for example, 4 bits). For example, the adaptive left shift unit 303 increases the bit depth of the image data read from the screen rearrangement buffer 302 from 8 bits to 12 bits. By increasing the bit depth in this way, it is possible to improve the accuracy of internal computations in each process such as orthogonal transform processing, quantization processing, lossless encoding processing, and prediction processing, and to suppress errors.

  The left shift amount (bit amount) is arbitrary, and may be a fixed value or variable. The left shift process may be omitted under the control of the PCM encoding unit 321.

  The adaptive left shift unit 303 supplies the image data after the left shift process (if the process is omitted, the image data output from the screen rearrangement buffer 302 as it is) to the calculation unit 304. The adaptive left shift unit 303 also supplies the image data to the intra prediction unit 317 and the motion prediction / compensation unit 318.

  As in the case of the calculation unit 103, the calculation unit 304 calculates a prediction image supplied from the intra prediction unit 317 or the motion prediction / compensation unit 318 via the selection unit 319 from the image supplied from the adaptive left shift unit 303. Subtract. The arithmetic unit 304 outputs the difference information to the orthogonal transform unit 305.

  For example, in the case of an image on which intra coding is performed, the calculation unit 304 subtracts the prediction image supplied from the intra prediction unit 317 from the image supplied from the adaptive left shift unit 303. For example, in the case of an image on which inter coding is performed, the arithmetic unit 304 subtracts the predicted image supplied from the motion prediction / compensation unit 318 from the image supplied from the adaptive left shift unit 303.

  Similar to the orthogonal transform unit 104, the orthogonal transform unit 305 performs orthogonal transform such as discrete cosine transform and Karhunen-Loeve transform on the difference information supplied from the calculation unit 304. Note that this orthogonal transformation method is arbitrary. The orthogonal transform unit 305 supplies the transform coefficient to the quantization unit 306.

  The quantization unit 306 quantizes the transform coefficient supplied from the orthogonal transform unit 305 as in the case of the quantization unit 105. The quantization unit 306 sets a quantization parameter based on the information regarding the target value of the code amount supplied from the rate control unit 320, and performs the quantization. Note that this quantization method is arbitrary. The quantization unit 306 supplies the quantized transform coefficient to the lossless encoding unit 307.

  As with the lossless encoding unit 106, the lossless encoding unit 307 performs lossless encoding such as variable length encoding and arithmetic encoding on the transform coefficient quantized by the quantization unit 306. Since the coefficient data is quantized under the control of the rate control unit 320, the code amount becomes the target value set by the rate control unit 320 (or approximates the target value).

  Note that, when the I_PCM mode is selected by the PCM encoding unit 321, the lossless encoding unit 307 uses the input image (unencoded data) supplied from the screen rearrangement buffer 302 as an encoding result (that is, actually Encoding is omitted).

  Further, the lossless encoding unit 307 acquires information indicating an intra prediction mode from the intra prediction unit 317 and acquires information indicating an inter prediction mode, motion vector information, and the like from the motion prediction / compensation unit 318. Further, the lossless encoding unit 307 acquires the filter coefficient used in the loop filter 312.

  The lossless encoding unit 307 encodes these filter coefficients, information indicating the mode of intra prediction or inter prediction, and various types of information such as quantization parameters in the same manner as in the case of the lossless encoding unit 106, and A part of the header information (multiplexed). The lossless encoding unit 307 supplies the encoded data (including non-encoded data in the case of the I_PCM mode) obtained by encoding to the accumulation buffer 308 for accumulation.

  For example, in the lossless encoding unit 307, as in the case of the lossless encoding unit 106, lossless encoding processing such as variable length encoding or arithmetic encoding is performed. Examples of variable length coding include H.264. CAVLC (Context-Adaptive Variable Length Coding) defined in H.264 / AVC format. Examples of arithmetic coding include CABAC (Context-Adaptive Binary Arithmetic Coding). Of course, the lossless encoding unit 307 may perform encoding by a method other than these methods.

  The accumulation buffer 308 temporarily holds the encoded data (including non-encoded data in the case of the I_PCM mode) supplied from the lossless encoding unit 307, as in the case of the accumulation buffer 107. The accumulation buffer 308 outputs the stored encoded data to, for example, a recording device (recording medium) (not shown) or a transmission path (not shown) at a predetermined timing at a predetermined timing.

  The transform coefficient quantized by the quantization unit 306 is also supplied to the inverse quantization unit 309. The inverse quantization unit 309 inversely quantizes the quantized transform coefficient by a method corresponding to the quantization performed by the quantization unit 306 as in the case of the inverse quantization unit 108. This inverse quantization method may be any method as long as it is a method corresponding to the quantization processing by the quantization unit 306. The inverse quantization unit 309 supplies the obtained transform coefficient to the inverse orthogonal transform unit 310.

  The inverse orthogonal transform unit 310 performs inverse orthogonal transform on the transform coefficient supplied from the inverse quantization unit 309 by a method corresponding to the orthogonal transform process by the orthogonal transform unit 305, as in the case of the inverse orthogonal transform unit 109. This inverse orthogonal transform method may be any method as long as it corresponds to the orthogonal transform processing by the orthogonal transform unit 305. The inverse orthogonal transformed output (restored difference information) is supplied to the calculation unit 311.

  Similar to the calculation unit 110, the calculation unit 311 converts the inverse orthogonal transform result supplied from the inverse orthogonal transform unit 310, that is, the restored difference information, into the intra prediction unit 317 or motion prediction via the selection unit 319. The prediction image supplied from the compensation unit 318 is added to obtain a locally decoded image (decoded image).

  For example, when the difference information corresponds to an image on which intra coding is performed, the calculation unit 311 adds the predicted image supplied from the intra prediction unit 317 to the difference information. For example, when the difference information corresponds to an image on which inter coding is performed, the calculation unit 311 adds the predicted image supplied from the motion prediction / compensation unit 318 to the difference information.

  The addition result (decoded image) is supplied to the loop filter 312 or the adaptive right shift unit 313.

  The loop filter 312 includes a deblock filter, an adaptive loop filter, and the like, and appropriately performs a filtering process on the decoded image supplied from the calculation unit 311. For example, the loop filter 312 performs a deblocking filter process similar to the deblocking filter 111 on the decoded image to remove block distortion of the decoded image. Further, for example, the loop filter 312 is controlled by the PCM encoding unit 321 and uses a Wiener Filter for the deblock filter processing result (decoded image from which block distortion has been removed). Image quality is improved by performing loop filter processing. Note that the adaptive loop filter process may be omitted under the control of the PCM encoding unit 321.

  Note that the loop filter 312 may perform arbitrary filter processing on the decoded image. Further, the loop filter 312 can supply the filter coefficient used for the filter processing to the lossless encoding unit 307 and encode it as necessary.

  The loop filter 312 supplies the filter processing result (the decoded image after the filter processing) to the adaptive right shift unit 313. As described above, the decoded image output from the calculation unit 311 can be supplied to the adaptive right shift unit 313 without going through the loop filter 312. That is, the filter process by the loop filter 312 can be omitted.

  The adaptive right shift unit 313 is controlled by the PCM encoding unit 321 to shift the image data supplied from the calculation unit 311 or the loop filter 312 to the right, and reduce the bit depth by a predetermined number of bits (for example, 4 bits). . That is, the adaptive right shift unit 313 right-shifts the number of bits left-shifted by the adaptive left shift unit 303, and the state before the bit depth of the image data is shifted to the left (read from the screen rearrangement buffer 302). Return to the current state.

  For example, the adaptive right shift unit 313 reduces the bit depth of the image data supplied from the calculation unit 311 or the loop filter 312 from 12 bits to 8 bits. By reducing the bit depth in this way, the amount of image data when stored in the frame memory can be reduced.

  This right shift amount (bit amount) is arbitrary as long as it matches the left shift amount in the adaptive left shift unit 303. That is, it may be a fixed value or variable. Also, the right shift process may be omitted under the control of the PCM encoding unit 321.

  The adaptive right shift unit 313 supplies the image data after the right shift process to the frame memory 314 (when the process is omitted, the image data output from the calculation unit 311 or the loop filter 312 is used as it is).

  As in the case of the frame memory 112, the frame memory 314 stores the supplied decoded image, and outputs the stored decoded image as a reference image to the adaptive left shift unit 315 at a predetermined timing.

  The adaptive left shift unit 315 is a processing unit similar to the adaptive left shift unit 303, and is controlled by the PCM encoding unit 321 to appropriately shift image data (reference image) read from the frame memory 314 leftward. The bit depth is increased by a predetermined number of bits (for example, 4 bits).

  For example, when not in the I_PCM mode, the input image data is shifted to the left by the adaptive left shift unit 303. Therefore, the adaptive left shift unit 315 shifts the reference image data read from the frame memory 314 to the left according to the control of the PCM encoding unit 321, and the bit depth is the same as the number of bits in the case of the adaptive left shift unit 303. Increase (eg, bit depth from 8 bits to 12 bits).

  Then, the adaptive left shift unit 315 supplies the image data after the left shift process to the selection unit 316. By increasing the bit depth in this way, the bit depth of the reference image can be matched with the bit depth of the input image, and the reference image can be added to the input image. In addition, it is possible to improve the accuracy of internal calculations such as prediction processing and suppress errors.

  On the other hand, for example, in the case of the I_PCM mode, the data of the input image is not shifted left by the adaptive left shift unit 303. Therefore, the adaptive left shift unit 315 supplies the reference image read from the frame memory 314 to the selection unit 316 without increasing the bit depth according to the control of the PCM encoding unit 321.

  As in the case of the selection unit 113, the selection unit 316 supplies the reference image supplied from the adaptive left shift unit 315 to the intra prediction unit 317 in the case of intra prediction. In the case of inter prediction, the selection unit 316 supplies the reference image supplied from the adaptive left shift unit 315 to the motion prediction / compensation unit 318 as in the case of the selection unit 113.

  The intra prediction unit 317 performs intra prediction (intra-screen prediction) that generates a prediction image using the reference image supplied from the adaptive left shift unit 315 via the selection unit 316. The intra prediction unit 317 performs this intra prediction in a plurality of modes (intra prediction modes) prepared in advance. The intra prediction unit 317 can also perform this intra prediction in any mode other than the mode defined in the AVC encoding method.

  The intra prediction unit 317 generates prediction images in all candidate intra prediction modes, evaluates the cost function value of each prediction image using the input image supplied from the adaptive left shift unit 303, and selects the optimal mode. select. When the intra prediction unit 317 selects an optimal intra prediction mode, the intra prediction unit 317 supplies a prediction image generated in the optimal mode to the calculation unit 304 or the calculation unit 311 via the selection unit 319.

  Also, as described above, the intra prediction unit 317 supplies information such as intra prediction mode information indicating the adopted intra prediction mode to the lossless encoding unit 307 as appropriate, and encodes the information.

  The motion prediction / compensation unit 318 uses an input image supplied from the adaptive left shift unit 303 and a reference image supplied from the adaptive left shift unit 315 via the selection unit 316 for the image to be inter-coded. Then, motion prediction (inter prediction) is performed, motion compensation processing is performed according to the detected motion vector, and a predicted image (inter predicted image information) is generated. The motion prediction / compensation unit 318 performs such inter prediction in a plurality of modes (inter prediction modes) prepared in advance. The motion prediction / compensation unit 318 can also perform this inter prediction in an arbitrary mode other than the mode defined in the AVC encoding method.

  The motion prediction / compensation unit 318 generates a prediction image in all candidate inter prediction modes, evaluates the cost function value of each prediction image, and selects an optimal mode. When the optimal inter prediction mode is selected, the motion prediction / compensation unit 318 supplies the prediction image generated in the optimal mode to the calculation unit 304 or the calculation unit 311 via the selection unit 319.

  Also, the motion prediction / compensation unit 318 supplies the inter prediction mode information indicating the adopted inter prediction mode and the motion vector information indicating the calculated motion vector to the lossless encoding unit 307 to be encoded.

  As in the case of the selection unit 116, the selection unit 319 supplies the output of the intra prediction unit 317 to the calculation unit 304 or the calculation unit 311 in the case of an image to be subjected to intra coding, and The output of the motion prediction / compensation unit 318 is supplied to the calculation unit 304 and the calculation unit 311.

  The rate control unit 320 controls the rate of the quantization operation of the quantization unit 306 based on the code amount of the encoded data stored in the storage buffer 308 so that overflow or underflow does not occur.

  Further, the rate control unit 320 supplies the code amount (generated code amount) of the encoded data stored in the storage buffer 308 to the PCM encoding unit 321.

  The PCM encoding unit 321 compares the code amount supplied from the rate control unit 320 with the data amount of the input image supplied from the screen rearrangement buffer 302, and selects whether to adopt the I_PCM mode. At that time, the PCM encoding unit 321 performs this selection in units of CUs smaller than the LCU. That is, the PCM encoding unit 321 controls in more detail whether or not it is the I_PCM mode.

  The PCM encoding unit 321 controls the operations of the lossless encoding unit 307, the adaptive left shift unit 303, the adaptive right shift unit 313, the adaptive left shift unit 315, and the loop filter 312 according to the selection result.

[Lossless encoding unit, PCM encoding unit, and loop filter]
FIG. 8 is a block diagram illustrating a main configuration example of the lossless encoding unit 307, the PCM encoding unit 321, and the loop filter 312 of FIG.

  As shown in FIG. 8, the lossless encoding unit 307 includes a NAL (Network Abstraction Layer) encoding unit 331 and a CU encoding unit 332.

  The NAL encoding unit 331 performs NAL such as a sequence parameter set (SPS (Sequence Parameter Set) or PPS (Picture Parameter Set)) based on a user instruction or a specification input via a user interface (not shown). The NAL encoding unit 331 supplies the encoded NAL (NAL data) to the accumulation buffer 308, and the encoded VCL (Video Coding Layer) supplied from the CU encoding unit 332 to the accumulation buffer 308. Is added to the CU data.

  The CU encoding unit 332 is controlled by the PCM encoding unit 321 (based on an On / Off control signal supplied from the PCM encoding unit 321), and encodes the VCL. For example, when the I_PCM mode is not selected by the PCM encoding unit 321 (when a control signal indicating “On” is supplied from the PCM encoding unit 321), the CU encoding unit 332 performs quantization for each CU. The orthogonal transform coefficient is encoded. The CU encoding unit 332 supplies the encoded data (CU data) of each encoded CU to the accumulation buffer 308.

  For example, when the I_PCM mode is selected by the PCM encoding unit 321 (when a control signal indicating “Off” is supplied from the PCM encoding unit 321), the CU encoding unit 332 displays the screen rearrangement buffer 302. The input pixel value supplied from is supplied to the accumulation buffer 308 as an encoding result (CU data).

  The CU encoding unit 332 also encodes a flag (I_PCM_flag) supplied from the PCM encoding unit 321 that indicates whether or not the encoding mode is the I_PCM mode, and supplies it to the accumulation buffer 308 as CU data. . Further, the CU encoding unit 332 encodes information related to filter processing such as an adaptive filter flag and a filter coefficient supplied from the loop filter 312 and supplies the encoded information to the accumulation buffer 308 as CU data.

  The encoding method by the CU encoding unit 332 is arbitrary (for example, CABAC, CAVLC, etc.). The NAL data and CU data supplied to the accumulation buffer 308 are synthesized and accumulated.

  In practice, the PCM encoding unit 321 controls whether or not to select the I_PCM mode using the code amount of encoded data obtained by encoding the orthogonal transform coefficient quantized by the CU encoding unit 332. To do.

  Therefore, for example, when the I_PCM mode is not selected, the encoded data already supplied to the accumulation buffer 308 is directly adopted as the encoded result of the quantized orthogonal transform coefficient of the CU. Therefore, the CU encoding unit 332 only needs to encode additional information such as I_PCM_flag.

  On the other hand, for example, when the I_PCM mode is selected, the CU encoding unit 332 stores the input pixel value of the CU supplied from the screen rearrangement buffer 302 in the accumulation buffer 308 as an encoding result (CU data). Supply. Therefore, in this case, the already supplied encoded data of the CU (encoded data obtained by encoding the quantized orthogonal transform coefficient) is discarded. That is, all the processes related to the generation of the encoded data are redundant processes.

  As illustrated in FIG. 8, the PCM encoding unit 321 includes an I_PCM_flag generation unit 341 and a PCM determination unit 342.

  The I_PCM_flag generation unit 341 generates I_PCM_flag according to the determination of the PCM determination unit 342 and determines its value. The I_PCM_flag generation unit 341 supplies the generated I_PCM_flag to the CU encoding unit 332 of the lossless encoding unit 307. For example, when the PCM determination unit 342 selects the I_PCM mode, the I_PCM_flag generation unit 341 sets the value of the I_PCM_flag to a value (for example, “1”) indicating that the I_PCM mode is selected, and sets the I_PCM_flag as a CU code To the conversion unit 332. Further, for example, when the PCM determination unit 342 does not select the I_PCM mode, the I_PCM_flag generation unit 341 sets the value of the I_PCM_flag to a value (for example, “0”) indicating that the I_PCM mode is not selected. Then, the I_PCM_flag is supplied to the CU encoding unit 332.

  The PCM determination unit 342 determines whether or not to set the encoding mode to the I_PCM mode. The PCM determination unit 342 obtains the data amount of the input pixel value supplied from the screen rearrangement buffer 302, compares it with the generated code amount supplied from the rate control unit 320, and selects the I_PCM mode based on the comparison result Decide whether or not to do. The PCM determination unit 342 controls the operation according to the selection result by supplying an On / Off control signal indicating the selection result to the CU encoding unit 332 and the I_PCM_flag generation unit 341.

  For example, when the data amount of the input pixel value is larger than the generated code amount, the PCM determination unit 342 does not select the I_PCM mode. In this case, the PCM determination unit 342 supplies a control signal indicating “On” to the CU encoding unit 332 and causes the CU encoding unit 332 to encode the quantized orthogonal transform coefficient. Further, the PCM determination unit 342 supplies a control signal indicating “On” to the I_PCM_flag generation unit 341, and generates I_PCM_flag having a value (for example, “0”) indicating that the I_PCM mode is not selected.

  On the other hand, for example, when the data amount of the input pixel value is equal to or less than the generated code amount, the PCM determination unit 342 selects the I_PCM mode. In this case, the PCM determination unit 342 supplies a control signal indicating “Off” to the CU encoding unit 332, and outputs an input pixel value as an encoding result (CU data). Further, the PCM determination unit 342 supplies a control signal indicating “Off” to the I_PCM_flag generation unit 341, and generates I_PCM_flag having a value (for example, “1”) indicating that the I_PCM mode is selected.

  The PCM determination unit 342 determines whether or not to select the I_PCM mode for each CU of any size (in any hierarchy) set in the sequence parameter set, not just the LCU. Can do. As a result, for example, it is possible to execute a process of restricting the generation of many bits in a low QP by the I_PCM mode in units of smaller CUs, and the code amount (uncoded by the I_PCM mode) (Including the data amount of the data) can be performed in more detail, and redundant processing that occurs in the I_PCM mode can be reduced.

  In addition, the PCM determination unit 342 supplies an On / Off control signal indicating the selection result to the adaptive left shift unit 303, the adaptive right shift unit 313, and the adaptive left shift unit 315, thereby controlling IBDI according to the selection result. I do. That is, when the CU is in the I_PCM mode, the PCM determination unit 342 performs control so that the bit shift increase / decrease process is not performed by the adaptive shift device.

  For example, when the I_PCM mode is not selected, the PCM determination unit 342 supplies a control signal indicating “On” to the adaptive left shift unit 303, the adaptive right shift unit 313, and the adaptive left shift unit 315, and performs a left shift process. Or right shift processing is executed so that the bit precision is expanded in the internal processing.

  On the other hand, for example, when the I_PCM mode is selected, the PCM determination unit 342 supplies a control signal indicating “Off” to the adaptive left shift unit 303, the adaptive right shift unit 313, and the adaptive left shift unit 315, The left shift process and the right shift process are omitted so that the bit precision is not expanded in the internal process.

  In the I_PCM mode, since the input image pixel value itself is transmitted to the image compression information, there is no calculation error, and increasing the bit calculation accuracy is a redundant process. The PCM determination unit 342 can eliminate such redundant processing by performing the processing as described above.

  Furthermore, the PCM determination unit 342 controls the adaptive loop filter processing (BALF) according to the selection result by supplying an On / Off control signal indicating the selection result to the loop filter 312. That is, the PCM determination unit 342 performs control so that the adaptive loop filter processing by the loop filter 312 is not performed when the CU is in the I_PCM mode.

  For example, when the I_PCM mode is not selected, the PCM determination unit 342 supplies a control signal indicating “On” to the loop filter 312 so that adaptive loop filter processing is performed. On the other hand, for example, when the I_PCM mode is selected, the PCM determination unit 342 supplies a control signal indicating “Off” to the loop filter 312 and omits the adaptive loop filter processing.

  In the I_PCM mode, since the input image pixel value itself is transmitted to the image compression information, there is no deterioration, and it is redundant to perform adaptive loop filter processing on this. The PCM determination unit 342 can eliminate such redundant processing by performing the processing as described above.

  As illustrated in FIG. 8, the loop filter 312 includes a deblock filter 351, a pixel sorting unit 352, a filter coefficient calculation unit 353, and a filtering unit 354.

  Similarly to the case of the deblocking filter 111, the deblocking filter 351 removes block distortion by performing a deblocking filter process on the decoded image (pixel value before deblocking filter) supplied from the calculation unit 311.

  Even if the CU to be processed is processed in the I_PCM mode, the CU adjacent to the CU is not necessarily processed in the I_PCM mode. Therefore, even when the CU is processed in the I_PCM mode, block distortion may occur. Therefore, the deblocking filter process is executed regardless of whether or not the CU is in the I_PCM mode.

  The deblock filter 351 supplies the filter processing result (pixel value after deblock filter) to the pixel sorting unit 352.

  In accordance with the value of the On / Off control signal supplied from the PCM determination unit 342, the pixel sorting unit 352 converts each filter processing result (pixel value after deblocking filter), a pixel value for performing adaptive loop filter processing, and an adaptive loop filter Sort into pixel values that are not processed.

  For example, when a control signal indicating “On” is supplied from the PCM determination unit 342, the pixel sorting unit 352 sorts the post-deblock filter pixel value of the CU into a pixel value for performing adaptive loop filter processing. On the other hand, for example, when a control signal indicating “Off” is supplied from the PCM determination unit 342, the pixel sorting unit 352 uses the pixel value after the deblocking filter of the CU as a pixel for which adaptive loop filter processing is not performed. Sort into values.

  The pixel sorting unit 352 supplies the pixel value (pixel value after deblocking filter) of each pixel that has been sorted to the filter coefficient calculation unit 353.

  The filter coefficient calculation unit 353 uses a filter coefficient (FIR filter coefficient) of the adaptive loop filter as the input image for the pixel value classified into the one that performs the adaptive loop filter process among the supplied pixel values after the deblocking filter. Is calculated by a Wiener filter so as to minimize the error with respect to. That is, the filter coefficient calculation unit 353 calculates the filter coefficient by excluding pixels processed in the I_PCM mode.

  The filter coefficient calculation unit 353 supplies the pixel value after the deblocking filter and the calculated filter coefficient to the filtering unit 354.

  The filtering unit 354 performs adaptive loop filter processing on the pixel values sorted by the adaptive loop filter processing using the supplied filter coefficients. The filtering unit 354 supplies the pixel value after the filter processing and the pixel value sorted to the side where the adaptive loop filter processing is not performed to the adaptive right shift unit 313 as the pixel value after adaptive filtering.

  Further, the filtering unit 354 generates an adaptive filter flag (on / off_flag) indicating whether or not the filtering process has been performed for each predetermined block set independently of the CU. The adaptive filter flag value setting method is arbitrary.

  For example, when an adaptive loop filter process is performed on some or all pixels in the block, the adaptive filter flag is set to a value (for example, “1”) indicating that the filter process has been performed. May be. For example, when all the pixels in the block are not subjected to the adaptive loop filter process, the adaptive filter flag is set to a value (for example, “0”) indicating that the filter process is not performed. Also good. The value of the adaptive filter flag may be set based on other criteria.

  The filtering unit 354 supplies the generated adaptive filter flag to the CU encoding unit 332 of the lossless encoding unit 307, encodes it, and provides it to the decoding side. However, when the value of the adaptive filter flag is a value (for example, “0”) indicating that the filtering process is not performed, the provision to the decoding side can be omitted.

  For example, the value of the adaptive filter flag is a value (for example, “0”) indicating that the filtering process has not been performed, and the encoding scheme of the lossless encoding unit 307 (CU encoding unit 332) is VLC. In this case, the filtering unit 354 omits the supply of the adaptive filter flag (not provided to the decoding side). Further, for example, the value of the adaptive filter flag is a value (for example, “0”) indicating that the filtering process is not performed, and the encoding scheme of the lossless encoding unit 307 (CU encoding unit 332) is In the case of CABAC, the filtering unit 354 supplies this adaptive filter flag to the CU encoding unit 332 of the lossless encoding unit 307 (provides it to the decoding side).

  This is because, in the case of VLC, it is possible to achieve higher encoding efficiency when the input information is small. However, in the case of CABAC, if the same input information continues, there is a bias in the probability of arithmetic encoding. This is because it is possible to achieve higher encoding efficiency.

  Further, the filtering unit 354 supplies the filter coefficient used for the adaptive loop filter process to the CU encoding unit 332 of the lossless encoding unit 307, encodes it, and provides it to the decoding side.

[PCM decision part]
FIG. 9 is a block diagram illustrating a main configuration example of the PCM determination unit 342 of FIG.

  As illustrated in FIG. 9, the PCM determination unit 342 includes an input data amount calculation unit 361, a PCM determination unit 362, an encoding control unit 363, an adaptive shift control unit 364, and a filter control unit 365.

  The input data amount calculation unit 361 calculates the input data amount that is the data amount of the input pixel value supplied from the screen rearrangement buffer 302 for the CU, and supplies the calculated input data amount to the PCM determination unit 362.

  The PCM determination unit 362 acquires the generated code amount (generated bits) supplied from the rate control unit 320, compares it with the input data amount supplied from the input data amount calculation unit 361, and based on the comparison result, Whether or not to select the I_PCM mode is determined. That is, the PCM determination unit 362 determines whether to select the I_PCM mode for each CU in an arbitrary hierarchy. The PCM determination unit 362 supplies the determination result to the encoding control unit 363, the adaptive shift control unit 364, and the filter control unit 365.

  Based on the determination result (information indicating whether or not the I_PCM mode is selected) supplied from the PCM determination unit 362, the encoding control unit 363 performs On / Off for the CU encoding unit 332 and the I_PCM_flag generation unit 341. Supply control signals.

  Therefore, the encoding control unit 363 can control the encoding mode for each CU in any hierarchy. As a result, the encoding control unit 363 can not only control the code amount (including the data amount of unencoded data in the I_PCM mode) in more detail, but also perform redundant processing when the I_PCM mode is selected. Can be reduced.

  Based on the determination result (information indicating whether the I_PCM mode is selected) supplied from the PCM determination unit 362, the adaptive shift control unit 364, the adaptive left shift unit 303, the adaptive right shift unit 313, and the adaptive left shift An On / Off control signal is supplied to the unit 315.

  Therefore, the adaptive shift control unit 364 can perform control so that the bit depth is not expanded in the internal calculation in the I_PCM mode. Thereby, the adaptive shift control unit 364 can reduce redundant processing.

  The filter control unit 365 supplies an On / Off control signal to the pixel sorting unit 352 based on the determination result (information indicating whether the I_PCM mode is selected) supplied from the PCM determination unit 362.

  Therefore, the filter control unit 365 can perform control so as not to execute the adaptive loop filter process in the I_PCM mode. Thereby, the filter control unit 365 can reduce redundant processing.

  As described above, the image encoding device 300 can reduce redundant processing and suppress reduction in encoding processing efficiency. Further, the image encoding device 300 can select the I_PCM mode in more detail (for each small data unit), and can improve the encoding efficiency. Therefore, the image encoding device 300 can improve the encoding efficiency while suppressing a decrease in the efficiency of the encoding process.

[Flow of encoding process]
Next, the flow of each process executed by the image encoding device 300 as described above will be described. First, an example of the flow of encoding processing will be described with reference to the flowchart of FIG.

  In step S301, the A / D converter 301 performs A / D conversion on the input image. In step S302, the screen rearrangement buffer 302 stores the A / D-converted image, and rearranges the picture from the display order to the encoding order.

  In step S303, the adaptive left shift unit 303 adaptively performs a left shift on the input image based on the control of the PCM encoding unit 321. In step S304, the adaptive left shift unit 315 adaptively performs a left shift on the reference image.

  In step S305, the intra prediction unit 317 performs an intra prediction process in the intra prediction mode using the reference image shifted to the left in step S304. In step S306, the motion prediction / compensation unit 318 performs inter motion prediction processing that performs motion prediction and motion compensation in the inter prediction mode using the reference image that has been shifted to the left in step S304.

  Actually, the process of shifting the bit depth of the reference image to the left may be performed when the reference image is read from the frame memory 314 in the intra prediction process or the inter motion prediction process.

  In step S307, the selection unit 319 determines an optimal mode based on the cost function values output from the intra prediction unit 317 and the motion prediction / compensation unit 318. That is, the selection unit 319 selects one of the prediction image generated by the intra prediction unit 317 and the prediction image generated by the motion prediction / compensation unit 318.

  Also, the selection information indicating which prediction image has been selected is supplied to the intra prediction unit 317 and the motion prediction / compensation unit 318 that has selected the prediction image. When the prediction image in the optimal intra prediction mode is selected, the intra prediction unit 317 supplies intra prediction mode information indicating the optimal intra prediction mode and the like to the lossless encoding unit 307. When the prediction image of the optimal inter prediction mode is selected, the motion prediction / compensation unit 318 sends information indicating the optimal inter prediction mode and, if necessary, information corresponding to the optimal inter prediction mode to the lossless encoding unit 307. Output. Information according to the optimal inter prediction mode includes motion vector information, flag information, reference frame information, and the like.

  In step S308, the calculation unit 304 calculates a difference between the image whose bit depth is shifted to the left by the process of step S303 and the predicted image selected by the process of step S307. The predicted image is supplied from the motion prediction / compensation unit 318 when performing inter prediction, or from the intra prediction unit 317 when performing intra prediction, to the calculation unit 304 via the selection unit 319.

  The data amount of the difference data is reduced compared to the original image data. Therefore, the data amount can be compressed as compared with the case where the image is encoded as it is.

  In step S309, the orthogonal transform unit 305 performs orthogonal transform on the difference information generated by the process in step S308. Specifically, orthogonal transformation such as discrete cosine transformation and Karhunen-Loeve transformation is performed, and transformation coefficients are output.

  In step S310, the quantization unit 306 quantizes the orthogonal transform coefficient obtained by the process in step S309.

  In step S311, the lossless encoding unit 307 encodes the transform coefficient quantized by the process in step S310. That is, lossless encoding such as variable length encoding or arithmetic encoding is performed on the difference image.

  The lossless encoding unit 307 encodes the quantization parameter calculated in step S310 and adds it to the encoded data. In addition, the lossless encoding unit 307 encodes information regarding the mode of the predicted image selected by the process of step S307, and adds the encoded information to the encoded data obtained by encoding the difference image. That is, the lossless encoding unit 307 also encodes the optimal intra prediction mode information supplied from the intra prediction unit 317 or the information corresponding to the optimal inter prediction mode supplied from the motion prediction / compensation unit 318, and the like. Append to data.

  Further, the lossless encoding unit 307 encodes the filter coefficient and flag information acquired from the loop filter 312 and adds them to the encoded data. Further, the lossless encoding unit 307 also encodes NAL data.

  In step S312, the accumulation buffer 308 accumulates the encoded data output from the lossless encoding unit 307. The encoded data stored in the storage buffer 308 is appropriately read and transmitted to the decoding side via a transmission path or a recording medium.

  In step S313, the rate control unit 320 calculates the code amount (generated code amount) of the encoded data accumulated in the accumulation buffer 308 by the process of step S312 and based on the calculated amount, the overflow or underflow does not occur. Controls the rate of the quantization operation of the quantization unit 306. Further, the rate control unit 320 supplies the generated code amount to the PCM encoding unit 321.

  In step S314, the PCM encoding unit 321 performs a PCM encoding control process using the generated code amount calculated in step S313. In step S315, the lossless encoding unit 307 performs a PCM encoding process according to the control of the PCM encoding unit 321.

  In step S316, the inverse quantization unit 309 through the frame memory 314 locally decode the difference information quantized by the process in step S310, and execute a reference image generation process for generating a reference image.

  When the process of step S316 ends, the encoding process ends. This encoding process is repeatedly executed for each CU, for example.

[PCM encoding control processing]
Next, an example of the flow of the PCM encoding control process executed in step S314 in FIG. 10 will be described with reference to the flowchart in FIG.

  When the PCM encoding control process is started, in step S331, the PCM determining unit 362 of the PCM determining unit 342 supplies the encoded data of the quantized orthogonal transform coefficient of the CU supplied from the rate control unit 320. Get the amount of generated code.

  In step S332, the input data amount calculation unit 361 calculates the input data amount of the input pixel value of the CU.

  In step S333, the PCM determination unit 362 compares the code amount acquired in step S331 with the input data amount calculated in step S332, and determines whether or not to perform encoding in the I_PCM mode.

  In step S334, the I_PCM_flag generation unit 341 generates I_PCM_flag based on the On / Off control signal indicating the determination result in step S333 supplied by the encoding control unit 363.

  In step S335, the encoding control unit 363 controls the encoding of CU data by supplying an On / Off control signal indicating the determination result of step S333 to the CU encoding unit 332.

  In step S336, the adaptive shift control unit 364 supplies an On / Off control signal indicating the determination result of step S333 to the adaptive left shift unit 303, the adaptive right shift unit 313, and the adaptive left shift unit 315. Control the adaptive shift process.

  In step S337, the encoding control unit 363 controls the adaptive loop filter process by supplying an On / Off control signal indicating the determination result in step S333 to the pixel sorting unit 352 of the loop filter 312.

  When the process of step S337 ends, the PCM determination unit 342 ends the PCM encoding control process, returns the process to step S314 of FIG. 10, and executes the processes after step S315.

[Flow of PCM encoding process]
Next, an example of the flow of the PCM encoding process executed in step S315 in FIG. 10 will be described with reference to the flowchart in FIG.

  When the PCM encoding process is started, in step S351, the CU encoding unit 332 determines whether to perform encoding in the I_PCM mode. When it is controlled to perform encoding in the I_PCM mode by the PCM encoding control process described above, the CU encoding unit 332 advances the process to step S352. In step S352, the CU encoding unit 332 selects an input pixel value of the CU as an encoding result. The CU encoding unit 332 discards the CU data of the CU in the accumulation buffer 308 and accumulates the input pixel value in the accumulation buffer 308.

  When the process of step S352 ends, the CU encoding unit 332 advances the process to step S353. In Step S351, when it is determined that encoding is not performed in the I_PCM mode, the CU encoding unit 332 advances the process to Step S353.

  In step S353, the CU encoding unit 332 encodes I_PCM_flag generated by the above-described PCM encoding control process, and accumulates it in the accumulation buffer 308.

  When the process of step S353 ends, the CU encoding unit 332 ends the PCM encoding process, returns the process to step S315 of FIG. 10, and causes the processes after step S316 to be executed.

[Flow of reference image generation processing]
Next, an example of the flow of the reference image generation process executed in step S316 in FIG. 10 will be described with reference to the flowchart in FIG.

  When the reference image generation process is started, the adaptive left shift unit 315 determines whether or not the I_PCM mode is selected based on the control of the adaptive shift control unit 364 in step S371. When the I_PCM mode is selected, the bit depth is not expanded in the internal calculation, so that the prediction process starts again. That is, if it is determined that the I_PCM mode is selected, the adaptive left shift unit 315 advances the process to step S372 without performing the left shift process on the reference image.

  In step S372, the intra prediction unit 317 performs an intra prediction process using a reference image whose bit depth is not shifted to the left. In step S373, the motion prediction / compensation unit 318 performs an inter motion prediction process using a reference image whose bit depth is not shifted to the left.

  In step S374, the selection unit 319 determines an optimal mode based on the cost function values output from the intra prediction unit 317 and the motion prediction / compensation unit 318. That is, the selection unit 319 selects one of the prediction image generated by the intra prediction unit 317 and the prediction image generated by the motion prediction / compensation unit 318.

  Also, the selection information indicating which prediction image has been selected is supplied to the intra prediction unit 317 and the motion prediction / compensation unit 318 that has selected the prediction image. When the prediction image in the optimal intra prediction mode is selected, the intra prediction unit 317 supplies intra prediction mode information indicating the optimal intra prediction mode and the like to the lossless encoding unit 307. When the prediction image of the optimal inter prediction mode is selected, the motion prediction / compensation unit 318 sends information indicating the optimal inter prediction mode and, if necessary, information corresponding to the optimal inter prediction mode to the lossless encoding unit 307. Supply. Information according to the optimal inter prediction mode includes motion vector information, flag information, reference frame information, and the like.

  When the I_PCM mode is selected, the bit depth is not expanded in the internal calculation. That is, the left shift process for the reference image by the adaptive left shift unit 303 is omitted. In step S375, the calculation unit 304 calculates the difference between the input image whose bit depth is not shifted to the left and the predicted image selected by the process in step S374. The predicted image is supplied from the motion prediction / compensation unit 318 when performing inter prediction, or from the intra prediction unit 317 when performing intra prediction, to the calculation unit 304 via the selection unit 319.

  In step S376, the orthogonal transform unit 305 performs orthogonal transform on the difference information generated by the process in step S375. Specifically, orthogonal transformation such as discrete cosine transformation and Karhunen-Loeve transformation is performed, and transformation coefficients are output. In step S377, the quantization unit 306 quantizes the orthogonal transform coefficient obtained by the process in step S376.

  When the orthogonal transform coefficient is quantized, the quantization unit 306 advances the processing to step S378, and generates a reference image using the orthogonal transform coefficient (data not subjected to the left shift process) quantized in step S377. Done.

  On the other hand, if it is determined in step S371 that the mode is not the I_PCM mode, the adaptive left shift unit 315 omits the processes in steps S372 to S377 and advances the process to step S378. That is, when the mode is not the I_PCM mode, the reference image is generated using the orthogonal transform coefficient (data subjected to the left shift process) quantized in step S310 of FIG.

  In step S378, the inverse quantization unit 309 performs inverse quantization on the quantized orthogonal transform coefficient (also referred to as a quantization coefficient) with characteristics corresponding to the characteristics of the quantization unit 306. In step S379, the inverse orthogonal transform unit 310 performs the inverse orthogonal transform on the orthogonal transform coefficient obtained by the process of step S378 with characteristics corresponding to the characteristics of the orthogonal transform unit 305.

  In step S380, the calculation unit 311 adds the predicted image to the locally decoded difference information, and generates a locally decoded image (an image corresponding to the input to the calculation unit 304). For example, in the case of the I_PCM mode, the calculation unit 311 adds a predicted image that has not been left-shifted to difference information that has not been left-shifted, and generates a decoded image that has not been left-shifted. For example, when not in the I_PCM mode, the arithmetic unit 311 adds the predicted image subjected to the left shift process to the difference information subjected to the left shift process, and generates a decoded image subjected to the left shift process.

  In step S381, the loop filter 312 performs loop filter processing on the local decoded image obtained by the processing in step S380 based on the control of the filter control unit 365, and performs deblocking filter processing, adaptive loop filter processing, and the like. Is appropriately performed.

  In step S382, the adaptive right shift unit 313 determines whether the I_PCM mode is selected based on the control of the adaptive shift control unit 364. If it is determined that the I_PCM mode is not selected, the adaptive right shift unit 313 advances the process to step S383.

  If the decoded image is not in the I_PCM mode, the bit depth of the decoded image is expanded in the internal calculation. Therefore, the adaptive right shift unit 313 performs the filter processing result (decoded image) obtained by the loop filter processing in step S381 in step S383. The right shift is adaptively performed with respect to the bit depth. When the process of step S383 ends, the adaptive right shift unit 313 advances the process to step S384.

  If it is determined in step S382 that the I_PCM mode is selected, the adaptive right shift unit 313 advances the process to step S384 without performing the right shift process.

  In step S384, the frame memory 314 stores the decoded image. When the process of step S384 ends, the frame memory 314 ends the reference image generation process, returns the process to step S316 of FIG. 10, and ends the encoding process.

[Flow of loop filter processing]
Next, an example of the flow of the loop filter process executed in step S381 in FIG. 13 will be described with reference to the flowchart in FIG.

  When the loop filter process is started, the deblock filter 351 of the loop filter 312 performs the deblock filter process on the decoded image (pre-deblock filter pixel value) supplied from the calculation unit 311 in step S401.

  In step S402, the pixel sorting unit 352 is controlled by the filter control unit 365 of the PCM determination unit 342, and sorts each pixel of the decoded image based on whether or not the mode is the I_PCM mode.

  In step S <b> 403, the filter coefficient calculation unit 353 calculates a filter coefficient for a pixel (processing target pixel) classified so as to perform the filter process. In step S404, the filtering unit 354 performs adaptive filter processing on the processing target pixel using the filter coefficient calculated in step S403.

  In step S405, the filtering unit 354 sets an adaptive filter flag for the processing target block, and supplies the adaptive filter flag and the filter coefficient to the CU encoding unit 332 for encoding.

  When the process of step S405 ends, the loop filter 312 ends the loop filter process, returns the process to step S381 of FIG. 13, and causes the processes after step S382 to be executed.

  By executing each process as described above, the image coding apparatus 300 can improve the coding efficiency while suppressing a reduction in the efficiency of the coding process.

<2. Second Embodiment>
[Image decoding device]
FIG. 15 is a block diagram illustrating a main configuration example of the image decoding apparatus according to the present embodiment. An image decoding apparatus 500 shown in FIG. 15 is basically the same apparatus as the image decoding apparatus 200 in FIG. 2, and decodes encoded data obtained by encoding image data.

  An image decoding apparatus 500 shown in FIG. 15 is a decoding apparatus corresponding to the image encoding apparatus 300 in FIG. The encoded data encoded by the image encoding device 300 is supplied to the image decoding device 500 via an arbitrary path such as a transmission path or a recording medium, and is decoded.

  As shown in FIG. 15, the image decoding apparatus 500 includes a storage buffer 501, a lossless decoding unit 502, an inverse quantization unit 503, an inverse orthogonal transform unit 504, a calculation unit 505, a loop filter 506, an adaptive right shift unit 507, a screen. A rearrangement buffer 508 and a D / A conversion unit 509 are included. The image decoding apparatus 500 includes a frame memory 510, an adaptive left shift unit 511, a selection unit 512, an intra prediction unit 513, a motion prediction / compensation unit 514, and a selection unit 515.

  The image decoding device 500 further includes a PCM decoding unit 516.

  The accumulation buffer 501 accumulates the transmitted encoded data as in the case of the accumulation buffer 201. This encoded data is encoded by the image encoding device 300.

  The lossless decoding unit 502 reads the encoded data from the accumulation buffer 501 at a predetermined timing, and decodes the encoded data by a method corresponding to the encoding method of the lossless encoding unit 307 in FIG. At that time, the lossless decoding unit 502 supplies I_PCM_flag included in the encoded data to the PCM decoding unit 516, and determines whether or not the mode is the I_PCM mode.

  In the I_PCM mode, since the CU data acquired from the accumulation buffer 501 is non-encoded data, the lossless decoding unit 502 supplies the CU data to the inverse quantization unit 503 according to the control of the PCM decoding unit 516.

  When not in the I_PCM mode, since the CU data acquired from the accumulation buffer 501 is encoded data, the lossless decoding unit 502 decodes the CU data according to the control of the PCM decoding unit 516, and the decoding result is an inverse quantization unit. 503 is supplied.

  For example, when the CU is intra-encoded, intra prediction mode information is stored in the header portion of the encoded data. The lossless decoding unit 502 also decodes the intra prediction mode information and supplies the information to the intra prediction unit 513. On the other hand, for example, when the CU is inter-encoded, motion vector information and inter-prediction mode information are stored in the header portion of the encoded data. The lossless decoding unit 502 also decodes the motion vector information and inter prediction mode information, and supplies the information to the motion prediction / compensation unit 514.

  When not in the I_PCM mode, the inverse quantization unit 503 converts the coefficient data (quantization coefficient) supplied from the lossless decoding unit 502 to the quantization unit 306 in FIG. Inverse quantization is performed using a method corresponding to the quantization method. That is, the inverse quantization unit 503 performs inverse quantization of the quantization coefficient by the same method as the inverse quantization unit 309 in FIG. The inverse quantization unit 503 supplies the inversely quantized coefficient data, that is, the orthogonal transform coefficient, to the inverse orthogonal transform unit 504.

  Also, the inverse quantization unit 503 supplies the CU data (unencoded image data) supplied from the lossless decoding unit 502 to the inverse orthogonal transform unit 504 when in the I_PCM mode.

  When the inverse orthogonal transform unit 504 is not in the I_PCM mode, the method corresponding to the orthogonal transform method of the orthogonal transform unit 305 in FIG. 7 (similar to the inverse orthogonal transform unit 310 in FIG. 7), as in the case of the inverse orthogonal transform unit 204. In this method, the orthogonal transform coefficient is subjected to inverse orthogonal transform. The inverse orthogonal transform unit 504 obtains decoded residual data corresponding to the residual data before being orthogonally transformed in the image coding apparatus 300 by the inverse orthogonal transform process. For example, fourth-order inverse orthogonal transform is performed. The inverse orthogonal transform unit 504 supplies the decoding residual data obtained by the inverse orthogonal transform to the calculation unit 505.

  Further, the inverse orthogonal transform unit 504 supplies the CU data (unencoded image data) supplied from the inverse quantization unit 503 to the calculation unit 505 when in the I_PCM mode.

  In addition, a prediction image is supplied to the calculation unit 505 from the intra prediction unit 513 or the motion prediction / compensation unit 514 via the selection unit 515.

  When not in the I_PCM mode, the calculation unit 505 adds the decoded residual data and the prediction image, and before the prediction image is subtracted by the calculation unit 304 of the image encoding device 300, as in the case of the calculation unit 205. The decoded image data corresponding to the image data is obtained. The arithmetic unit 505 supplies the decoded image data to the loop filter 506.

  Further, the arithmetic unit 505 supplies the CU data (uncoded image data) supplied from the inverse orthogonal transform unit 504 to the loop filter 506 in the I_PCM mode. In this case, since the CU data is not residual information, it is not necessary to add the predicted image.

  The loop filter 506 is controlled by the PCM decoding unit 516 and appropriately performs loop filter processing including deblock filter processing and adaptive loop filter processing on the decoded image supplied from the calculation unit 505.

  More specifically, the loop filter 506 removes block distortion of the decoded image by performing deblocking filter processing similar to the deblocking filter 206 on the decoded image. Then, under the control of the PCM decoding unit 516, the loop filter 506 performs loop filter processing on the deblock filter processing result (decoded image from which block distortion has been removed) using a Wiener filter (Wiener Filter). To improve the image quality.

  Note that the loop filter 506 may perform arbitrary filter processing on the decoded image. Further, the loop filter 506 may perform the filter process using the filter coefficient supplied from the image encoding device 300 in FIG.

  The loop filter 506 supplies the filter processing result (the decoded image after the filter processing) to the adaptive right shift unit 507.

  The adaptive right shift unit 507 is a processing unit similar to the adaptive right shift unit 313 (FIG. 7), and is controlled by the PCM decoding unit 516. When the I_PCM mode is not set, the decoded image data supplied from the loop filter 506 is shifted to the right. Shift in the direction and reduce the bit depth by a predetermined number of bits (eg, 4 bits) (eg, reduce the bit depth from 12 bits to 8 bits). That is, the adaptive right shift unit 507 right-shifts by the number of bits left-shifted in the image encoding device 300 and the bit depth of the image data before the left shift (from the screen rearrangement buffer 302 (FIG. 7)). Return to the state at the time of reading.

  The adaptive right shift unit 507 supplies the image data after the right shift process to the screen rearrangement buffer 508 and the frame memory 510.

  Note that this right shift amount (bit amount) is arbitrary as long as it matches the left shift amount in the image encoding device 300 (adaptive left shift unit 303). That is, it may be a fixed value or variable. For example, the shift amount may be determined in advance (shared in advance between the image encoding device 300 and the image decoding device 500). In the image decoding device 500, the image encoding device 300 The shift amount may be calculated, or information indicating the shift amount may be provided from the image encoding device 300.

  Further, the right shift process may be omitted under the control of the PCM decoding unit 516. For example, in the I_PCM mode, the image data is not subjected to the left shift process in the image encoding device 300. Therefore, in this case, the adaptive right shift unit 507 is controlled by the PCM decoding unit 516 and supplies the decoded image data supplied from the loop filter 506 to the screen rearrangement buffer 508 and the frame memory 510 without shifting in the right direction. .

  The screen rearrangement buffer 508 rearranges images in the same manner as the screen rearrangement buffer 207. That is, the order of frames rearranged for the encoding order by the screen rearrangement buffer 302 in FIG. 7 is rearranged in the original display order. The screen rearrangement buffer 508 supplies the decoded image data of each frame to the D / A conversion unit 509 in the rearranged order.

  As in the case of the D / A conversion unit 208, the D / A conversion unit 509 D / A converts the frame image supplied from the screen rearrangement buffer 508, and outputs and displays it on a display (not shown).

  The frame memory 510 stores the decoded image supplied from the adaptive right shift unit 507, and supplies the stored decoded image as a reference image to the adaptive left shift unit 511 at a predetermined timing.

  The adaptive left shift unit 511 is a processing unit similar to the adaptive left shift unit 315, is controlled by the PCM decoding unit 516, and appropriately shifts the image data (reference image) read from the frame memory 510 in the left direction, The bit depth is increased by a predetermined number of bits (for example, 4 bits).

  For example, when not in the I_PCM mode, the decoded image data supplied from the inverse orthogonal transform unit 504 to the calculation unit 505 is image data that has been left-shifted in the image encoding device 300 (for example, the bit depth is 12 bits). ). Therefore, the adaptive left shift unit 511 increases the bit depth of the reference image read from the frame memory 510 by a predetermined number of bits according to the control of the PCM decoding unit 516 (for example, the bit depth is changed from 8 bits to 12 bits).

  Then, the adaptive left shift unit 511 supplies the image data after the left shift process to the selection unit 512. By increasing the bit depth in this manner, the bit depth of the reference image can be matched with the bit depth of the decoded image, and the reference image can be added to the decoded image. In addition, it is possible to improve the accuracy of internal calculations such as prediction processing and suppress errors.

  On the other hand, for example, in the case of the I_PCM mode, the decoded image data supplied from the inverse orthogonal transform unit 504 to the calculation unit 505 is image data that has not been left-shifted in the image encoding device 300 (for example, Bit depth is 8 bits). Therefore, the adaptive left shift unit 511 supplies the reference image read from the frame memory 510 to the selection unit 512 without increasing the bit depth according to the control of the PCM decoding unit 516.

  In the case of intra prediction, the selection unit 512 supplies the reference image supplied from the adaptive left shift unit 511 to the intra prediction unit 513. In the case of inter prediction, the selection unit 512 supplies the reference image supplied from the adaptive left shift unit 511 to the motion prediction / compensation unit 514.

  Information indicating the intra prediction mode obtained by decoding the header information is appropriately supplied from the lossless decoding unit 502 to the intra prediction unit 513. The intra prediction unit 513 performs intra prediction using the reference image acquired from the frame memory 510 in the intra prediction mode used in the intra prediction unit 317, and generates a predicted image. That is, similarly to the intra prediction unit 317, the intra prediction unit 513 can perform this intra prediction in an arbitrary mode other than the mode defined in the AVC encoding method.

  The intra prediction unit 513 supplies the generated predicted image to the selection unit 515.

  Information (prediction mode information, motion vector information, reference frame information, flags, various parameters, etc.) obtained by decoding the header information is supplied from the lossless decoding unit 502 to the motion prediction / compensation unit 514.

  The motion prediction / compensation unit 514 performs inter prediction using the reference image acquired from the frame memory 510 in the inter prediction mode used in the motion prediction / compensation unit 318, and generates a prediction image. That is, similarly to the motion prediction / compensation unit 318, the motion prediction / compensation unit 514 can perform this intra prediction in an arbitrary mode other than the mode defined in the AVC encoding method.

  The motion prediction / compensation unit 514 supplies the generated predicted image to the selection unit 515.

  The selection unit 515 selects the prediction image generated by the motion prediction / compensation unit 514 or the intra prediction unit 513 and supplies the selected prediction image to the calculation unit 505 as in the case of the selection unit 213.

  The PCM decoding unit 516 controls the lossless decoding unit 502, the loop filter 506, the adaptive right shift unit 507, and the adaptive left shift unit 511 based on I_PCM_flag supplied from the lossless decoding unit 502.

[Reversible decoding unit, PCM decoding unit, and loop filter]
FIG. 16 is a block diagram illustrating a main configuration example of the lossless decoding unit 502, the PCM decoding unit 516, and the loop filter 506 of FIG.

  As shown in FIG. 16, the lossless decoding unit 502 includes a NAL decoding unit 531 and a CU decoding unit 532. The NAL decoding unit 531 decodes the encoded NAL data supplied from the accumulation buffer 501 and supplies the decoded NAL data to the CU decoding unit 532.

  The CU decoding unit 532 decodes the encoded CU data supplied from the accumulation buffer 501 based on the NAL data supplied from the NAL decoding unit 531.

  In addition, when the CU decoding unit 532 acquires I_PCM_flag by decoding CU data, the CU decoding unit 532 supplies the I_PCM_flag to the PCM decoding unit 516. The CU decoding unit 532 decodes the encoded CU data according to the control of the PCM decoding unit 516 based on the I_PCM_flag (based on the On / Off control signal supplied from the PCM encoding unit 321).

  For example, when I_PCM_flag is a value indicating that the mode is not I_PCM mode and a control signal indicating “On” is acquired from the PCM decoding unit 516, the CU decoding unit 532 is based on NAL data supplied from the NAL decoding unit 531. The encoded CU data supplied from the accumulation buffer 501 is decoded, and the quantized orthogonal transform coefficient is supplied to the inverse quantization unit 503.

  On the other hand, for example, when I_PCM_flag is a value indicating that the I_PCM mode is set and a control signal indicating “Off” is acquired from the PCM decoding unit 516, the CU decoding unit 532 receives the CU supplied from the accumulation buffer 501. Data (uncoded image data) is supplied to the inverse quantization unit 503.

  Note that the CU decoding unit 532 supplies information such as filter coefficients and adaptive filter flags obtained by decoding CU data to the loop filter 506.

  As illustrated in FIG. 8, the PCM decoding unit 516 includes an I_PCM_flag buffer 541 and a PCM control unit 542.

  The I_PCM_flag buffer 541 stores I_PCM_flag supplied from the CU decoding unit 532 of the lossless decoding unit 502 and supplies it to the PCM control unit 542 at a predetermined timing.

  Based on the value of I_PCM_flag acquired from the I_PCM_flag buffer 541, the PCM control unit 542 determines whether or not the I_PCM mode is selected at the time of encoding in the image encoding device 300. Based on the determination result, the PCM control unit 542 supplies an On / Off control signal to the CU decoding unit 532 and controls its operation.

  For example, when it is determined that the I_PCM mode is not selected, the PCM control unit 542 supplies a control signal indicating “On” to the CU decoding unit 532, and causes the CU decoding unit 532 to decode the encoded CU data, The inverse quantization unit 503 is supplied with the quantized orthogonal transform coefficient obtained by the decoding.

  On the other hand, for example, when it is determined that the I_PCM mode is selected, the PCM control unit 542 supplies a control signal indicating “Off” to the CU decoding unit 532, and outputs CU data that is unencoded data as an output pixel. The value is supplied to the inverse quantization unit 503 as a value.

  Through such control, the CU decoding unit 532 can appropriately decode the encoded data supplied from the image encoding device 300. That is, the CU decoding unit 532 can perform more appropriate decoding even when the I_PCM mode is controlled in units of CUs smaller than the LCU.

  Also, the PCM control unit 542 supplies an On / Off control signal to the adaptive right shift unit 507 and the adaptive left shift unit 511 based on the determination result of the I_PCM mode, and controls its operation.

  For example, when it is determined that the I_PCM mode has not been selected, the PCM control unit 542 supplies a control signal indicating “On” to the adaptive right shift unit 507 and the adaptive left shift unit 511, and performs left shift processing and right shift processing. The bit precision is expanded in the internal processing.

  On the other hand, for example, when it is determined that the I_PCM mode is selected, the PCM control unit 542 supplies a control signal indicating “Off” to the adaptive right shift unit 507 and the adaptive left shift unit 511, The right shift process is omitted, and the bit precision is not expanded in the internal process.

  With such control, the PCM control unit 542 can appropriately eliminate redundant processing even when the I_PCM mode is controlled in units of CUs smaller than the LCU.

  Further, the PCM control unit 542 supplies an On / Off control signal to the loop filter 506 based on the determination result of the I_PCM mode, and controls its operation. For example, when it is determined that the I_PCM mode has not been selected, the PCM control unit 542 supplies a control signal indicating “On” to the loop filter 506. On the other hand, for example, when it is determined that the I_PCM mode is selected, the PCM control unit 542 supplies a control signal indicating “Off” to the loop filter 506.

  With such control, the PCM control unit 542 can more appropriately eliminate redundant processing even when the I_PCM mode is controlled in units of CUs smaller than the LCU.

  As illustrated in FIG. 8, the loop filter 506 includes a deblock filter 551, a pixel sorting unit 552, and a filtering unit 553.

  Similar to the case of the deblocking filter 206, the deblocking filter 551 removes block distortion by performing a deblocking filter process on the decoded image (pre-deblocking filter pixel value) supplied from the arithmetic unit 505.

  Similar to the case of the image encoding device 300, the deblocking filter process is executed regardless of whether or not the CU is in the I_PCM mode. The deblock filter 551 supplies the filter processing result (pixel value after deblock filter) to the pixel sorting unit 552.

  The pixel sorting unit 552, according to the value of the On / Off control signal supplied from the PCM control unit 542, each filter processing result (pixel value after deblocking filter), the pixel value for performing the adaptive loop filter processing, and the adaptive loop filter Sort into pixel values that are not processed.

  For example, when a control signal indicating “On” is supplied from the PCM control unit 542, the pixel sorting unit 552 sorts the post-deblock filter pixel value of the CU into a pixel value to be subjected to adaptive loop filter processing. On the other hand, for example, when a control signal indicating “Off” is supplied from the PCM control unit 542, the pixel sorting unit 552 uses the post-deblock filter pixel value of the CU as a pixel for which the adaptive loop filter process is not performed. Sort into values.

  The pixel sorting unit 552 supplies the pixel value (pixel value after deblocking filter) of each pixel that has been sorted to the filtering unit 553.

  Based on the adaptive loop filter flag supplied from the CU decoding unit 532, the filtering unit 553 performs adaptive loop filter processing on the pixel values sorted to perform adaptive loop filter processing.

  The adaptive loop filter process is performed for each predetermined block set independently of the CU. When the value of the adaptive loop filter flag of the block is a value (for example, “0”) indicating that the filter process has not been performed on the encoding side, the filtering unit 553 omits the adaptive loop filter process for the block. The supplied pixel value after the deblocking filter is supplied to the adaptive right shift unit 507 as the pixel value after the adaptive filter.

  By the way, the setting method of the adaptive loop filter flag depends on the specification of the image encoding device 300 and the like. Therefore, depending on the setting method of the adaptive loop filter flag by the image encoding device 300, the value of the adaptive loop filter flag of the block is a value (for example, “1”) indicating that the filter processing has been performed on the encoding side. Even in this case, there is a possibility that a pixel in the I_PCM mode is included in the block.

  Therefore, when there is a pixel value classified in the block that does not perform the adaptive loop filter process, the filtering unit 553 performs the filter process on the encoding side based on the value of the adaptive loop filter flag of the block. Even if it is a value indicating that (for example, “1”), the adaptive loop filter process for the block is omitted, and the supplied pixel value after the deblocking filter is used as the pixel value after the adaptive filter to the adaptive right shift unit 507. Supply.

  That is, the filtering unit 553 has a value (for example, “1”) indicating that the value of the adaptive loop filter flag of the block has been filtered on the encoding side, and all the pixels in the block Only when the adaptive loop filter processing is performed, the adaptive loop filter processing is performed on the block.

  When performing the adaptive loop filter process, the filtering unit 553 performs the adaptive loop filter process using the filter coefficient supplied from the CU decoding unit 532 (the filter coefficient used in the adaptive loop filter process on the encoding side). .

  The filtering unit 553 supplies the pixel value that has been subjected to the adaptive loop filter processing to the adaptive right shift unit 507 as the post-adaptive filter pixel value.

  With such control, the PCM control unit 542 can more appropriately eliminate redundant processing even when the I_PCM mode is controlled in units of CUs smaller than the LCU.

  As described above, the image decoding apparatus 500 can realize the improvement of the encoding efficiency while suppressing the reduction of the efficiency of the encoding process by the processing of each processing unit.

[Decoding process flow]
Next, the flow of each process executed by the image decoding apparatus 500 as described above will be described. First, an example of the flow of decoding processing will be described with reference to the flowcharts of FIGS. 17 and 18.

  When the decoding process is started, in step S501, the accumulation buffer 501 accumulates the transmitted image (encoded data). In step S502, the CU decoding unit 532 acquires an adaptive filter flag from the encoded data accumulated in step S501. In step S503, the CU decoding unit 532 acquires I_PCM_flag from the encoded data accumulated in step S501. The I_PCM_flag buffer 541 stores the I_PCM_flag.

  In step S504, the PCM control unit 542, based on the value of I_PCM_flag stored in the I_PCM_flag buffer 541, encodes the encoded data (CU data) accumulated in step S501 in the I_PCM mode (that is, Whether it is non-encoded data) or not.

  When it is determined that the encoding mode is not the I_PCM mode, the PCM control unit 542 advances the process to step S505. In step S505, the lossless decoding unit 502 performs lossless decoding processing, decodes the encoded data (CU data) accumulated in step S501, and obtains quantized orthogonal transform coefficients, filter coefficients, and the like.

  In step S506, the inverse quantization unit 503 inversely quantizes the quantized orthogonal transform coefficient obtained by the process in step S505. In step S507, the inverse orthogonal transform unit 504 performs inverse orthogonal transform on the orthogonal transform coefficient inversely quantized by the processing in step S506 to generate decoded image data.

  In step S508, the adaptive left shift unit 511 acquires a reference image corresponding to the CU from the frame memory 510, and performs a left shift process on the reference image according to the control of the PCM decoding unit 516.

  In step S509, the intra prediction unit 513 or the motion prediction / compensation unit 514 performs a prediction process using the reference image or the like left-shifted in step S508, and generates a predicted image.

  In step S510, the calculation unit 505 adds the prediction image generated by the intra prediction unit 513 or the motion prediction / compensation unit 514 in step S509 to the difference information obtained by the process of step S507.

  In step S511, the loop filter 506 performs loop filter processing on the addition result in step S510.

  In step S512, the adaptive right shift unit 507 performs right shift processing on the loop filter processing result in step S511 according to the control of the PCM decoding unit 516.

  In step S513, the frame memory 510 stores the decoded image data as a reference image.

  In step S514, the screen rearrangement buffer 508 rearranges the frames of the decoded image data. That is, the order of the frames of the decoded image data rearranged for encoding by the screen rearrangement buffer 302 (FIG. 7) of the image encoding device 300 is rearranged to the original display order.

  In step S515, the D / A conversion unit 509 performs D / A conversion on the decoded image data in which the frames are rearranged in step S514. The decoded image data is output to a display (not shown), and the image is displayed.

  If it is determined in step S504 that the encoding mode is the I_PCM mode, the PCM control unit 542 advances the processing to step S521 in FIG. In step S521, the lossless decoding unit 502 performs a lossless decoding process, and sets the encoded data (uncompressed data) accumulated in step S501 as an encoding result (output pixel value).

  In step S522, the loop filter 506 performs loop filter processing on the output pixel value obtained in step S521. When the process of step S522 ends, the loop filter 506 returns the process to step S513 of FIG. 17 to execute the subsequent processes.

[Flow of loop filter processing]
Next, an example of the flow of the loop filter process executed in step S511 in FIG. 17 and step S522 in FIG. 18 will be described with reference to the flowchart in FIG.

  When the loop filter process is started, the deblock filter 551 performs the deblock filter process on the pre-deblock filter pixel value obtained in step S510 or step S521 in step S541.

  In step S542, the filtering unit 553 acquires a filter coefficient. In step S543, the pixel sorting unit 552 determines whether to perform the adaptive loop filter based on the value of the adaptive loop filter flag. If it is determined that the adaptive loop filter process is to be performed, the pixel sorting unit 552 advances the process to step S544.

  In step S544, the pixel sorting unit 552 sorts the pixel values after the deblocking filter according to whether or not the mode is the I_PCM mode according to the control of the PCM control unit 542.

  In step S545, the filtering unit 545 performs adaptive loop filter processing using the filter coefficient acquired in step S542 on the post-deblock filter pixel values sorted to perform adaptive loop filter processing. When the process of step S545 ends, the filtering unit 545 ends the loop filter process, returns the process to step S511 of FIG. 17 or step S522 of FIG. 18, and performs the process of step S512 of FIG. 17 or step S513 of FIG. Proceed.

  If it is determined in step S543 in FIG. 19 that the adaptive loop filter process is not performed, the pixel sorting unit 552 ends the loop filter process, and returns the process to step S511 in FIG. 17 or step S522 in FIG. The process proceeds to step S512 in FIG. 17 or step S513 in FIG.

  By executing each process as described above, the image decoding apparatus 500 can achieve an improvement in encoding efficiency while suppressing a reduction in the efficiency of the encoding process.

[I_PCM information location]
As described above, by enabling selection control of the I_PCM mode in units of CUs, for example, as illustrated in FIG. 20A, only some CUs in the LCU perform encoding in the I_PCM mode. It can also be done. FIG. 20A shows the structure of a CU in one LCU. The LCU shown in FIG. 20A includes seven CUs CU0 to CU6. Each number indicates the processing order in the LCU. In the example of FIG. 20A, it is assumed that CUs (CU1, CU3, CU6) shown in gray are encoded in the I_PCM mode.

  In such a case, as shown in FIG. 20B, I_PCM information that is information related to I_PCM inside the LCU may be added to the head of the LCU data in the code stream.

  The I_PCM information includes, for example, flag information indicating whether or not the LCU includes a CU encoded in the I_PCM mode. In the example of FIG. 20, since CU1, CU3, and CU6 are CUs encoded in the I_PCM mode, this flag information is a value indicating that the LCU includes a CU encoded in the I_PCM mode (for example, “1”).

  As described above, the image encoding device 300 adds information related to I_PCM in the LCU to the head of the LCU, so that the image decoding device 500 has the I_PCM mode in the LCU before decoding the LCU. It is possible to easily grasp whether or not there is a CU that is encoded by.

  Further, the I_PCM information includes, for example, information indicating a CU encoded in the I_PCM mode included in the LCU. In the example of FIG. 20, since CU1, CU3, and CU6 are CUs encoded in the I_PCM mode, the I_PCM information includes information indicating these CUs (CU1, CU3, CU6).

  In this way, the image coding apparatus 300 adds information on I_PCM in the LCU to the head of the LCU, so that the image decoding apparatus 500 can determine which CU in the LCU before decoding the LCU. Can be easily ascertained whether or not is encoded in the I_PCM mode.

  In the above description, information other than image data such as I_PCM_flag, adaptive filter flag, filter coefficient, and I_PCM information has been described as being provided from the image coding apparatus 300 to the image decoding apparatus 400 as necessary. However, these pieces of information may be added to arbitrary positions of the encoded data. For example, it may be added to the head of a CU or LCU, may be added to a slice header, or stored in a sequence parameter set (SPS), a picture parameter set (PPS), or the like. It may be. Further, for example, these pieces of information may be stored in a parameter set (eg, sequence or picture header) such as SEI (Suplemental Enhancement Information).

  Further, these pieces of information may be transmitted to the decoding side separately from the encoded data. In that case, it is necessary to clarify the correspondence between these pieces of information and encoded data (so that the information can be grasped on the decoding side), but the method is arbitrary. For example, table information indicating the correspondence relationship may be created separately, or link information indicating the correspondence destination data may be embedded in each other's data.

  These pieces of information may be shared in advance between the image encoding device 300 and the image decoding device 400. In that case, transmission of these pieces of information may be omitted.

<3. Third Embodiment>
[Personal computer]
The series of processes described above can be executed by hardware or can be executed by software. In this case, for example, a personal computer as shown in FIG. 21 may be configured.

  In FIG. 21, a CPU (Central Processing Unit) 601 of the personal computer 600 performs various processes according to a program stored in a ROM (Read Only Memory) 602 or a program loaded from a storage unit 613 into a RAM (Random Access Memory) 603. Execute the process. The RAM 603 also appropriately stores data necessary for the CPU 601 to execute various processes.

  The CPU 601, ROM 602, and RAM 603 are connected to each other via a bus 604. An input / output interface 610 is also connected to the bus 604.

  The input / output interface 610 includes an input unit 611 including a keyboard and a mouse, a display including a CRT (Cathode Ray Tube) and an LCD (Liquid Crystal Display), an output unit 612 including a speaker, a hard disk, and the like. A communication unit 614 including a storage unit 613 and a modem is connected. The communication unit 614 performs communication processing via a network including the Internet.

  A drive 615 is connected to the input / output interface 610 as necessary, and a removable medium 621 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory is appropriately mounted, and a computer program read from them is loaded. It is installed in the storage unit 613 as necessary.

  When the above-described series of processing is executed by software, a program constituting the software is installed from a network or a recording medium.

  For example, as shown in FIG. 21, the recording medium is distributed to distribute the program to the user separately from the apparatus main body, and includes a magnetic disk (including a flexible disk) on which the program is recorded, an optical disk ( It is only composed of removable media 621 consisting of CD-ROM (compact disc-read only memory), DVD (including digital versatile disc), magneto-optical disc (including MD (mini disc)), or semiconductor memory. Rather, it is composed of a ROM 602 storing a program and a hard disk included in the storage unit 613, which is distributed to the user in a state of being incorporated in the apparatus main body in advance.

  The program executed by the computer may be a program that is processed in time series in the order described in this specification, or in parallel or at a necessary timing such as when a call is made. It may be a program for processing.

  Further, in the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but may be performed in parallel or It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus composed of a plurality of devices (apparatuses).

  In addition, in the above description, the configuration described as one device (or processing unit) may be divided and configured as a plurality of devices (or processing units). Conversely, the configurations described above as a plurality of devices (or processing units) may be combined into a single device (or processing unit). Of course, a configuration other than that described above may be added to the configuration of each device (or each processing unit). Furthermore, if the configuration and operation of the entire system are substantially the same, a part of the configuration of a certain device (or processing unit) may be included in the configuration of another device (or other processing unit). . That is, the embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

  For example, the lossless encoding unit 307, the loop filter 312 and the PCM encoding unit 321 shown in FIG. 8 may be configured as independent devices. Further, the NAL encoding unit 331, the CU encoding unit 332, the I_PCM_flag generation unit 341, the PCM determination unit 342, the deblocking filter 351, the pixel sorting unit 352, the filter coefficient calculation unit 353, and the filtering unit 354 shown in FIG. Each may be configured as an independent device.

  Furthermore, the input data amount calculation unit 361, the PCM determination unit 362, the encoding control unit 363, the adaptive shift control unit 364, and the filter control unit shown in FIG. 9 may be configured as independent devices. .

  Further, these processing units may be arbitrarily combined and configured as an independent device. Of course, it may be combined with an arbitrary processing unit shown in FIGS. 7 to 9, or may be combined with a processing unit (not shown).

  The same applies to the image decoding device 400. For example, the lossless decoding unit 502, the loop filter 506, and the PCM decoding unit 516 shown in FIG. 15 may be configured as independent devices. Also, the NAL decoding unit 531, the CU decoding unit 532, the I_PCM_flag buffer 541, the PCM control unit 542, the deblock filter 551, the pixel sorting unit 552, and the filtering unit 553 shown in FIG. 16 are configured as independent devices. You may do it.

  Furthermore, these processing units may be arbitrarily combined and configured as an independent device. Of course, it may be combined with any processing unit shown in FIGS. 15 and 16, or may be combined with a processing unit not shown.

  In addition, for example, the above-described image encoding device and image decoding device can be applied to any electronic device. Examples thereof will be described below.

<4. Fourth Embodiment>
[Television receiver]
FIG. 22 is a block diagram illustrating a main configuration example of a television receiver using the image decoding device 500 of the present embodiment.

  A television receiver 1000 shown in FIG. 22 includes a terrestrial tuner 1013, a video decoder 1015, a video signal processing circuit 1018, a graphic generation circuit 1019, a panel drive circuit 1020, and a display panel 1021.

  The terrestrial tuner 1013 receives a broadcast wave signal of analog terrestrial broadcasting via an antenna, demodulates it, acquires a video signal, and supplies it to the video decoder 1015. The video decoder 1015 performs a decoding process on the video signal supplied from the terrestrial tuner 1013 and supplies the obtained digital component signal to the video signal processing circuit 1018.

  The video signal processing circuit 1018 performs predetermined processing such as noise removal on the video data supplied from the video decoder 1015 and supplies the obtained video data to the graphic generation circuit 1019.

  The graphic generation circuit 1019 generates video data of a program to be displayed on the display panel 1021, image data by processing based on an application supplied via a network, and the generated video data and image data to the panel drive circuit 1020. Supply. The graphic generation circuit 1019 generates video data (graphics) for displaying a screen used by the user for selecting an item and superimposing it on the video data of the program. A process of supplying data to the panel drive circuit 1020 is also appropriately performed.

  The panel drive circuit 1020 drives the display panel 1021 based on the data supplied from the graphic generation circuit 1019 and causes the display panel 1021 to display a program video and the various screens described above.

  The display panel 1021 includes an LCD (Liquid Crystal Display) or the like, and displays a program video or the like according to control by the panel drive circuit 1020.

  The television receiver 1000 also includes an audio A / D (Analog / Digital) conversion circuit 1014, an audio signal processing circuit 1022, an echo cancellation / audio synthesis circuit 1023, an audio amplification circuit 1024, and a speaker 1025.

  The terrestrial tuner 1013 acquires not only the video signal but also the audio signal by demodulating the received broadcast wave signal. The terrestrial tuner 1013 supplies the acquired audio signal to the audio A / D conversion circuit 1014.

  The audio A / D conversion circuit 1014 performs A / D conversion processing on the audio signal supplied from the terrestrial tuner 1013 and supplies the obtained digital audio signal to the audio signal processing circuit 1022.

  The audio signal processing circuit 1022 performs predetermined processing such as noise removal on the audio data supplied from the audio A / D conversion circuit 1014, and supplies the obtained audio data to the echo cancellation / audio synthesis circuit 1023.

  The echo cancellation / voice synthesis circuit 1023 supplies the voice data supplied from the voice signal processing circuit 1022 to the voice amplification circuit 1024.

  The audio amplifying circuit 1024 performs D / A conversion processing and amplification processing on the audio data supplied from the echo cancellation / audio synthesizing circuit 1023, adjusts to a predetermined volume, and then outputs the audio from the speaker 1025.

  Furthermore, the television receiver 1000 also includes a digital tuner 1016 and an MPEG decoder 1017.

  The digital tuner 1016 receives a broadcast wave signal of a digital broadcast (terrestrial digital broadcast, BS (Broadcasting Satellite) / CS (Communications Satellite) digital broadcast) via an antenna, demodulates, and MPEG-TS (Moving Picture Experts Group). -Transport Stream) and supply it to the MPEG decoder 1017.

  The MPEG decoder 1017 cancels the scramble applied to the MPEG-TS supplied from the digital tuner 1016, and extracts a stream including program data to be played back (viewing target). The MPEG decoder 1017 decodes the audio packet constituting the extracted stream, supplies the obtained audio data to the audio signal processing circuit 1022, decodes the video packet constituting the stream, and converts the obtained video data into the video This is supplied to the signal processing circuit 1018. Also, the MPEG decoder 1017 supplies EPG (Electronic Program Guide) data extracted from the MPEG-TS to the CPU 1032 via a path (not shown).

  The television receiver 1000 uses the above-described image decoding device 500 as the MPEG decoder 1017 for decoding video packets in this way. Note that MPEG-TS transmitted from a broadcasting station or the like is encoded by the image encoding device 300.

  As in the case of the image decoding apparatus 500, the MPEG decoder 1017 appropriately decodes encoded data in which selection of the I_PCM mode is controlled in units of CUs smaller than the LCU. Therefore, the MPEG decoder 1017 can realize reduction of redundant processing during encoding and reduction of redundant information included in the encoded data. Thereby, the MPEG decoder 1017 can implement | achieve improving encoding efficiency, suppressing the reduction of the efficiency of an encoding process.

  The video data supplied from the MPEG decoder 1017 is subjected to predetermined processing in the video signal processing circuit 1018 as in the case of the video data supplied from the video decoder 1015, and the generated video data in the graphic generation circuit 1019. Are appropriately superimposed and supplied to the display panel 1021 via the panel drive circuit 1020, and the image is displayed.

  The audio data supplied from the MPEG decoder 1017 is subjected to predetermined processing in the audio signal processing circuit 1022 as in the case of the audio data supplied from the audio A / D conversion circuit 1014, and an echo cancellation / audio synthesis circuit 1023. Are supplied to the audio amplifier circuit 1024 through which D / A conversion processing and amplification processing are performed. As a result, sound adjusted to a predetermined volume is output from the speaker 1025.

  The television receiver 1000 also includes a microphone 1026 and an A / D conversion circuit 1027.

  The A / D conversion circuit 1027 receives a user's voice signal captured by a microphone 1026 provided in the television receiver 1000 for voice conversation, and performs A / D conversion processing on the received voice signal. The obtained digital audio data is supplied to the echo cancellation / audio synthesis circuit 1023.

  When the audio data of the user (user A) of the television receiver 1000 is supplied from the A / D conversion circuit 1027, the echo cancellation / audio synthesis circuit 1023 performs echo cancellation on the audio data of the user A. The voice data obtained by combining with other voice data is output from the speaker 1025 via the voice amplifier circuit 1024.

  The television receiver 1000 further includes an audio codec 1028, an internal bus 1029, an SDRAM (Synchronous Dynamic Random Access Memory) 1030, a flash memory 1031, a CPU 1032, a USB (Universal Serial Bus) I / F 1033, and a network I / F 1034. .

  The A / D conversion circuit 1027 receives a user's voice signal captured by a microphone 1026 provided in the television receiver 1000 for voice conversation, and performs A / D conversion processing on the received voice signal. The obtained digital audio data is supplied to the audio codec 1028.

  The audio codec 1028 converts the audio data supplied from the A / D conversion circuit 1027 into data of a predetermined format for transmission via the network, and supplies the data to the network I / F 1034 via the internal bus 1029.

  The network I / F 1034 is connected to the network via a cable attached to the network terminal 1035. For example, the network I / F 1034 transmits the audio data supplied from the audio codec 1028 to another device connected to the network. In addition, the network I / F 1034 receives, for example, audio data transmitted from another device connected via the network via the network terminal 1035, and receives the audio data via the internal bus 1029 to the audio codec 1028. Supply.

  The audio codec 1028 converts the audio data supplied from the network I / F 1034 into data of a predetermined format, and supplies it to the echo cancellation / audio synthesis circuit 1023.

  The echo cancellation / speech synthesis circuit 1023 performs echo cancellation on the speech data supplied from the speech codec 1028, and synthesizes speech data obtained by combining with other speech data via the speech amplification circuit 1024. And output from the speaker 1025.

  The SDRAM 1030 stores various data necessary for the CPU 1032 to perform processing.

  The flash memory 1031 stores a program executed by the CPU 1032. The program stored in the flash memory 1031 is read by the CPU 1032 at a predetermined timing such as when the television receiver 1000 is activated. The flash memory 1031 also stores EPG data acquired via digital broadcasting, data acquired from a predetermined server via a network, and the like.

  For example, the flash memory 1031 stores MPEG-TS including content data acquired from a predetermined server via a network under the control of the CPU 1032. The flash memory 1031 supplies the MPEG-TS to the MPEG decoder 1017 via the internal bus 1029, for example, under the control of the CPU 1032.

  The MPEG decoder 1017 processes the MPEG-TS as in the case of the MPEG-TS supplied from the digital tuner 1016. In this way, the television receiver 1000 receives content data including video and audio via the network, decodes it using the MPEG decoder 1017, displays the video, and outputs audio. Can do.

  The television receiver 1000 also includes a light receiving unit 1037 that receives an infrared signal transmitted from the remote controller 1051.

  The light receiving unit 1037 receives infrared light from the remote controller 1051 and outputs a control code representing the content of the user operation obtained by demodulation to the CPU 1032.

  The CPU 1032 executes a program stored in the flash memory 1031 and controls the overall operation of the television receiver 1000 according to a control code supplied from the light receiving unit 1037. The CPU 1032 and each part of the television receiver 1000 are connected via a path (not shown).

  The USB I / F 1033 transmits and receives data to and from a device external to the television receiver 1000 connected via a USB cable attached to the USB terminal 1036. The network I / F 1034 is connected to the network via a cable attached to the network terminal 1035, and transmits / receives data other than audio data to / from various devices connected to the network.

  By using the image decoding apparatus 500 as the MPEG decoder 1017, the television receiver 1000 can improve the efficiency of encoding processing when generating broadcast wave signals received via an antenna and content data acquired via a network. It is possible to improve the coding efficiency of the data while suppressing the reduction.

<5. Fifth embodiment>
[Mobile phone]
FIG. 23 is a block diagram illustrating a main configuration example of a mobile phone using the image encoding device 300 and the image decoding device 500 according to the present embodiment.

  A cellular phone 1100 shown in FIG. 23 includes a main control unit 1150, a power supply circuit unit 1151, an operation input control unit 1152, an image encoder 1153, a camera I / F unit 1154, an LCD control, which are configured to control each unit in an integrated manner. Section 1155, image decoder 1156, demultiplexing section 1157, recording / reproducing section 1162, modulation / demodulation circuit section 1158, and audio codec 1159. These are connected to each other via a bus 1160.

  The mobile phone 1100 includes an operation key 1119, a CCD (Charge Coupled Devices) camera 1116, a liquid crystal display 1118, a storage unit 1123, a transmission / reception circuit unit 1163, an antenna 1114, a microphone (microphone) 1121, and a speaker 1117.

  When the end call and the power key are turned on by the user's operation, the power supply circuit unit 1151 starts up the mobile phone 1100 in an operable state by supplying power from the battery pack to each unit.

  The mobile phone 1100 transmits and receives voice signals, e-mails and image data, and images in various modes such as a voice call mode and a data communication mode based on the control of the main control unit 1150 including a CPU, a ROM, a RAM, and the like. Various operations such as shooting or data recording are performed.

  For example, in the voice call mode, the mobile phone 1100 converts the voice signal collected by the microphone (microphone) 1121 into digital voice data by the voice codec 1159, performs spectrum spread processing by the modulation / demodulation circuit unit 1158, and transmits and receives The unit 1163 performs digital / analog conversion processing and frequency conversion processing. The cellular phone 1100 transmits the transmission signal obtained by the conversion process to a base station (not shown) via the antenna 1114. The transmission signal (voice signal) transmitted to the base station is supplied to the mobile phone of the other party via the public telephone line network.

  Further, for example, in the voice call mode, the cellular phone 1100 amplifies the received signal received by the antenna 1114 by the transmission / reception circuit unit 1163, further performs frequency conversion processing and analog-digital conversion processing, and the modulation / demodulation circuit unit 1158 performs spectrum despreading processing Then, the audio codec 1159 converts it into an analog audio signal. The cellular phone 1100 outputs an analog audio signal obtained by the conversion from the speaker 1117.

  Further, for example, when transmitting an e-mail in the data communication mode, the mobile phone 1100 receives e-mail text data input by operating the operation key 1119 in the operation input control unit 1152. The cellular phone 1100 processes the text data in the main control unit 1150 and displays it on the liquid crystal display 1118 as an image via the LCD control unit 1155.

  In addition, the mobile phone 1100 generates e-mail data in the main control unit 1150 based on text data received by the operation input control unit 1152, user instructions, and the like. The cellular phone 1100 performs spread spectrum processing on the e-mail data by the modulation / demodulation circuit unit 1158 and digital / analog conversion processing and frequency conversion processing by the transmission / reception circuit unit 1163. The cellular phone 1100 transmits the transmission signal obtained by the conversion process to a base station (not shown) via the antenna 1114. The transmission signal (e-mail) transmitted to the base station is supplied to a predetermined destination via a network and a mail server.

  Further, for example, when receiving an e-mail in the data communication mode, the mobile phone 1100 receives and amplifies the signal transmitted from the base station by the transmission / reception circuit unit 1163 via the antenna 1114, and further performs frequency conversion processing and Analog-digital conversion processing. The cellular phone 1100 performs spectrum despreading processing on the received signal by the modulation / demodulation circuit unit 1158 to restore the original e-mail data. The cellular phone 1100 displays the restored e-mail data on the liquid crystal display 1118 via the LCD control unit 1155.

  Note that the mobile phone 1100 can also record (store) the received e-mail data in the storage unit 1123 via the recording / playback unit 1162.

  The storage unit 1123 is an arbitrary rewritable storage medium. The storage unit 1123 may be, for example, a semiconductor memory such as a RAM or a built-in flash memory, a hard disk, or a removable disk such as a magnetic disk, a magneto-optical disk, an optical disk, a USB memory, or a memory card. It may be media. Of course, other than these may be used.

  Furthermore, for example, when transmitting image data in the data communication mode, the mobile phone 1100 generates image data with the CCD camera 1116 by imaging. The CCD camera 1116 has an optical device such as a lens and a diaphragm and a CCD as a photoelectric conversion element, images a subject, converts the intensity of received light into an electrical signal, and generates image data of the subject image. The CCD camera 1116 encodes the image data by the image encoder 1153 via the camera I / F unit 1154 and converts the encoded image data into encoded image data.

  The cellular phone 1100 uses the above-described image encoding device 300 as the image encoder 1153 that performs such processing. The image encoder 1153 controls the selection of the I_PCM mode in units of CU smaller than the LCU, as in the case of the image encoding device 300. That is, the image encoder 1153 can further reduce redundant processing at the time of encoding, and can further reduce redundant information included in the encoded data. Accordingly, the image encoder 1153 can improve the encoding efficiency while suppressing a decrease in the efficiency of the encoding process.

  At the same time, the cellular phone 1100 converts the sound collected by the microphone (microphone) 1121 during imaging by the CCD camera 1116 from analog to digital at the audio codec 1159 and further encodes it.

  The cellular phone 1100 multiplexes the encoded image data supplied from the image encoder 1153 and the digital audio data supplied from the audio codec 1159 in a demultiplexing unit 1157 using a predetermined method. The cellular phone 1100 performs spread spectrum processing on the multiplexed data obtained as a result by the modulation / demodulation circuit unit 1158 and digital / analog conversion processing and frequency conversion processing by the transmission / reception circuit unit 1163. The cellular phone 1100 transmits the transmission signal obtained by the conversion process to a base station (not shown) via the antenna 1114. A transmission signal (image data) transmitted to the base station is supplied to a communication partner via a network or the like.

  When image data is not transmitted, the mobile phone 1100 can also display the image data generated by the CCD camera 1116 on the liquid crystal display 1118 via the LCD control unit 1155 without using the image encoder 1153.

  Further, for example, when receiving data of a moving image file linked to a simple homepage or the like in the data communication mode, the mobile phone 1100 transmits a signal transmitted from the base station to the transmission / reception circuit unit 1163 via the antenna 1114. Receive, amplify, and further perform frequency conversion processing and analog-digital conversion processing. The cellular phone 1100 restores the original multiplexed data by subjecting the received signal to spectrum despreading processing by the modulation / demodulation circuit unit 1158. In the cellular phone 1100, the demultiplexing unit 1157 separates the multiplexed data and divides it into encoded image data and audio data.

  The cellular phone 1100 generates reproduced moving image data by decoding the encoded image data in the image decoder 1156, and displays it on the liquid crystal display 1118 via the LCD control unit 1155. Thereby, for example, the moving image data included in the moving image file linked to the simple homepage is displayed on the liquid crystal display 1118.

  The cellular phone 1100 uses the above-described image decoding device 500 as the image decoder 1156 that performs such processing. That is, the image decoder 1156 appropriately decodes the encoded data in which the selection of the I_PCM mode is controlled in units of CUs smaller than the LCU, as in the case of the image decoding device 500. Accordingly, the image decoder 1156 can realize reduction of redundant processing at the time of encoding and reduction of redundant information included in the encoded data. As a result, the image decoder 1156 can improve the encoding efficiency while suppressing a decrease in the efficiency of the encoding process.

  At this time, the cellular phone 1100 simultaneously converts digital audio data into an analog audio signal in the audio codec 1159 and outputs the analog audio signal from the speaker 1117. Thereby, for example, audio data included in the moving image file linked to the simple homepage is reproduced.

  As in the case of e-mail, the mobile phone 1100 can record (store) the data linked to the received simplified home page in the storage unit 1123 via the recording / playback unit 1162. .

  Further, the mobile phone 1100 can analyze the two-dimensional code captured by the CCD camera 1116 and acquire information recorded in the two-dimensional code in the main control unit 1150.

  Further, the cellular phone 1100 can communicate with an external device by infrared rays at the infrared communication unit 1181.

  By using the image encoding device 300 as the image encoder 1153, the cellular phone 1100 encodes image data generated in the CCD camera 1116, for example, while suppressing reduction in the efficiency of the encoding process when transmitting the encoded image data. Efficiency can be improved.

  In addition, the cellular phone 1100 uses the image decoding apparatus 500 as the image decoder 1156, so that, for example, the efficiency of encoding processing when generating data (encoded data) of a moving image file linked to a simple homepage or the like is improved. It is possible to improve the coding efficiency of the data while suppressing the reduction.

  In the above description, the mobile phone 1100 is described as using the CCD camera 1116. However, instead of the CCD camera 1116, an image sensor (CMOS image sensor) using a CMOS (Complementary Metal Oxide Semiconductor) is used. May be. Also in this case, the mobile phone 1100 can capture an image of a subject and generate image data of the image of the subject, as in the case where the CCD camera 1116 is used.

  In the above description, the cellular phone 1100 has been described. For example, an imaging function similar to that of the cellular phone 1100 such as a PDA (Personal Digital Assistants), a smartphone, an UMPC (Ultra Mobile Personal Computer), a netbook, a notebook personal computer, or the like. As long as it is a device having a communication function, the image encoding device 300 and the image decoding device 400 of the present embodiment can be applied to any device as in the case of the mobile phone 1100.

<6. Sixth Embodiment>
[Hard Disk Recorder]
FIG. 24 is a block diagram illustrating a main configuration example of a hard disk recorder using the image encoding device 300 and the image decoding device 500 of the present embodiment.

  A hard disk recorder (HDD recorder) 1200 shown in FIG. 24 receives audio data and video data of a broadcast program included in a broadcast wave signal (television signal) transmitted from a satellite or a ground antenna received by a tuner. This is an apparatus for storing in a built-in hard disk and providing the stored data to the user at a timing according to the user's instruction.

  The hard disk recorder 1200 can extract, for example, audio data and video data from broadcast wave signals, appropriately decode them, and store them in a built-in hard disk. The hard disk recorder 1200 can also acquire audio data and video data from other devices via a network, for example, decode them as appropriate, and store them in a built-in hard disk.

  Further, the hard disk recorder 1200, for example, decodes audio data and video data recorded on the built-in hard disk, supplies them to the monitor 1260, displays the image on the screen of the monitor 1260, and displays the sound from the speaker of the monitor 1260. Can be output. Further, the hard disk recorder 1200 decodes audio data and video data extracted from a broadcast wave signal acquired via a tuner, or audio data and video data acquired from another device via a network, for example. The image can be supplied to the monitor 1260, the image can be displayed on the screen of the monitor 1260, and the sound can be output from the speaker of the monitor 1260.

  Of course, other operations are possible.

  As shown in FIG. 24, the hard disk recorder 1200 includes a receiving unit 1221, a demodulating unit 1222, a demultiplexer 1223, an audio decoder 1224, a video decoder 1225, and a recorder control unit 1226. The hard disk recorder 1200 further includes an EPG data memory 1227, a program memory 1228, a work memory 1229, a display converter 1230, an OSD (On Screen Display) control unit 1231, a display control unit 1232, a recording / playback unit 1233, a D / A converter 1234, And a communication unit 1235.

  In addition, the display converter 1230 includes a video encoder 1241. The recording / playback unit 1233 includes an encoder 1251 and a decoder 1252.

  The receiving unit 1221 receives an infrared signal from a remote controller (not shown), converts it into an electrical signal, and outputs it to the recorder control unit 1226. The recorder control unit 1226 is constituted by, for example, a microprocessor and executes various processes according to a program stored in the program memory 1228. At this time, the recorder control unit 1226 uses the work memory 1229 as necessary.

  The communication unit 1235 is connected to a network and performs communication processing with other devices via the network. For example, the communication unit 1235 is controlled by the recorder control unit 1226, communicates with a tuner (not shown), and mainly outputs a channel selection control signal to the tuner.

  The demodulator 1222 demodulates the signal supplied from the tuner and outputs the demodulated signal to the demultiplexer 1223. The demultiplexer 1223 separates the data supplied from the demodulation unit 1222 into audio data, video data, and EPG data, and outputs them to the audio decoder 1224, the video decoder 1225, or the recorder control unit 1226, respectively.

  The audio decoder 1224 decodes the input audio data and outputs it to the recording / playback unit 1233. The video decoder 1225 decodes the input video data and outputs it to the display converter 1230. The recorder control unit 1226 supplies the input EPG data to the EPG data memory 1227 for storage.

  The display converter 1230 encodes the video data supplied from the video decoder 1225 or the recorder control unit 1226 into, for example, NTSC (National Television Standards Committee) video data by the video encoder 1241 and outputs the encoded video data to the recording / reproducing unit 1233. The display converter 1230 converts the screen size of the video data supplied from the video decoder 1225 or the recorder control unit 1226 into a size corresponding to the size of the monitor 1260, and converts the video data to NTSC video data by the video encoder 1241. Then, it is converted into an analog signal and output to the display control unit 1232.

  The display control unit 1232 superimposes the OSD signal output from the OSD (On Screen Display) control unit 1231 on the video signal input from the display converter 1230 under the control of the recorder control unit 1226, and displays it on the monitor 1260 display. Output and display.

  The monitor 1260 is also supplied with the audio data output from the audio decoder 1224 after being converted into an analog signal by the D / A converter 1234. The monitor 1260 outputs this audio signal from a built-in speaker.

  The recording / playback unit 1233 includes a hard disk as a storage medium for recording video data, audio data, and the like.

  For example, the recording / reproducing unit 1233 encodes the audio data supplied from the audio decoder 1224 by the encoder 1251. The recording / playback unit 1233 encodes the video data supplied from the video encoder 1241 of the display converter 1230 by the encoder 1251. The recording / playback unit 1233 combines the encoded data of the audio data and the encoded data of the video data by a multiplexer. The recording / playback unit 1233 amplifies the synthesized data by channel coding, and writes the data to the hard disk via the recording head.

  The recording / playback unit 1233 plays back the data recorded on the hard disk via the playback head, amplifies it, and separates it into audio data and video data by a demultiplexer. The recording / playback unit 1233 uses the decoder 1252 to decode the audio data and the video data. The recording / playback unit 1233 performs D / A conversion on the decoded audio data and outputs it to the speaker of the monitor 1260. In addition, the recording / playback unit 1233 performs D / A conversion on the decoded video data and outputs it to the display of the monitor 1260.

  The recorder control unit 1226 reads the latest EPG data from the EPG data memory 1227 based on the user instruction indicated by the infrared signal from the remote controller received via the receiving unit 1221, and supplies it to the OSD control unit 1231. To do. The OSD control unit 1231 generates image data corresponding to the input EPG data, and outputs the image data to the display control unit 1232. The display control unit 1232 outputs the video data input from the OSD control unit 1231 to the display of the monitor 1260 for display. As a result, an EPG (electronic program guide) is displayed on the display of the monitor 1260.

  Also, the hard disk recorder 1200 can acquire various data such as video data, audio data, or EPG data supplied from another device via a network such as the Internet.

  The communication unit 1235 is controlled by the recorder control unit 1226, acquires encoded data such as video data, audio data, and EPG data transmitted from another device via the network, and supplies the encoded data to the recorder control unit 1226. To do. For example, the recorder control unit 1226 supplies the encoded data of the acquired video data and audio data to the recording / playback unit 1233 and stores it in the hard disk. At this time, the recorder control unit 1226 and the recording / playback unit 1233 may perform processing such as re-encoding as necessary.

  Also, the recorder control unit 1226 decodes the acquired encoded data of video data and audio data, and supplies the obtained video data to the display converter 1230. Similar to the video data supplied from the video decoder 1225, the display converter 1230 processes the video data supplied from the recorder control unit 1226, supplies the processed video data to the monitor 1260 via the display control unit 1232, and displays the image. .

  In accordance with the image display, the recorder control unit 1226 may supply the decoded audio data to the monitor 1260 via the D / A converter 1234 and output the sound from the speaker.

  Further, the recorder control unit 1226 decodes the encoded data of the acquired EPG data, and supplies the decoded EPG data to the EPG data memory 1227.

  The hard disk recorder 1200 as described above uses the image decoding device 500 as a decoder built in the video decoder 1225, the decoder 1252, and the recorder control unit 1226. That is, the decoder incorporated in the video decoder 1225, the decoder 1252, and the recorder control unit 1226, as in the case of the image decoding device 500, selects encoded data that is controlled in units of CU smaller than the LCU. Decode properly. Therefore, the video decoder 1225, the decoder 1252, and the decoder incorporated in the recorder control unit 1226 can realize reduction of redundant processing at the time of encoding and reduction of redundant information included in the encoded data. Thereby, the decoder incorporated in the video decoder 1225, the decoder 1252, and the recorder control part 1226 can implement | achieve improving encoding efficiency, suppressing the reduction of the efficiency of an encoding process.

  Therefore, the hard disk recorder 1200, for example, the efficiency of encoding processing when generating video data (encoded data) received by the tuner or communication unit 1235 or video data (encoded data) reproduced by the recording / reproducing unit 1233. It is possible to improve the encoding efficiency of the data while suppressing the reduction of the data.

  The hard disk recorder 1200 uses the image encoding device 300 as the encoder 1251. Therefore, the encoder 1251 controls the selection of the I_PCM mode in units of CUs smaller than the LCU, as in the case of the image encoding device 300. That is, the encoder 1251 can further reduce redundant processing at the time of encoding, and can further reduce redundant information included in the encoded data. Thereby, the encoder 1251 can improve encoding efficiency, suppressing the reduction of the efficiency of an encoding process.

  Therefore, the hard disk recorder 1200 can improve the encoding efficiency while suppressing the reduction of the efficiency of the encoding process, for example, when generating the encoded data to be recorded on the hard disk.

  In the above description, the hard disk recorder 1200 for recording video data and audio data on the hard disk has been described. Of course, any recording medium may be used. For example, even in a recorder to which a recording medium other than a hard disk, such as a flash memory, an optical disk, or a video tape, is applied, the image encoding device 300 and the image decoding device 500 of the present embodiment are provided as in the case of the hard disk recorder 1200 described above. Can be applied.

<7. Seventh Embodiment>
[camera]
FIG. 25 is a block diagram illustrating a main configuration example of a camera using the image encoding device 300 and the image decoding device 500 according to the present embodiment.

  A camera 1300 shown in FIG. 25 captures a subject and displays an image of the subject on the LCD 1316 or records it on the recording medium 1333 as image data.

  The lens block 1311 causes light (that is, an image of the subject) to enter the CCD / CMOS 1312. The CCD / CMOS 1312 is an image sensor using CCD or CMOS, converts the intensity of received light into an electric signal, and supplies it to the camera signal processing unit 1313.

  The camera signal processing unit 1313 converts the electrical signal supplied from the CCD / CMOS 1312 into Y, Cr, and Cb color difference signals, and supplies them to the image signal processing unit 1314. The image signal processing unit 1314 performs predetermined image processing on the image signal supplied from the camera signal processing unit 1313 or encodes the image signal with the encoder 1341 under the control of the controller 1321. The image signal processing unit 1314 supplies encoded data generated by encoding the image signal to the decoder 1315. Further, the image signal processing unit 1314 acquires display data generated in the on-screen display (OSD) 1320 and supplies it to the decoder 1315.

  In the above processing, the camera signal processing unit 1313 appropriately uses a DRAM (Dynamic Random Access Memory) 1318 connected via the bus 1317, and image data or a code obtained by encoding the image data as necessary. The digitized data or the like is held in the DRAM 1318.

  The decoder 1315 decodes the encoded data supplied from the image signal processing unit 1314 and supplies the obtained image data (decoded image data) to the LCD 1316. In addition, the decoder 1315 supplies the display data supplied from the image signal processing unit 1314 to the LCD 1316. The LCD 1316 appropriately combines the image of the decoded image data supplied from the decoder 1315 and the image of the display data, and displays the combined image.

  Under the control of the controller 1321, the on-screen display 1320 outputs display data such as menu screens and icons composed of symbols, characters, or graphics to the image signal processing unit 1314 via the bus 1317.

  The controller 1321 executes various processes based on a signal indicating the content instructed by the user using the operation unit 1322, and also via the bus 1317, an image signal processing unit 1314, a DRAM 1318, an external interface 1319, an on-screen display. 1320, media drive 1323, and the like are controlled. The FLASH ROM 1324 stores programs and data necessary for the controller 1321 to execute various processes.

  For example, the controller 1321 can encode the image data stored in the DRAM 1318 or decode the encoded data stored in the DRAM 1318 instead of the image signal processing unit 1314 and the decoder 1315. At this time, the controller 1321 may be configured to perform encoding / decoding processing by a method similar to the encoding / decoding method of the image signal processing unit 1314 or the decoder 1315, or the image signal processing unit 1314 or the decoder 1315 is compatible. The encoding / decoding process may be performed by a method that is not performed.

  For example, when the start of image printing is instructed from the operation unit 1322, the controller 1321 reads out image data from the DRAM 1318 and supplies it to the printer 1334 connected to the external interface 1319 via the bus 1317. Let it print.

  Further, for example, when image recording is instructed from the operation unit 1322, the controller 1321 reads the encoded data from the DRAM 1318 and supplies it to the recording medium 1333 mounted on the media drive 1323 via the bus 1317. Remember.

  The recording medium 1333 is an arbitrary readable / writable removable medium such as a magnetic disk, a magneto-optical disk, an optical disk, or a semiconductor memory. Of course, the recording medium 1333 may be of any kind as a removable medium, and may be a tape device, a disk, or a memory card. Of course, a non-contact IC card or the like may be used.

  Further, the media drive 1323 and the recording medium 1333 may be integrated and configured by a non-portable storage medium such as a built-in hard disk drive or SSD (Solid State Drive).

  The external interface 1319 is composed of, for example, a USB input / output terminal, and is connected to the printer 1334 when printing an image. In addition, a drive 1331 is connected to the external interface 1319 as necessary, and a removable medium 1332 such as a magnetic disk, an optical disk, or a magneto-optical disk is appropriately mounted, and a computer program read from them is loaded as necessary. Installed in the FLASH ROM 1324.

  Furthermore, the external interface 1319 has a network interface connected to a predetermined network such as a LAN or the Internet. For example, the controller 1321 can read the encoded data from the DRAM 1318 in accordance with an instruction from the operation unit 1322 and supply the encoded data to the other device connected via the network from the external interface 1319. In addition, the controller 1321 acquires encoded data and image data supplied from another device via the network via the external interface 1319, holds the data in the DRAM 1318, or supplies it to the image signal processing unit 1314. Can be.

  The camera 1300 as described above uses the image decoding device 500 as the decoder 1315. That is, as in the case of the image decoding apparatus 500, the decoder 1315 appropriately decodes the encoded data in which the selection of the I_PCM mode is controlled in units of CU smaller than the LCU. Therefore, the decoder 1315 can realize reduction of redundant processing at the time of encoding and reduction of redundant information included in the encoded data. Thereby, the decoder 1315 can implement | achieve improving encoding efficiency, suppressing the reduction | decrease in the efficiency of an encoding process.

  Accordingly, the camera 1300 generates, for example, image data generated in the CCD / CMOS 1312, encoded data of video data read from the DRAM 1318 or the recording medium 1333, or encoded data of video data acquired via the network. It is possible to improve the encoding efficiency of the data while suppressing the reduction in the efficiency of the encoding process.

  The camera 1300 uses the image encoding device 300 as the encoder 1341. The encoder 1341 controls the selection of the I_PCM mode in units of CU smaller than the LCU, as in the case of the image encoding device 300. That is, the encoder 1341 can further reduce redundant processing at the time of encoding, and can further reduce redundant information included in the encoded data. Thereby, the encoder 1341 can improve encoding efficiency, suppressing the reduction of the efficiency of an encoding process.

  Accordingly, the camera 1300, for example, can generate encoding efficiency while suppressing reduction in the efficiency of encoding processing when generating encoded data to be recorded on the DRAM 1318 or the recording medium 1333 or encoded data to be provided to other devices. Can be improved.

  Note that the decoding method of the image decoding device 500 may be applied to the decoding process performed by the controller 1321. Similarly, the encoding method of the image encoding device 300 may be applied to the encoding process performed by the controller 1321.

  The image data captured by the camera 1300 may be a moving image or a still image.

  Of course, the image encoding apparatus 300 and the image decoding apparatus 500 of the present embodiment can be applied to apparatuses and systems other than the apparatuses described above.

  The present invention, for example, MPEG, H.26x, etc., image information (bit stream) compressed by orthogonal transformation such as discrete cosine transformation and motion compensation, such as satellite broadcasting, cable TV, the Internet, mobile phones, etc. The present invention can be applied to an image encoding device and an image decoding device that are used when receiving via a network medium or when processing on a storage medium such as an optical, magnetic disk, or flash memory.

  300 image encoding device, 303 adaptive left shift unit, 307 lossless encoding unit, 312 loop filter, 313 adaptive right shift unit, 315 adaptive left shift unit, 320 rate control unit, 321 PCM encoding unit, 331 NAL encoding unit , 332 CU encoding unit, 341 I_PCM_flag generation unit, 342 PCM determination unit, 351 deblock filter, 352 pixel sorting unit, 353 filter coefficient calculation unit, 354 filtering unit, 361 input data amount calculation unit, 362 PCM determination unit, 363 Coding control unit, 364 adaptive shift control unit, 365 filter control unit, 500 image decoding device, 502 lossless decoding unit, 506 loop filter, 507 adaptive right shift unit, 511 adaptive left shift unit, 516 PCM decoding unit, 531 NAL decoding Department, 532 CU recovery Department, 541 I_PCM_flag buffer, 542 PCM controller, 551 de-block filter, 552 pixel sorting unit 553 the filtering unit

Claims (20)

Coding unit set as a non-compression mode, which is a coding mode that outputs image data as unencoded data without encoding image data for the encoding unit obtained by recursively dividing the maximum coding unit And an identification unit that is set as a compression mode that is an encoding mode that encodes the image data and outputs the encoded data, and a filter identification that identifies whether a filter process performed after the deblock filter process is performed information and the encoded data including the identification information identifying whether uncoded data is included in the coding unit, decoding unit for generating image data by decoding for each of the coding unit,
An image processing apparatus comprising: a control unit that controls the filter processing on the image data generated by the decoding unit using the filter identification information and the identification information . The decoding unit acquires the filter identification information and the identification information from the encoded data,
Wherein the control unit uses the said identification information and the filter identification information acquired from the decoding unit, the image processing according to claim 1 for controlling the filtering process on the image data generated by the decoding unit apparatus. The encoded data includes, as the filter identification information, loop filter identification information that identifies whether the deblock filter processing and the filter processing are performed as loop filter processing,
The image processing according to claim 1, wherein the control unit controls the deblocking filter processing and the filtering processing on the image data generated by the decoding unit using the filter identification information and the identification information. apparatus. The decoding unit acquires the filter identification information from the encoded data,
The image according to claim 3, wherein the control unit controls the deblocking filter processing and the filtering processing on the image data generated by the decoding unit using filter identification information acquired from the decoding unit. Processing equipment. A deblocking filter unit that performs the deblocking filter process on the image data generated by the decoding unit;
The image processing apparatus according to claim 1, further comprising: a filter unit that performs the filtering process on the image data on which the deblocking filter process has been performed by the deblocking filter unit. The image processing apparatus according to claim 5, wherein the filter process is an adaptive loop filter process using a Wiener filter. The image processing apparatus according to claim 1, wherein the coding unit is recursively divided from the maximum coding unit according to a quadtree structure . The maximum coding unit is a coding unit of the uppermost layer in the quadtree
The image processing apparatus according to claim 7 . The maximum coding unit is a fixed-size block in sequence units,
The image processing apparatus according to claim 8, wherein the coding unit is a variable-size block . Coding unit set as a non-compression mode, which is a coding mode that outputs image data as unencoded data without encoding image data for the encoding unit obtained by recursively dividing the maximum coding unit And an identification unit that is set as a compression mode that is an encoding mode that encodes the image data and outputs the encoded data, and a filter identification that identifies whether a filter process performed after the deblock filter process is performed Decoding encoded data including information and identification information for identifying whether the non-encoded data is included in an encoding unit, and generating image data for each encoding unit;
The filter processing for the generated image data is controlled using the filter identification information and the identification information.
Image processing method. Obtaining the filter identification information and the identification information from the encoded data;
Using the acquired filter identification information and the identification information, control the filter processing for the generated image data
The image processing method according to claim 10 . The encoded data includes, as the filter identification information, loop filter identification information that identifies whether the deblock filter processing and the filter processing are performed as loop filter processing ,
The used filter identification information and the identification information, the image processing method according to claim 10 for controlling said filtering and the deblocking filter processing for the generated the image data. Obtaining the filter identification information from the encoded data;
Using the acquired filter identification information, the deblock filter processing and the filter processing for the generated image data are controlled.
The image processing method according to claim 12 . Performing the deblocking filter process on the generated image data;
The image processing method according to claim 10 , further comprising performing the filter processing on the image data on which the deblocking filter processing has been performed . The filter process is an adaptive loop filter process using a Wiener filter.
The image processing method according to claim 14 . The coding unit is recursively divided according to a quadtree structure with the maximum coding unit.
The image processing method according to claim 10 . The maximum coding unit is a coding unit of the highest layer in the quadtree structure.
The image processing method according to claim 16 . The maximum coding unit is a fixed-size block in sequence units,
The coding unit is a variable-size block.
The image processing method according to claim 17 . Computer
Coding unit set as a non-compression mode, which is a coding mode that outputs image data as unencoded data without encoding image data for the encoding unit obtained by recursively dividing the maximum coding unit And an identification unit that is set as a compression mode that is an encoding mode that encodes the image data and outputs the encoded data, and a filter identification that identifies whether a filter process performed after the deblock filter process is performed A decoding unit that decodes encoded data including information and identification information that identifies whether the non-encoded data is included in an encoding unit, and generates image data for each encoding unit;
A control unit that controls the filtering process on the image data generated by the decoding unit using the filter identification information and the identification information;
Program to make it work. Computer
Coding unit set as a non-compression mode, which is a coding mode that outputs image data as unencoded data without encoding image data for the encoding unit obtained by recursively dividing the maximum coding unit And an identification unit that is set as a compression mode that is an encoding mode that encodes the image data and outputs the encoded data, and a filter identification that identifies whether a filter process performed after the deblock filter process is performed A decoding unit that decodes encoded data including information and identification information that identifies whether the non-encoded data is included in an encoding unit, and generates image data for each encoding unit;
A control unit that controls the filtering process on the image data generated by the decoding unit using the filter identification information and the identification information;
The computer-readable recording medium which recorded the program for functioning as a computer.

Images