US20130128972A1 - Image encoding apparatus and image decoding apparatus - Google Patents

Image encoding apparatus and image decoding apparatus Download PDF

Info

Publication number
US20130128972A1
US20130128972A1 US13/740,841 US201313740841A US2013128972A1 US 20130128972 A1 US20130128972 A1 US 20130128972A1 US 201313740841 A US201313740841 A US 201313740841A US 2013128972 A1 US2013128972 A1 US 2013128972A1
Authority
US
United States
Prior art keywords
transform
prediction
orthogonal transformation
transform matrix
intra
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/740,841
Other languages
English (en)
Inventor
Jun Yamaguchi
Akiyuki Tanizawa
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Toshiba Corp
Original Assignee
Toshiba Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Toshiba Corp filed Critical Toshiba Corp
Assigned to KABUSHIKI KAISHA TOSHIBA reassignment KABUSHIKI KAISHA TOSHIBA ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: TANIZAWA, AKIYUKI, YAMAGUCHI, JUN
Publication of US20130128972A1 publication Critical patent/US20130128972A1/en
Priority to US14/186,959 priority Critical patent/US9706226B2/en
Priority to US14/186,990 priority patent/US9426467B2/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • H04N19/00793—
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/44—Decoders specially adapted therefor, e.g. video decoders which are asymmetric with respect to the encoder
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/11—Selection of coding mode or of prediction mode among a plurality of spatial predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/186—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a colour or a chrominance component

Definitions

  • Embodiments described herein relate generally to orthogonal transformation and inverse orthogonal transformation in encoding and decoding of videos.
  • H.264 carries out discrete cosine transformation (DCT) and inverse discrete cosine transformation (IDCT) as orthogonal transformation and inverse orthogonal transformation on prediction errors in a target pixel block regardless of a prediction scheme applied to the target pixel block.
  • DCT discrete cosine transformation
  • IDCT inverse discrete cosine transformation
  • Extended H.264 is expected to carry out orthogonal transformation and inverse orthogonal transformation using individual transform bases for the respective nine types of prediction modes specified for intra-picture prediction (intra-prediction), thus improving the coding efficiency.
  • Software implementation enables not only DCT matrices but also individual transform matrices for the respective types of prediction directions to be loaded from a memory as appropriate and held in a cache memory as appropriate.
  • the desired orthogonal transformation and inverse orthogonal transformation can be implemented by a general-purpose multiplier.
  • costs increase as a result of an increase in memory bandwidth or in cache memory size.
  • FIG. 1 is a block diagram illustrating an image encoding apparatus according to a first embodiment.
  • FIG. 2 is a block diagram illustrating an orthogonal transformation unit according to the first embodiment.
  • FIG. 3 is a block diagram illustrating an inverse orthogonal transformation unit according to the first embodiment.
  • FIG. 4A is a table illustrating the correspondence between prediction modes and a vertical transform index and a horizontal transform index according to the first embodiment.
  • FIG. 4B is a table illustrating the correspondence between the vertical transform index and 1D transform matrices according to the first embodiment.
  • FIG. 4C is a table illustrating the correspondence between the horizontal transform index and 1D transform matrices according to the first embodiment.
  • FIG. 4D is a table illustrating the correspondence between a transform index and the vertical transform index and the horizontal transform index according to the first embodiment.
  • FIG. 4E is a table obtained by integrating FIG. 4A with FIG. 4D .
  • FIG. 5A is a block diagram illustrating a coefficient order control unit according to the first embodiment.
  • FIG. 5B is a block diagram illustrating the coefficient order control unit according to the first embodiment.
  • FIG. 6A is a diagram illustrating the order of predictive coding of a pixel block.
  • FIG. 6B is a diagram illustrating an example of a pixel block size.
  • FIG. 6C is a diagram illustrating another example of the pixel block size.
  • FIG. 6D is a diagram illustrating another example of the pixel block size.
  • FIG. 7A is a diagram illustrating an intra-prediction mode.
  • FIG. 7B is a diagram illustrating the arrangement relation between prediction target pixels and reference pixels.
  • FIG. 7C is a diagram illustrating an intra-prediction mode 1.
  • FIG. 7D is a diagram illustrating an intra-prediction mode 4.
  • FIG. 8A is a diagram illustrating a zigzag scan.
  • FIG. 8B is a diagram illustrating the zigzag scan.
  • FIG. 8C is a table showing a 2D-1D transformation utilizing the zigzag scan.
  • FIG. 9 is a table illustrating individual 2D-1D transformations for the respective prediction modes.
  • FIG. 10A is a flowchart illustrating processing carried out on a coding target block by the image encoding apparatus in FIG. 1 .
  • FIG. 10B is a flowchart illustrating the processing carried out on the coding target block by the image encoding apparatus in FIG. 1 .
  • FIG. 11 is a diagram illustrating a syntax structure.
  • FIG. 12 is a diagram illustrating slice header syntax.
  • FIG. 13 is a diagram illustrating coding tree unit syntax.
  • FIG. 14 is a diagram illustrating transform unit syntax.
  • FIG. 15 is a block diagram illustrating an orthogonal transformation unit which carries out orthogonal transformation on each of nine types of prediction directions using a particular transform basis.
  • FIG. 16 is a block diagram illustrating an orthogonal transformation unit according to a second embodiment.
  • FIG. 17 is a block diagram illustrating an inverse orthogonal transformation unit according to the second embodiment.
  • FIG. 18A is a table illustrating the correspondence between the prediction modes and the vertical transform index and the horizontal transform index according to the second embodiment.
  • FIG. 18B is a table illustrating the correspondence between the vertical transform index and 1D transform matrices according to the second embodiment.
  • FIG. 18C is a table illustrating the correspondence between the horizontal transform index and 1D transform matrices according to the second embodiment.
  • FIG. 18D is a table illustrating the correspondence between the transform index and the vertical transform index and the horizontal transform index according to the second embodiment.
  • FIG. 18E is a table obtained by integrating FIG. 18A with FIG. 18D .
  • FIG. 19 is a block diagram illustrating an orthogonal transformation unit according to a third embodiment.
  • FIG. 20 is a block diagram illustrating an inverse orthogonal transformation unit according to the third embodiment.
  • FIG. 21A is a table illustrating the correspondence between the prediction modes and the vertical transform index and the horizontal transform index according to the third embodiment.
  • FIG. 21B is a table illustrating the correspondence between the vertical transform index and 1D transform matrices according to the third embodiment.
  • FIG. 21C is a table illustrating the correspondence between the horizontal transform index and 1D transform matrices according to the third embodiment.
  • FIG. 21D is a table illustrating the correspondence between the transform index and the vertical transform index and the horizontal transform index according to the third embodiment.
  • FIG. 21E is a table obtained by integrating FIG. 21A with FIG. 21D .
  • FIG. 22 is a block diagram illustrating an image decoding apparatus according to a fourth embodiment.
  • FIG. 23A is a block diagram illustrating a coefficient order control unit according to the fourth embodiment.
  • FIG. 23B is a block diagram illustrating a coefficient order control unit according to the fourth embodiment.
  • an image encoding apparatus includes an intra-prediction unit, a setting unit, a first transforming unit and an encoding unit.
  • the intra-prediction unit predicts a prediction error of a target image to be encoded based on an intra-prediction mode.
  • the setting unit sets a combination of a vertical transform matrix and a horizontal transform matrix corresponding to the target image based on a predetermined relation.
  • the first transforming unit transforms the prediction error using the combination of the vertical transform matrix and the horizontal transform matrix, to obtain transform coefficients.
  • the encoding unit encodes the transform coefficients and information indicating the intra-prediction mode for the target image.
  • the combination includes any of a plurality of transform matrices including a first transform matrix and a second transform matrix which increases a coefficient density compared to the first transform matrix if a one-dimensional orthogonal transformation in a direction orthogonal to a line of a group of reference pixels on at least one line is performed on the prediction error in the intra-prediction mode in which the group of reference pixels is referenced to generate an intra-prediction image.
  • image may be replaced with the term “image signal”, “image data”, or the like as appropriate.
  • a first embodiment relates to an image encoding apparatus.
  • An image decoding apparatus corresponding to the image encoding apparatus according to the present embodiment will be described in a fourth embodiment.
  • the image encoding apparatus can be implemented by hardware such as LSI (Large-Scale Integration) chips, DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), and the like.
  • the image encoding apparatus can also be implemented by allowing a computer to execute an image encoding program.
  • the image encoding apparatus includes a subtraction unit 101 , an orthogonal transformation unit 102 , a quantization unit 103 , a de-quantization unit 104 , an inverse orthogonal transformation unit 105 , an addition unit 106 , a reference image memory 107 , an intra-prediction unit 108 , an inter-prediction unit 109 , a prediction selection unit 110 , a prediction selection switch 111 , a 1D (one-dimensional) transformation matrix set unit 112 , a coefficient order control unit 113 , an entropy encoding unit 114 , an output buffer 115 , and an encoding control unit 116 .
  • the image encoding apparatus in FIG. 1 divides each of the frame or fields forming an input image 118 into a plurality of pixel blocks, carries out predictive coding on the pixel blocks resulting from the division, and outputs encoded data 130 .
  • predictive coding is hereinafter assumed to be carried out on a pixel block from upper left to lower right as shown in FIG. 6A .
  • FIG. 6A shows a coding target frame f in which an encoded pixel blocks p are positioned to the left of and above a coding target pixel block c.
  • the pixel block refers to, for example, a coding tree unit, a macro block, a sub-block, or a pixel.
  • the pixel block is hereinafter basically used to mean a coding tree unit but may be interpreted to have another meaning as appropriate.
  • the coding tree unit is typically, for example, a 16 ⁇ 16 pixel block shown in FIG. 6B but may be a 32 ⁇ 32 pixel block shown in FIG. 6C , a 64 ⁇ 64 pixel block shown in FIG. 6D , or an 8 ⁇ 8 pixel block or a 4 ⁇ 4 pixel block not shown in the drawings.
  • the coding tree unit need not necessarily be square.
  • a coding target block or coding tree unit in the input image 118 is hereinafter sometimes referred to as a “prediction target block”.
  • the coding unit is not limited to the pixel block such as the coding tree unit but may be a frame, a field, or a combination thereof.
  • the image encoding apparatus in FIG. 1 carries out intra-prediction (also referred to as intra-frame, intra-frame prediction, or the like) or inter-prediction (also referred to as inter-picture prediction, inter-frame prediction, or the like) to generate a predicted image 127 .
  • the image encoding apparatus orthogonally transforms and quantizes a prediction error 119 between the pixel block (input image 118 ) and the predicted image 127 .
  • the image encoding apparatus then carries out entropy encoding on the transformed and quantized prediction error to generate and output encoded data 130 .
  • the image encoding apparatus in FIG. 1 carries out encoding by selective application of a plurality of prediction modes involving different block sizes and different methods for generating the predicted image 127 .
  • the method for generating the predicted image 127 is roughly classified into two types: intra-prediction which carries out prediction within the coding target frame and inter-prediction which carries out prediction using one or more reference frames temporally different from one another.
  • intra-prediction which carries out prediction within the coding target frame
  • inter-prediction which carries out prediction using one or more reference frames temporally different from one another.
  • orthogonal transformation and inverse orthogonal transformation carried out to generate a predicted image using intra-prediction will be described in detail.
  • the subtractor 101 subtracts the corresponding predicted image 127 from the coding target block in the input image 118 to obtain the prediction error 119 .
  • the subtractor 101 inputs the prediction error 119 to the orthogonal transformation unit 102 .
  • the orthogonal transformation unit 102 orthogonally transforms the prediction error 119 from the subtractor 101 to obtain transform coefficients 120 .
  • the orthogonal transformation unit 102 will be described below in detail.
  • the orthogonal transformation unit 102 inputs the transform coefficients 120 to the quantization unit 103 .
  • the quantization unit 103 quantizes the transform coefficients from the orthogonal transformation unit 102 to obtain quantized transform coefficients 121 .
  • the quantization unit 103 carries out quantization in accordance with quantization information such as a quantization parameter and a quantization matrix which is specified by the encoding control unit 116 .
  • the quantization parameter is indicative of the resolution of the quantization.
  • the quantization matrix is used to weight the resolution of the quantization for each component of the transform coefficients.
  • the quantization unit 103 inputs the quantized transform coefficients 121 to the coefficient order control unit 113 and the de-quantization unit 104 .
  • the coefficient order control unit 113 transforms the quantized transform coefficients 121 which are a two-dimensional (2D) expression into a quantized transform coefficient sequence 117 which is a one-dimensional (1D) expression.
  • the coefficient order control unit 113 then inputs the quantized transform coefficient sequence 117 to the entropy encoding unit 114 .
  • the coefficient control unit 113 will be described below in detail.
  • the entropy encoding unit 114 carries out entropy encoding (for example, Huffman encoding or arithmetic encoding) on various coding parameters such as the quantized transform coefficient sequence 117 from the coefficient control unit 113 , the prediction information 126 from the prediction selection unit 110 , and the quantization information which are specified by the encoding control unit 116 .
  • the coding parameters are required for decoding and include the prediction information 126 , information on the transform coefficients, and information on the quantization.
  • the coding parameters are held in an internal memory (not shown in the drawings) in the encoding control unit 116 .
  • the coding parameters for the already coded adjacent pixel block can be used. For example, H.264 intra-prediction enables a predicted value for the prediction mode of the prediction target block to be derived from the prediction mode information on the coded adjacent block.
  • the encoded data generated by the entropy encoding unit 114 is, for example, multiplexed and then temporarily accumulated in the output buffer 115 .
  • the data is then output as the encoded data 130 in accordance with an appropriate output timing managed by the encoding control unit 116 .
  • the encoded data 130 is output to, for example, an accumulation system (accumulation medium) or a transmission system (communication line) which is not shown in the drawings.
  • the de-quantization unit 104 de-quantizes the quantized transform coefficients 121 from the quantization unit 103 to obtain a restored transform coefficients 122 . Specifically, the de-quantization unit 104 carries out de-quantization in accordance with the quantization information used in the quantization unit 103 . The quantization information used in the quantization unit 103 is loaded from the internal memory in the encoding control unit 116 . The de-quantization unit 104 inputs the restored transform coefficients 122 to the inverse orthogonal transformation unit 105 .
  • the inverse orthogonal transformation unit 105 carries out an inverse orthogonal transformation corresponding to the orthogonal transformation performed by the orthogonal transformation unit 102 on the restored transform coefficients 122 from the de-quantization unit 104 to obtain a restored prediction error 123 .
  • the inverse orthogonal transformation unit 105 will be described below in detail.
  • the inverse orthogonal transformation unit 105 inputs the restored prediction error 123 to the addition unit 106 .
  • the addition unit 106 adds the restored prediction error 123 and the corresponding predicted image 127 together to generate a local decoded image 124 .
  • the local decoded image 124 is saved to the reference image memory 107 .
  • the local decoded image 124 saved to the reference image memory 107 is referenced by the intra-prediction unit 108 and the inter-prediction unit 109 as a reference image 125 as necessary.
  • the intra-prediction unit 108 carries out intra-prediction utilizing the reference image 125 saved to the reference image memory 107 .
  • H.264 utilizes the coded reference pixel value for the block adjacent to the prediction target bock to compensate for pixels (copy or interpolate pixels) along a prediction direction such as the vertical direction or the horizontal direction to generate an intra-predicted image.
  • FIG. 7A shows prediction directions for the intra-prediction according to H.264.
  • FIG. 7B shows the arrangement relation between reference pixels and coding target pixels according to H.264.
  • FIG. 7C shows a method for generating a predicted image in a mode 1 (horizontal prediction).
  • FIG. 7D shows a method for generating a predicted image in a mode 4 (diagonal down-right prediction; Intra_N ⁇ N_Diagonal_Down_Right in FIG. 4A ).
  • the intra-prediction unit 108 may interpolate pixel values using a predetermined interpolation method and then copy the interpolated pixel values in a predetermined prediction direction.
  • the prediction directions for the intra-prediction according to H.264 are illustrated, but any number of prediction modes such as 17 or 33 types of prediction modes are made available by specifying a more detailed classification of prediction directions.
  • H.264 defines prediction angles at intervals of 22.5 degrees, but 17 types of prediction modes including DC prediction are made available by specifying prediction angles at intervals of 11.25 degrees.
  • 33 types of prediction modes including DC prediction are made available by specifying prediction angles at intervals of 5.625 degrees.
  • angles of the prediction directions may each be expressed by a straight line which joins a first reference point to a second reference point obtained by moving the first reference point in the horizontal direction and in the vertical direction.
  • the number of prediction modes can be easily increased, and the present embodiment is applicable regardless of the number of prediction modes.
  • the inter-prediction unit 109 carries out inter-prediction utilizing the reference image 125 saved to the reference image memory 107 . Specifically, the inter-prediction unit 109 carries out a block matching process between the prediction target block and the reference image 125 to derive the amount of deviation in motion (motion vector). The inter-prediction unit 109 carries out an interpolation process (motion compensation) based on the motion vector to generate an inter-predicted image. H.264 enables the interpolation process to be achieved to an accuracy of a 1 ⁇ 4 pixel.
  • the derived motion vector is subjected to entropy encoding as a part of the prediction information 126 .
  • the selection switch 111 selects an output end of the intra-prediction unit 108 or an output end of the inter-prediction unit 109 in accordance with the prediction information 126 from the prediction selection unit 110 .
  • the selection switch 111 then inputs an intra-predicted image or an inter-predicted image to the subtraction unit 101 and the addition unit 106 as the predicted image 127 . If the prediction information 126 is indicative of intra-prediction, the selection switch 110 acquires the intra-predicted image from the intra-prediction unit 108 as a predicted image 127 . On the other hand, if the prediction information 126 is indicative of inter-prediction, the selection switch 110 acquires the inter-predicted image from the inter-prediction unit 109 as a predicted image 127 .
  • the prediction selection unit 110 has a function to set the prediction information 126 in accordance with the prediction mode controlled by the encoding control unit 116 .
  • the intra-prediction or the inter-prediction can be selected for generation of the predicted image 127 .
  • a plurality of modes can further be selected for each of the intra-prediction or the inter-prediction.
  • the encoding control unit 116 determines one of the plurality of prediction modes for the intra-prediction and the inter-prediction to be the optimum prediction mode.
  • the prediction selection unit 110 sets the prediction information 126 according to the determined optimum prediction mode.
  • the prediction mode information from the encoding control unit 116 is specified in the intra-prediction unit 108 .
  • the intra-prediction unit 108 generates the predicted image 127 .
  • the encoding control unit 116 may specify a plurality of pieces of prediction mode information in order of increasing number of the prediction mode or decreasing number of the prediction mode.
  • the encoding control unit 116 may limit the prediction mode in accordance with the characteristics of the input image.
  • the encoding control unit 116 need not specify all the prediction modes but may specify at least one piece of prediction mode information for the coding target block.
  • the encoding control unit 116 determines the optimum prediction mode using a cost function shown in:
  • OH denotes the amount of code for the prediction information 126 (for example, motion vector information and prediction block size information)
  • SAD denotes the sum of absolute difference between the prediction target block and the predicted image 127 (that is, the accumulated sum of the absolute values of the prediction error 119 ).
  • ⁇ denotes a Lagrange multiplier determined based on the value of the quantization information (quantization parameter)
  • K denotes a coding cost. If Expression (1) is used, the prediction mode which minimizes the coding cost K is determined to be optimum in terms of the amount of generated code and prediction errors.
  • the coding cost may be evaluated only from the OH or the SAD or by utilizing a value obtained by carrying out an Hadamard transformation on the SAD or a value approximate thereto.
  • the optimum prediction mode can be determined using a provisionally encoding unit (not shown in the drawings).
  • the encoding control unit 116 determines the optimum prediction mode using a cost function shown in:
  • D denotes the sum of square differences (that is, coding distortion) between the prediction target block and a local decoded image
  • R denotes the amount of code estimated by provisionally encoding the predicted error between the prediction target block and the predicted image 127 for the prediction mode
  • J denotes the coding cost.
  • a provisionally encoding process and a local decoding process need to be carried out in each prediction mode. This increases the scale of relevant circuits or the amount of calculation.
  • the coding cost J is derived based on a more accurate coding distortion and a more accurate amount of code. Therefore, the optimum prediction mode is accurately determined to allow a high coding efficiency to be easily maintained.
  • the coding cost may be evaluated only from the R or the D or by utilizing a value approximate to the R or the D.
  • the encoding control unit 116 may, in advance, narrow down the number of candidates for the prediction mode one of which is determined using Expression (1) or Expression (2), based on information pre-obtained for the prediction target block (prediction modes for the surrounding pixel blocks, the results of image analysis, and the like).
  • the encoding control unit 116 controls the components of the image encoding apparatus in FIG. 1 . Specifically, the encoding control unit 116 performs various control operations for an encoding process including the above-described operations.
  • the 1D transform matrix set unit 112 generates 1D transform matrix set information 129 based on the prediction mode information included in the prediction information 126 from the prediction selection unit 110 .
  • the 1D transform matrix set unit 112 then inputs the 1D transform matrix set information 129 to the orthogonal transformation unit 102 and the inverse orthogonal transformation unit 105 .
  • the 1D transform matrix set information 129 will be described below in detail.
  • the orthogonal transformation unit 102 according to the present embodiment will be described below in detail with reference to FIG. 2 .
  • the orthogonal transformation unit 102 includes a selection switch 201 , a vertical transformation unit 202 , a transposition unit 203 , a selection switch 204 , and a horizontal transformation unit 205 .
  • the vertical transformation unit 202 includes a 1D orthogonal transformation unit A 206 and a 1D orthogonal transformation unit B 207 .
  • the horizontal transformation unit 205 includes a 1D orthogonal transformation unit A 208 and a 1D orthogonal transformation unit B 209 .
  • the order of the vertical transformation unit 202 and the horizontal transformation unit 205 is illustrative and may be reversed.
  • the 1D orthogonal transformation unit A 206 and the 1D orthogonal transformation unit A 208 have common functions in that both units multiply an input matrix by a 1D transform matrix A.
  • the 1D orthogonal transformation unit B 207 and the 1D orthogonal transformation unit B 209 have common functions in that both units multiply the input matrix by a 1D transform matrix B.
  • the 1D orthogonal transformation unit A 206 and the 1D orthogonal transformation unit A 208 can also be implemented by using physically the same hardware in a time division manner. This also applies to the 1D orthogonal transformation unit B 207 and the 1D orthogonal transformation unit B 209 .
  • the selection switch 201 leads the prediction error 119 to one of the 1D orthogonal transformation unit A 206 and the 1D orthogonal transformation unit B 207 in accordance with a vertical transform index included in the 1D transform matrix set information 129 .
  • the 1D orthogonal transformation unit A 206 multiplies the input prediction error (matrix) 119 by a 1D transform matrix A and outputs the product.
  • the 1D orthogonal transformation unit B 207 multiplies the input prediction error 119 by a 1D transform matrix B and outputs the product.
  • the 1D orthogonal transformation unit A 206 and the 1D orthogonal transformation unit B 207 (that is, the vertical transformation unit 202 ) carries out a one-dimensional orthogonal transformation shown in Expression (3) to eliminate a vertical correlation in the prediction error 119 .
  • X denotes a matrix (N ⁇ N) of the prediction error 119
  • V comprehensively denotes the 1D transform matrix A and the 1D transform matrix B (both are N ⁇ N matrices)
  • Y denotes an output matrix (N ⁇ N) from each of the 1D orthogonal transformation unit A 206 and the 1D orthogonal transformation unit B 207 .
  • the transform matrix V is an N ⁇ N transform matrix in which a transform basis designed to eliminate the vertical correlation in the matrix X is vertically as row vectors.
  • the 1D transform matrix A and the 1D transform matrix B are designed in different manners and have different types of nature.
  • the 1D transform matrix A and the 1D transform matrix B may use integers obtained by the designed transform basis subjected to scalar multiplication.
  • the size of the block to be orthogonally transformed may also be M ⁇ N.
  • the transposition unit 203 transposes the output matrix (Y) from the vertical transformation unit 202 and provides the transposed output matrix (Y) to the selection switch 204 .
  • the transposition unit 203 is illustrative, and the corresponding hardware need not necessarily be prepared.
  • the output matrix (Y) can be transposed without the need to prepare the hardware corresponding to the transposition unit 203 by saving the results of a 1D orthogonal transformation carried out by the vertical transformation unit 202 (each of the elements of the output matrix from the vertical transformation unit 202 ) and loading the results in an appropriate order when the horizontal transformation unit 205 performs a 1D orthogonal transformation.
  • the selection switch 204 leads the input matrix from the transposition unit 203 to one of the 1D orthogonal transformation unit A 208 and the 1D orthogonal transformation unit B 209 in accordance with a horizontal transform index included in the 1D transform matrix set information 129 .
  • the 1D orthogonal transformation unit A 208 multiplies the input matrix by the 1D transform matrix A and outputs the product.
  • the 1D orthogonal transformation unit B 209 multiplies the input matrix by the 1D transform matrix B and outputs the product.
  • the 10 orthogonal transformation unit A 208 and the 1D orthogonal transformation unit B 209 (that is, the horizontal transformation unit 205 ) carries out a one-dimensional orthogonal transformation shown in Expression (4) to eliminate a horizontal correlation in the prediction error.
  • H comprehensively denotes the 1D transform matrix A and the 1D transform matrix B (both are N ⁇ N matrices), and Z denotes an output matrix (N ⁇ N) from each of the 1D orthogonal transformation unit A 208 and the 1D orthogonal transformation unit B 209 ; the output matrix is indicative of the transform coefficients 120 .
  • the transform matrix H is an N ⁇ N transform matrix in which a transform basis designed to eliminate the horizontal correlation in the matrix Y is vertically arranged as row vectors.
  • the 1D transform matrix A and the 1D transform matrix B are designed in different manners and have different types of nature.
  • the 1D transform matrix A and the 1D transform matrix B may use integers obtained by the designed transform basis subjected to scalar multiplication.
  • the orthogonal transformation unit 102 carries out an orthogonal transformation in accordance with the 1D transform matrix set information 129 input from the 1D transform matrix set unit 112 , on the prediction error (matrix) 119 to generate the transform coefficients (matrix) 120 .
  • the orthogonal transformation unit 102 may include a DCT unit (not shown in the drawings) or one of the 1D transform matrix A and the 1D transform matrix B may be replaced with a matrix for DCT.
  • the 1D transform matrix B may be a transform matrix for DCT.
  • the orthogonal transformation unit 102 may implement, in addition to DCT, various orthogonal transformations such as a Hadamard transformation, Karhunen Loeve transformation described below, and discrete sine transformation.
  • Some intra-prediction modes supported by H.264 and the like generate a predicted image by copying, along a prediction direction, a group of reference pixels on one or both of adjacent lines located to the left of and above the prediction target block or carrying out similar copying after interpolation. That is, this intra-prediction mode selects at least one reference pixel from the group of reference pixels in accordance with the prediction direction and copies the reference pixel or carries out interpolation using the reference pixels, to generate a predicted image.
  • the intra-prediction mode utilizes the spatial correlation in an image and thus has a prediction accuracy which tends to decrease with increasing distance from the reference pixel.
  • the absolute value of the prediction error is likely to increase consistently with the distance from the reference pixel.
  • the tendency is similarly exhibited regardless of the prediction direction. More specifically, in connection with the intra-prediction modes (for example, the mode 1 and mode 8 in FIG. 7A ) in which only the group of reference pixels on the line located to the left of and adjacent to the prediction target block is referenced (the pixel values of the reference pixels are copied or interpolation is carried out using the reference pixels), the prediction error exhibits the tendency in the horizontal direction. In connection with the prediction modes in which only the group of reference pixels on the line located above and adjacent to the prediction target block is referenced (for example, the mode 0, mode 3, and mode 7 in FIG. 7A ), the prediction error exhibits the tendency in the vertical direction.
  • the intra-prediction modes for example, the mode 1 and mode 8 in FIG. 7A
  • the prediction error exhibits the tendency in the horizontal direction.
  • the prediction modes in which only the group of reference pixels on the line located above and adjacent to the prediction target block is referenced for example, the mode
  • the prediction error exhibits such a tendency in the horizontal direction and vertical direction.
  • the tendency is exhibited in the direction orthogonal to the line of the group of reference pixels utilized to generate a predicted image.
  • the 1D transform matrix A is generated by pre-designing a common transform basis so as to increase, compared to the 1D transform matrix B, a coefficient density after 1D orthogonal transformation (that is, to reduce the rate of nonzero coefficients in the quantized transform coefficients 121 ) in the orthogonal direction (vertical direction or horizontal direction).
  • the 1D transform matrix B is generated by designing a general-purpose transform matrix having no such nature.
  • the general-purpose transformation is DCT.
  • the efficiency of transformation of the prediction error in the intra-prediction and thus the coding efficiency are improved by carrying out a 1D orthogonal transformation in the orthogonal direction using the 1D transform matrix A.
  • the prediction error 119 in the mode 0 exhibits the tendency in the vertical direction but not in the horizontal direction.
  • efficient orthogonal transformation can be achieved by carrying out a 1D orthogonal transformation in the vertical transformation unit 202 using the 1D transform matrix A and carrying out a 1D orthogonal transformation in the horizontal transformation unit 205 using the 1D transform matrix B.
  • the inverse orthogonal transformation unit 105 according to the present embodiment will be described below in detail with reference to FIG. 3 .
  • the inverse orthogonal transformation unit 105 includes a selection switch 301 , a vertical inverse transformation unit 302 , a transposition unit 303 , a selection switch 304 , and a horizontal inverse transformation unit 305 .
  • the vertical inverse transformation unit 302 includes a 1D inverse orthogonal transformation unit A 306 and a 1D inverse orthogonal transformation unit B 307 .
  • the horizontal inverse transformation unit 305 includes a 1D inverse orthogonal transformation unit A 308 and a 1D inverse orthogonal transformation unit B 309 .
  • the order of the vertical inverse transformation unit 302 and the horizontal inverse transformation unit 305 is illustrative and may be reversed.
  • the 1D inverse orthogonal transformation unit A 306 and the 1D inverse orthogonal transformation unit A 308 have common functions in that both units multiply an input matrix by a transposed matrix of the 1D transform matrix A.
  • the 1D inverse orthogonal transformation unit B 307 and the 1D inverse orthogonal transformation unit B 309 have common functions in that both units multiply an input matrix by a transposed matrix of the 1D transform matrix B.
  • the 1D inverse orthogonal transformation unit A 306 and the 1D inverse orthogonal transformation unit A 308 can also be implemented by using physically the same hardware in a time division manner. This also applies to the 1D inverse orthogonal transformation unit B 307 and the 1D inverse orthogonal transformation unit B 309 .
  • the selection switch 301 leads the restored transform coefficients 122 to one of the 1D inverse orthogonal transformation unit A 306 and the 1D inverse orthogonal transformation unit B 307 in accordance with the vertical transform index included in the 1D transform matrix set information 129 .
  • the 1D inverse orthogonal transformation unit A 306 multiplies the input restored transform coefficients 122 (matrix form) by a transposed matrix of the 1D transform matrix A and outputs the product.
  • the 1D inverse orthogonal transformation unit B 307 multiplies the input restored transform coefficients 122 by a transposed matrix of the 1D transform matrix B and outputs the product.
  • the 1D inverse orthogonal transformation unit A 306 and the 1D inverse orthogonal transformation unit B 307 carry out a one-dimensional inverse orthogonal transformation shown in:
  • Z′ denotes a matrix (N ⁇ N) of the restored transform coefficients 122
  • V T comprehensively denotes the transposed matrices of the 1D transform matrix A and the 1D transform matrix B (both are N ⁇ N matrices)
  • Y′ denotes an output matrix (N ⁇ N) from each of the 1D inverse orthogonal transformation unit A 306 and the 1D inverse orthogonal transformation unit B 307 .
  • the transposition unit 303 transposes the output matrix (Y′) from the vertical inverse transformation unit 302 and outputs the transposed output matrix to the selection switch 304 .
  • the transposition unit 303 is illustrative, and the corresponding hardware need not necessarily be prepared.
  • the output matrix (Y′) can be transposed without the need to prepare the hardware corresponding to the transposition unit 303 by saving the results of a 1D inverse orthogonal transformation carried out by the vertical transformation unit 302 (each of the elements of the output matrix from the vertical inverse transformation unit 302 ) and loading the results in an appropriate order when the horizontal inverse transformation unit 305 performs a 1D inverse orthogonal transformation.
  • the selection switch 304 leads the input matrix from the transposition unit 303 to one of the 1D inverse orthogonal transformation unit A 308 and the 1D inverse orthogonal transformation unit B 309 in accordance with a horizontal transform index included in the 1D transform matrix set information 129 .
  • the 1D inverse orthogonal transformation unit A 308 multiplies the input matrix by the transposed matrix of the 10 transform matrix A and outputs the product.
  • the 1D inverse orthogonal transformation unit B 309 multiplies the input matrix by the transposed matrix of the 1D transform matrix B and outputs the product.
  • the 10 inverse orthogonal transformation unit A 308 and the 10 inverse orthogonal transformation unit B 309 carry out a one-dimensional inverse orthogonal transformation shown in:
  • HT comprehensively denotes the transposed matrices of the 1D transform matrix A and the 1D transform matrix B (both are N ⁇ N matrices), and X′ denotes an output matrix (N ⁇ N) from each of the 10 inverse orthogonal transformation unit A 308 and the 1D inverse orthogonal transformation unit B 309 ; the output matrix is indicative of the restored prediction error 123 .
  • the inverse orthogonal transformation unit 105 carries out an orthogonal transformation in accordance with the 1D transform matrix set information 129 input from the 10 transform matrix set unit 112 , on the restored transform coefficients (matrix) 122 to generate the restored prediction error (matrix) 123 .
  • the inverse orthogonal transformation unit 105 may include an IDCT unit (not shown in the drawings) or one of the 1D transform matrix A and the 1D transform matrix B may be replaced with a matrix for DCT.
  • the 1D transform matrix B may be a transform matrix for DCT.
  • the inverse orthogonal transformation unit 105 may implement, in addition to IDCT, inverse orthogonal transformations corresponding to various orthogonal transformations such as an Hadamard transformation, Karhunen Loeve transformation described below, and discrete sine transformation for coordination with the orthogonal transformation unit 102 .
  • the 1D transform matrix set information 129 according to the present embodiment which is generated by the 1D transform matrix set unit 112 will be described below in detail.
  • the 1D transform matrix set information 129 directly or indirectly indicates the vertical transform index for selection of a transform matrix for use in vertical orthogonal transformation and vertical inverse orthogonal transformation and the horizontal transform index for selection of a transform matrix for use in horizontal orthogonal transformation and horizontal inverse orthogonal transformation.
  • the 1D transform matrix set information 129 can be expressed by a transform index (TrasformIdx) shown in FIG. 4D .
  • a transform index shown in FIG. 4D .
  • Reference to the table in FIG. 4D allows a vertical transform index (Vertical Transform Idx) and a horizontal transform index (Horizontal Transform Idx) to be derived from the transform index.
  • a vertical transform index of “0” allows selection of the 1D transform matrix A (1D_Transform_Matrix_A) or the transposed matrix thereof for vertical orthogonal transformation or vertical inverse orthogonal transformation.
  • a vertical transform index of “1” allows selection of the 1D transform matrix B (1D_Transform_Matrix_B) or the transposed matrix thereof for vertical orthogonal transformation or vertical inverse orthogonal transformation.
  • a horizontal transform index of “0” allows selection of the 1D transform matrix A (1D_Transform_Matrix_A) or the transposed matrix thereof for horizontal orthogonal transformation or horizontal inverse orthogonal transformation.
  • a horizontal transform index of “1” allows selection of the 1D transform matrix B (1D_Transform_Matrix_B) or the transposed matrix thereof for horizontal orthogonal transformation or horizontal inverse orthogonal transformation.
  • FIG. 4A illustrates the index for each (intra-) prediction mode (IntraN ⁇ NPredModeIndex), the name thereof (Name of IntraN ⁇ NPredMode), and the corresponding vertical transform index and horizontal transform index.
  • the size of the prediction target block can be expanded to “M ⁇ N” (that is, rectangles other than squares).
  • FIG. 4E is obtained by integrating FIG. 4A and FIG. 4D together and shows the index for each prediction mode, the name of the index, and the corresponding transform index.
  • the 1D transform matrix set unit 112 detects the index of the prediction mode from the prediction mode information included in the prediction information 126 .
  • the 1D transform matrix set unit 112 then generates the corresponding 1D transform matrix set information 129 .
  • the tables shown in FIG. 4A , FIG. 4B , FIG. 4C , FIG. 4D , and FIG. 4E are illustrative.
  • the 1D transform matrix set unit 112 may generate the 1D transform matrix set information 129 while avoiding the use of some or all of the tables.
  • TransformIdx indicative of 0 means that the vertical transform index indicates 0 and that the horizontal transform index indicates 0.
  • These index values also mean that the transposed matrix of the 1D transform matrix A is used for vertical inverse orthogonal transformation and that the transposed matrix of the 1D transform matrix A is used for horizontal inverse orthogonal transformation.
  • TransformIdx indicative of 1 means that the vertical transform index indicates 0 and that the horizontal transform index indicates 1. This means that the 1D transform matrix A is used for vertical orthogonal transformation and that the 1D transform matrix B is used for horizontal orthogonal transformation. These index values also mean that the transposed matrix of the 1D transform matrix A is used for vertical inverse orthogonal transformation and that the transposed matrix of the 1D transform matrix B is used for horizontal inverse orthogonal transformation.
  • TransformIdx indicative of 2 means that the vertical transform index indicates 1 and that the horizontal transform index indicates 0. This means that the 1D transform matrix B is used for vertical orthogonal transformation and that the 1D transform matrix A is used for horizontal orthogonal transformation. These index values also mean that the transposed matrix of the 1D transform matrix B is used for vertical inverse orthogonal transformation and that the transposed matrix of the 1D transform matrix A is used for horizontal inverse orthogonal transformation.
  • TransformIdx indicative of 3 means that the vertical transform index indicates 1 and that the horizontal transform index indicates 1. This means that the 1D transform matrix B is used for vertical orthogonal transformation and that the 1D transform matrix B is used for horizontal orthogonal transformation. These index values also mean that the transposed matrix of the 1D transform matrix B is used for vertical inverse orthogonal transformation and that the transposed matrix of the 1D transform matrix B is used for horizontal inverse orthogonal transformation.
  • the table shown in FIG. 4A assigns the 1D transform matrix set information 129 taking the above-described tendency of each intra-prediction mode into account. That is, 0 is assigned to the vertical transform index for the prediction modes exhibiting the tendency in the vertical direction of the prediction error. 0 is assigned to the horizontal transform index for the prediction modes exhibiting the tendency in the horizontal direction. On the other hand, 1 is assigned to each of the directions not exhibiting the tendency.
  • the vertical and horizontal directions of the prediction modes are classified into two classes depending whether or not the tendency is present and the 1D transform matrix A or the 1D transform matrix B is adaptively applied to each of the vertical and horizontal directions, a higher transform efficiency can be achieved than in a case where fixed orthogonal transformation such as DCT is uniformly applied to the prediction modes.
  • the coefficient order control unit 113 will be described below in detail.
  • the coefficient order control unit 113 transforms the quantized transform coefficients 121 , which are a two-dimensional expression, into the quantized transform coefficient sequence 117 which is a one-dimensional expression by arranging the elements of the quantized transform coefficients 121 according to a predetermined order.
  • the coefficient order control unit 113 can carry out a common 2D-1D transformation regardless of the prediction mode.
  • the coefficient order control unit 113 can utilize a zigzag scan as is the case with H.264.
  • the zigzag scan arranges the elements of the quantized transform coefficients 121 in such an order as shown in FIG. 8A and transforms the elements into such a quantized transform coefficient sequence 117 as shown in FIG. 8B .
  • FIG. 8B (i, j) denotes the coordinates (position information) of each element in the quantized transform coefficients (matrix) 121 .
  • FIG. 8C shows a 2D-1D transformation utilizing the zigzag scan (in the case of a 4 ⁇ 4 pixel block).
  • FIG. 8C shows an index (idx) indicative of the order of coefficients (the order of scans) in the quantized transform coefficient sequence 117 subjected to the 2D-1D transformation utilizing the zigzag scan, and the corresponding elements (cij) of the quantized transform coefficients 121 .
  • cij denotes the elements with the coordinates (i, j) in the quantized transform coefficients (matrix) 121 .
  • the coefficient order control unit 113 can carry out individual 2D-1D transformations for the respective prediction modes.
  • the coefficient order control unit 113 performing such an operation is illustrated in FIG. 5A .
  • the coefficient order control unit 113 includes a selection switch 501 and individual 2D-1D transformation units 502 , . . . , and 510 for the respective nine types of prediction modes.
  • the selection switch 501 leads the quantized transform coefficients 121 to the 2D-1D transformation unit corresponding to the prediction mode (one of the 2D-1D transformation units 502 , . . . , 510 ).
  • a prediction mode index of 0 allows the selection switch 501 to lead the quantized transform coefficients 121 to the 2D-1D transformation unit 502 .
  • the prediction modes and the 2D-1D transformation units are in a one-to-one correspondence.
  • the quantized transform coefficients 121 are led to one 2D-1D transformation unit corresponding to the prediction mode.
  • FIG. 9 illustrates the 2D-1D transformation carried out by each of the 2D-1D transformation units 502 , . . . , 510 (in the case of a 4 ⁇ 4 pixel block). A specific design technique for the 2D-1D transformation for each prediction mode as shown in FIG. 9 will be described below.
  • FIG. 9 shows the index (idx) indicative of the order of coefficients (the order of scans) in the quantized transform coefficient sequence 117 subjected to the 2D-1D transformation by the 2D-1D transformation unit corresponding to each prediction mode, and the corresponding elements (cij) of the quantized transform coefficients 121 .
  • cij denotes the elements with the coordinates (i, j) in the quantized transform coefficients (matrix) 121 .
  • each prediction mode is expressed by its name, and the correspondence between the names and the prediction mode index is as shown in FIG. 4A .
  • the application of the individual 2D-1D transformations for the respective prediction modes allows the coefficients to be scanned in an order suitable for the tendency to generate nonzero coefficients in the quantized transform coefficients 121 for each prediction mode. This improves the coding efficiency.
  • the example regarding the 4 ⁇ 4 pixel block is shown.
  • the individual 2D-1D transformations for the respective prediction modes can similarly be defined.
  • the pixel block is a rectangular block expressed as M ⁇ N
  • the size of the block to be subjected to 2D-1D transformation may also be M ⁇ N.
  • such individual 2D-1D transformations as illustrated in FIG. 9 may be defined for the respective prediction modes.
  • the coefficient order control unit 113 may dynamically update the scan order for the 2D-1D transformation.
  • the coefficient order control unit 113 performing such an operation is illustrated in FIG. 5B .
  • the coefficient order control unit 113 includes the selection switch 501 , the individual 2D-1D transformation units 502 , . . . , and 510 for the respective nine types of prediction modes, an occurrence frequency count unit 511 , and a coefficient order update unit 512 .
  • the selection switch 501 is as described with reference to FIG. 5A .
  • the individual 2D-1D transformation units 502 , . . . , and 510 for the respective nine types of prediction modes are different from the 2D-1D transformation units shown in FIG. 5A in that the scan order for the 2D-1D transformation units 502 , . . . , and 510 is updated by the coefficient order update unit 512 .
  • the occurrence frequency count unit 511 creates, for each prediction mode, a histogram of the number of occurrences of nonzero coefficients in each element of the quantized transform coefficient sequence 117 .
  • the occurrence frequency count unit 511 inputs the created histogram 513 to the coefficient order update unit 512 .
  • the coefficient order update unit 512 updates the order of coefficients at a predetermined timing based on the histogram 513 .
  • the timing may be, for example, a timing when an encoding process carried out on a coding tree unit is finished or a timing when an encoding process carried out on one line in the coding tree unit is finished.
  • the coefficient order update unit 512 references the histogram 513 to update the order of coefficients for a prediction mode with an element for which the counted number of occurrences of nonzero coefficients is equal to or larger than a threshold.
  • the coefficient order update unit 512 performs the updating for a prediction mode with an element for which the counted number of occurrences of nonzero coefficients is 16 or more. Setting a threshold for the number of occurrences allows the order of coefficients to be globally updated, thus avoiding convergence to a local optimum solution.
  • the coefficient order update unit 512 sorts, for the updating-target prediction mode, the elements in order of decreasing occurrence frequency of nonzero coefficients.
  • the sorting can be achieved in accordance with an existing algorithm, for example, bubble sort or quick sort.
  • the coefficient order update unit 512 inputs coefficient order update information 514 indicative of the sorted order of the elements to the 2D-1D transformation unit corresponding to the updating-target prediction mode.
  • the 2D-1D transformation unit carries out a 2D-1D transformation in accordance with the updated scan order. If the scan order is dynamically updated, initial scan orders for the 2D-1D transformation units need to be set. For example, the zigzag scan or the scan order illustrated in FIG. 9 can be utilized as the initial scan order.
  • the dynamic updating of the scan order is expected to achieve a stably high coding efficiency even if the tendency to generate nonzero coefficients in the quantized transform coefficients 121 varies depending on the nature of the predicted image, the quantization information (quantization parameter), and the like. Specifically, the amount of code generated in run length coding in the entropy encoding unit 114 can be reduced.
  • FIG. 10A and FIG. 10B Processing carried out on the coding target block (coding tree unit) by the image encoding apparatus in FIG. 1 will be described below with reference to FIG. 10A and FIG. 10B .
  • the orthogonal transformation and inverse orthogonal transformation according to the present embodiment that is, the adaptive orthogonal transformation and inverse orthogonal transformation based on the 1D transform matrix set information 129
  • the syntax may be specified to make the orthogonal transformation and inverse orthogonal transformation according to the present embodiment disable.
  • step S 601 a process of encoding the coding target block is started.
  • the intra-prediction unit 108 and the inter-prediction unit 109 uses the reference image 125 saved to the reference image memory 107 to generate an intra-predicted image and an inter-predicted image (step S 602 ).
  • the encoding control unit 116 determines the optimum prediction mode in terms of the above-described coding cost to generate the prediction information 126 (step S 603 ).
  • the prediction information 126 is input to each element by the prediction selection unit 110 as described above. If the prediction information 126 generated in step S 603 is indicative of intra-prediction, the processing proceeds to step S 605 . If the prediction information 126 generated in step S 603 is indicative of inter-prediction, the processing proceeds to step S 605 ′.
  • step S 605 the subtraction unit 101 subtracts the (intra-) predicted image 127 from the coding target block to generate the prediction error 119 .
  • the processing then proceeds to step S 606 .
  • step S 605 ′ the subtraction unit 101 subtracts the (inter-) predicted image 127 from the coding target block to generate the prediction error 119 .
  • step S 614 ′ the subtraction unit 101 subtracts the (inter-) predicted image 127 from the coding target block to generate the prediction error 119 .
  • step S 606 the 1D transform matrix set unit 112 extracts the prediction mode information included in the prediction information 126 generated in step S 603 . Based on the extracted prediction mode information (for example, with reference to the table in FIG. 4A ), the 1D transform matrix set unit 112 generates the 1D transform matrix set information 129 (step S 607 ). The 1D transform matrix set unit 112 inputs the 1D transform matrix set information 129 to the orthogonal transformation unit 102 and the inverse orthogonal transformation unit 105 .
  • the selection switch 201 in the orthogonal transformation unit 102 selects the 1D orthogonal transformation unit A 206 or the 1D orthogonal transformation unit B 207 based on the 1D transform matrix set information 129 (step S 608 , step S 609 , and step S 610 ).
  • the selection switch 204 in the orthogonal transformation unit 102 selects the 1D orthogonal transformation unit A 208 or the 1D orthogonal transformation unit B 209 based on the 1D transform matrix set information 129 (step S 611 , step S 612 , and step S 613 ). The processing then proceeds to step S 614 .
  • the selection switch 201 selects the 1D orthogonal transformation unit A 206 in the vertical transformation unit 202 (step S 609 ). If TransformIdx is 0, the selection switch 204 selects the 1D orthogonal transformation unit A 208 in the horizontal transformation unit 205 (step S 612 ). If TransformIdx is 1, the selection switch 201 selects the 1D orthogonal transformation unit A 206 in the vertical transformation unit 202 (step S 609 ). If TransformIdx is 1, the selection switch 204 selects the 1D orthogonal transformation unit B 209 in the horizontal transformation unit 205 (step S 613 ).
  • TransformIdx an example of the 1D transform matrix set information 129
  • the selection switch 201 selects the 1D orthogonal transformation unit B 207 in the vertical transformation unit 202 (step S 610 ). If TransformIdx is 2, the selection switch 204 selects the 1D orthogonal transformation unit A 208 in the horizontal transformation unit 205 (step S 612 ). If TransformIdx is 3, the selection switch 201 selects the 1D orthogonal transformation unit B 207 in the vertical transformation unit 202 (step S 610 ). If TransformIdx is 3, the selection switch 204 selects the 1D orthogonal transformation unit B 209 in the horizontal transformation unit 205 (step S 613 ).
  • step S 614 the orthogonal transformation unit 102 carries out a vertical transformation and a horizontal transformation which correspond to the settings made in step S 608 , . . . , and step S 613 , on the prediction error 119 to generate the transform coefficients 120 .
  • the quantization unit 103 quantizes the transform coefficients 120 generated in step S 614 (step S 615 ). The processing then proceeds to step S 616 .
  • step S 614 ′ the orthogonal transformation unit 102 carries out fixed orthogonal transformation, for example, DCT, on the prediction error 119 to generate the transform coefficients 120 .
  • the quantization unit 103 quantizes the transform coefficients 120 generated in step S 614 ′ to generate the quantized transform coefficients 121 (step S 615 ′).
  • the processing proceeds to step S 617 ′.
  • the orthogonal transformation carried out in step S 614 ′ may be implemented by the DCT unit (not shown in the drawings) or by the 1D orthogonal transformation unit B 207 and the 1D orthogonal transformation unit B 209 .
  • step S 616 the coefficient order control unit 113 sets the scan order (that is, in the example shown in FIG. 5A and FIG. 5B , the unit to which the selection switch 501 is connected) based on the prediction mode information included in the prediction information 126 generated in step S 603 .
  • the processing proceeds to step S 617 . If the coefficient order control unit 113 carries out a common 2D-1D transformation regardless of the prediction mode, step S 616 may be omitted.
  • step S 617 the coefficient order control unit 113 carries out a 2D-1D transformation corresponding to the setting made in step S 616 , on the quantized transform coefficients 121 to generate the quantized transform coefficient sequence 117 .
  • the entropy encoding unit 114 carries out entropy encoding on the coding parameters including the quantized transform coefficient sequence 117 (step S 618 ).
  • the encoded data 130 is output at the appropriate timing managed by the encoding control unit 116 .
  • the de-quantization unit 104 de-quantizes the quantized transform coefficients 121 to generate the restored transform coefficients 122 (step S 619 ).
  • the processing proceeds to step S 620 .
  • step S 617 ′ the coefficient order control unit 113 carries out, for example, fixed 2D-1D transformation such as the zigzag scan or 2D-1D transformation corresponding to Intra_N ⁇ N_DC in FIG. 9 , on the quantized transform coefficient sequence 121 to generate the quantized transform coefficient sequence 117 .
  • the entropy encoding unit 114 carries out entropy encoding on the coding parameters including the quantized transform coefficient sequence 117 (step S 618 ′).
  • the encoded data 130 is output at the appropriate timing managed by the encoding control unit 116 .
  • the de-quantization unit 104 de-quantizes the quantized transform coefficients 121 to generate the restored transform coefficients 122 (step S 619 ′). The processing then proceeds to step S 626 ′.
  • the selection switch 301 in the inverse orthogonal transformation unit 105 selects the 1D inverse orthogonal transformation unit A 306 or the 1D inverse orthogonal transformation unit B 307 based on the 1D transform matrix set information 129 (step S 620 , step S 621 , and step S 622 ).
  • the selection switch 304 in the inverse orthogonal transformation unit 105 selects the 1D inverse orthogonal transformation unit A 308 or the 1D inverse orthogonal transformation unit B 309 based on the 1D transform matrix set information 129 (step S 623 , step S 624 , and step S 625 ). The processing then proceeds to step S 626 .
  • the selection switch 301 selects the 1D inverse orthogonal transformation unit A 306 in the vertical inverse transformation unit 302 (step S 621 ). If TransformIdx is 0, the selection switch 304 selects the 1D inverse orthogonal transformation unit A 308 in the horizontal inverse transformation unit 305 (step S 624 ). If TransformIdx is 1, the selection switch 301 selects the 1D inverse orthogonal transformation unit A 306 in the vertical inverse transformation unit 302 (step S 621 ).
  • TransformIdx is 1, the selection switch 304 selects the 1D inverse orthogonal transformation unit B 309 in the horizontal inverse transformation unit 305 (step S 625 ). If TransformIdx is 2, the selection switch 301 selects the 1D inverse orthogonal transformation unit B 307 in the vertical inverse transformation unit 302 (step S 622 ). If TransformIdx is 2, the selection switch 304 selects the 1D inverse orthogonal transformation unit A 308 in the horizontal inverse transformation unit 305 (step S 624 ). If TransformIdx is 3, the selection switch 301 selects the 1D inverse orthogonal transformation unit B 307 in the vertical inverse transformation unit 302 (step S 622 ). If TransformIdx is 3, the selection switch 304 selects the 1D inverse orthogonal transformation unit B 309 in the horizontal inverse transformation unit 305 (step S 625 ).
  • step S 626 the inverse orthogonal transformation unit 105 carries out a vertical transformation and a horizontal transformation which correspond to the settings made in step S 620 , . . . , and step S 625 , on the restored transform coefficients 122 to generate the restored prediction error 123 .
  • the processing proceeds to step S 627 .
  • step S 626 ′ the inverse orthogonal transformation unit 105 carries out an inverse orthogonal transformation, for example, IDCT, on the restored transform coefficients 123 to generate the restored prediction error 123 .
  • the processing then proceeds to step S 627 .
  • the fixed inverse orthogonal transformation carried out in step S 626 ′ may be implemented by the IDCT unit (not shown in the drawings) or by the 1D inverse orthogonal transformation unit B 307 and the 1D inverse orthogonal transformation unit B 309 .
  • step S 627 the addition unit 106 adds the restored prediction error 123 generated in step S 626 or step S 626 ′ to the predicted image 127 to generate the local decoded image 124 .
  • the local decoded image 124 is saved to the reference image memory 107 as a reference image.
  • the process of encoding the coding target block ends (step S 628 ).
  • the prediction error 119 for each prediction mode is generated. Those of the prediction errors 119 for the respective prediction modes are collected which exhibit, in the vertical direction or the horizontal direction, the above-described tendency to have an absolute value increasing with the distance from the reference pixel. Then, singular value decomposition is carried out on a matrix in which the direction with the tendency exhibited therein is defined as a column direction and in which the prediction errors 119 are arranged in a row direction. Thus, a 1D orthogonal basis is designed which eliminates the vertical correlation in the matrix. The 1D orthogonal basis is arranged in the column direction as row vectors to generate the 1D transform matrix A.
  • singular value decomposition is carried out on a matrix in which the direction with the tendency not exhibited therein is defined as the column direction and in which the prediction errors 119 are arranged in the row direction.
  • a 1D orthogonal basis is generated which eliminates the vertical correlation in the matrix.
  • the 1D orthogonal basis is arranged in the column direction as row vectors to generate the 1D transform matrix B.
  • the 1D transform matrix B may simply be replaced with a matrix for DCT.
  • the design for the 4 ⁇ 4 pixel block has been illustrated.
  • 1D transform matrices for the 8 ⁇ 8 pixel block and the 16 ⁇ 16 pixel block can similarly be designed.
  • the above-described design technique is illustrative and the design may be carried out as appropriate with the above-described nature of a prediction error taken into account.
  • the scan order for each prediction mode can be designed based on the quantized transform coefficients 121 generated by the quantization unit 103 .
  • the design for the 4 ⁇ 4 pixel block prepares a plurality of training images to generate a prediction error 119 for each of the nine types of prediction modes.
  • the orthogonal transformations shown in Expression (3) and Expression (4) are carried out on each of the prediction error 119 to generate the transform coefficients 120 .
  • the transform coefficients 120 are quantized.
  • the number of occurrences of nonzero coefficients is cumulatively added for each of the elements of the 4 ⁇ 4 pixel block.
  • the cumulative addition is carried out on all the training images to create a histogram indicative of the occurrence frequency of nonzero coefficients for each of the 16 elements of the 4 ⁇ 4 pixel block. Based on the histogram, indices of 0 to 15 are assigned to the elements in order of decreasing occurrence frequency. Such assignment of the indices is individually carried out on all the prediction modes. The order of the assigned indices is used as the scan order corresponding to each prediction mode.
  • the design for the 4 ⁇ 4 pixel block has been illustrated.
  • the scan orders for the 8 ⁇ 8 pixel block and the 16 ⁇ 16 pixel block can similarly be designed.
  • the design can be achieved using a similar technique.
  • the method for dynamically updating the scan order is as described with reference to FIG. 5B .
  • the syntax is indicative of the structure of encoded data (for example, the encoded data 130 in FIG. 1 ) obtained by the image encoding apparatus encoding video data.
  • the image decoding apparatus references the same syntax structure to interpret the syntax.
  • FIG. 11 illustrates syntax 700 utilized by the image encoding apparatus in FIG. 1 .
  • the syntax 700 includes three parts, that is, high-level syntax 701 , slice-level syntax 702 , and coding tree-level syntax 703 .
  • the high-level syntax 701 includes syntax information on a layer higher than a slice.
  • the slice refers to a rectangular area or a continuous area included in a frame or a field.
  • the slice-level syntax 702 includes information required to decode each slice.
  • the coding tree-level syntax 703 includes information required to decode each coding tree (that is, each coding tree unit). Each of these parts includes a further detailed syntax.
  • the high-level syntax 701 includes sequence-level syntax and picture-level syntax such as sequence parameter set syntax 704 and picture parameter set syntax 705 .
  • the slice-level syntax 702 includes slice header syntax 706 and slice data syntax 707 .
  • the coding tree level syntax 703 includes coding tree unit syntax 708 and prediction unit syntax 709 .
  • the coding tree unit syntax 708 may have a quadtree structure. Specifically, as a syntax element of the coding tree unit syntax 708 , the coding tree unit syntax 708 can further be recursively invoked. That is, one coding tree unit can be divided into pieces using quadtrees. Furthermore, the coding tree unit syntax 708 contains transform unit syntax 710 . The transform unit syntax 710 is invoked at each coding tree unit syntax 708 located at the terminal of the quadtree. The transform unit syntax 710 has information on inverse orthogonal transformation, quantization, and the like described therein.
  • FIG. 12 illustrates the slice header syntax 706 according to the present embodiment.
  • slice_directional_unified_transform_flag shown in FIG. 12 is, for example, a syntax element which indicates whether the orthogonal transformation and inverse orthogonal transformation according to the present embodiment are enable or disable for a certain slice.
  • slice_directional_unified_transform_flag of 0 means that the orthogonal transformation and inverse orthogonal transformation according to the present embodiment are enable in the slice.
  • the orthogonal transformation unit 102 and the inverse orthogonal transformation unit 105 carry out fixed orthogonal transformation and inverse orthogonal transformation such as DCT and IDCT.
  • the fixed orthogonal transformation and inverse orthogonal transformation may be carried out by the 1D orthogonal transformation unit B 207 , the 1D orthogonal transformation unit B 209 , the 1D inverse orthogonal transformation unit B 307 , and the 1D inverse orthogonal transformation unit B 309 (that is, by the 1D transform matrix B) or by the DCT unit and IDCT unit (neither of the units is shown in the drawings).
  • the coefficient order control unit 113 carries out fixed 2D-1D transformation (for example, the zigzag scan).
  • the fixed 2D-1D transformation may be carried out by the 2D-1D transformation unit (mode 2) 504 or by a 2D-1D transformation unit not shown in the drawings.
  • slice_directional_unified_transform_flag of 1 means that the orthogonal transformation and inverse orthogonal transformation according to the present embodiment are enable throughout the slice. That is, the encoding process is carried out in accordance with the above-described flowchart of encoding in FIG. 10A and FIG. 10B . That is, the selection switch 201 selects the 1D orthogonal transformation unit A 206 or the 1D orthogonal transformation unit B 207 based on the 1D transform matrix set information 129 . The selection switch 204 selects the 1D orthogonal transformation unit A 208 or the 1D orthogonal transformation unit B 209 based on the 1D transform matrix set information 129 .
  • the selection switch 301 selects the 1D inverse orthogonal transformation unit A 306 or the 1D inverse orthogonal transformation unit B 307 based on the 1D transform matrix set information 129 .
  • the selection switch 304 selects the 1D inverse orthogonal transformation unit A 308 or the 1D inverse orthogonal transformation unit B 309 based on the 1D transform matrix set information 129 .
  • the selection switch 501 selects one of the 2D-1D transformation units 502 , . . . , and 510 in accordance with the prediction mode information included in the prediction information 126 .
  • slice_directional_unified_transform_flag 1
  • a syntax in a lower layer (such as the coding tree unit or the transform unit) may be specified to indicate whether the orthogonal transformation and inverse orthogonal transformation according to the present embodiment are enable or disable for each local area inside the slice.
  • FIG. 13 illustrates the coding tree unit syntax 708 according to the present embodiment.
  • ctb_directional_unified_transform_flag shown in FIG. 13 is a syntax element indicating whether the orthogonal transformation and inverse orthogonal transformation according to the present embodiment are enable or disable for a certain coding unit.
  • pred_mode shown in FIG. 13 is a syntax element included in the prediction unit syntax 709 and indicates the type of coding in the coding tree unit or a macro block. MODE_INTRA indicates that the coding type is intra-prediction.
  • ctb_directional_unified_transform_flag is encoded only when the above-described slice_directional_unified_transform_flag is 1 and when the coding type of the coding tree unit is intra-prediction.
  • ctb_directional_unified_transform_flag of 0 means that the orthogonal transformation and inverse orthogonal transformation according to the present embodiment are disable in the coding tree unit.
  • the orthogonal transformation unit 102 and the inverse orthogonal transformation unit 105 carry out fixed orthogonal transformation and inverse orthogonal transformation such as DCT and IDCT.
  • the fixed orthogonal transformation and inverse orthogonal transformation may be carried out by the 1D orthogonal transformation unit B 207 , the 1D orthogonal transformation unit B 209 , the 1D inverse orthogonal transformation unit B 307 , and the 1D inverse orthogonal transformation unit B 309 (that is, by the 1D transform matrix B) or by the DCT unit and IDCT unit (neither of the units is shown in the drawings).
  • the coefficient order control unit 113 carries out fixed 2D-1D transformation (for example, the zigzag scan).
  • the fixed 2D-1D transformation may be carried out by the 2D-1D transformation unit (mode 2) 504 or by the 2D-1D transformation unit not shown in the drawings.
  • ctb_directional_unified_transform_flag of 1 makes the orthogonal transformation and inverse orthogonal transformation according to the present embodiment enable in the coding unit.
  • the encoding process is carried out in accordance with the above-described flowchart of encoding in FIG. 10A and FIG. 10B . That is, the selection switch 201 selects the 10 orthogonal transformation unit A 206 or the 1D orthogonal transformation unit B 207 based on the 1D transform matrix set information 129 .
  • the selection switch 204 selects the 1D orthogonal transformation unit A 208 or the 1D orthogonal transformation unit B 209 based on the 1D transform matrix set information 129 .
  • the selection switch 301 selects the 1D inverse orthogonal transformation unit A 306 or the 1D inverse orthogonal transformation unit B 307 based on the 1D transform matrix set information 129 .
  • the selection switch 304 selects the 1D inverse orthogonal transformation unit A 308 or the 1D inverse orthogonal transformation unit B 309 based on the 1D transform matrix set information 129 .
  • the selection switch 501 selects one of the 2D-1D transformation units 502 , . . . , and 510 in accordance with the prediction mode information included in the prediction information 126 .
  • the flag specifying whether the orthogonal transformation and inverse orthogonal transformation according to the present embodiment are enable or disable is encoded in the coding tree unit syntax 708 as in the example in FIG. 13 .
  • the amount of information (the amount of code) is increased compared to when the flag is not encoded.
  • encoding the flag enables the optimum orthogonal transformation to be carried out for each local area (that is, for each coding tree unit).
  • FIG. 14 illustrates the transform unit syntax 710 according to the present embodiment.
  • tu_directional_unified_transform_flag shown in FIG. 14 is a syntax element indicating whether the orthogonal transformation and inverse orthogonal transformation according to the present embodiment are enable or disable for a certain transform unit.
  • pred_mode shown in FIG. 14 is a syntax element included in the prediction unit syntax 709 and indicates the type of coding in a certain coding tree unit or macro block. MODE_INTRA indicates that the coding type is intra-prediction.
  • tu_directional_unified_transform_flag is encoded only when the above-described slice_directional_unified_transform_flag is 1 and when the coding type of the coding tree unit is intra-prediction.
  • the orthogonal transformation unit 102 and the inverse orthogonal transformation unit 105 carry out fixed orthogonal transformation and inverse orthogonal transformation such as DCT and IDCT.
  • the fixed orthogonal transformation and inverse orthogonal transformation may be carried out by the 1D orthogonal transformation unit B 207 , the 1D orthogonal transformation unit B 209 , the 1D inverse orthogonal transformation unit B 307 , and the 1D inverse orthogonal transformation unit B 309 (that is, by the 1D transform matrix B) or by the DCT unit and IDCT unit (neither of the units is shown in the drawings).
  • the coefficient order control unit 113 carries out fixed 2D-1D transformation (for example, the zigzag scan).
  • the fixed 2D-1D transformation may be carried out by the 2D-1D transformation unit (mode 2) 504 or by the 2D-1D transformation unit not shown in the drawings.
  • tu_directional_unified_transform_flag of 1 makes the orthogonal transformation and inverse orthogonal transformation according to the present embodiment enable in the transform unit.
  • the encoding process is carried out in accordance with the above-described flowchart of encoding in FIG. 10A and FIG. 10B . That is, the selection switch 201 selects the 1D orthogonal transformation unit A 206 or the 1D orthogonal transformation unit B 207 based on the 1D transform matrix set information 129 .
  • the selection switch 204 selects the 1D orthogonal transformation unit A 208 or the 1D orthogonal transformation unit B 209 based on the 1D transform matrix set information 129 .
  • the selection switch 301 selects the 1D inverse orthogonal transformation unit A 306 or the 1D inverse orthogonal transformation unit B 307 based on the 1D transform matrix set information 129 .
  • the selection switch 304 selects the 1D inverse orthogonal transformation unit A 308 or the 1D inverse orthogonal transformation unit B 309 based on the 1D transform matrix set information 129 .
  • the selection switch 501 selects one of the 2D-1D transformation units 502 , . . . , and 510 in accordance with the prediction mode information included in the prediction information 126 .
  • the amount of information (the amount of code) is increased compared to when the flag is not coded.
  • encoding the flag enables the optimum orthogonal transformation to be carried out for each local area (that is, for each transform unit).
  • syntax elements not specified according to the present embodiment or descriptions of other conditional branches may be inserted between rows in the syntax tables illustrated in FIG. 12 , FIG. 13 , and FIG. 14 .
  • the syntax table may be divided into a plurality of tables or a plurality of the syntax tables may be unified. Additionally, the illustrated terms for the syntax elements may be optionally changed.
  • the image encoding apparatus utilizes the tendency of the intra-prediction to have a prediction accuracy decreasing with increasing distance from the reference pixel.
  • the image encoding apparatus classifies the vertical and horizontal directions of the prediction modes into two classes depending on whether or not the tendency is present.
  • the image encoding apparatus further adaptively applies the 1D transform matrix A or the 1D transform matrix B to each of the vertical and horizontal directions.
  • the 1D transform matrix A is generated by pre-designing a common transform basis so as to increase the coefficient density after 1D orthogonal transformation (that is, to reduce the rate of nonzero coefficients in the quantized transform coefficients 121 ) in a direction orthogonal to the line of the group of reference pixels (vertical direction or horizontal direction).
  • the 1D transform matrix B is generated by designing a general-purpose transform matrix having no such nature.
  • the general-purpose transformation is DCT.
  • the image encoding apparatus according to the present embodiment achieves a high transform efficiency compared to the case where a fixed orthogonal transformation such as DCT is uniformly applied to the prediction modes.
  • orthogonal transformation unit 102 and inverse orthogonal transformation unit 105 are suited both for hardware implementation and for software implementation.
  • Expression (3) to Expression (6) express multiplications of fixed matrices.
  • the orthogonal transformation unit and the inverse orthogonal transformation unit are implemented by hardware, the implementation is expected to be configured using hard wired logic instead of multipliers.
  • the nine 2D orthogonal transformation units or 18 1D orthogonal transformation units carry out multiplications of different transform matrices.
  • the nine 2D orthogonal transformation units or 18 1D orthogonal transformation units are additionally provided. This increases the scale of relevant circuits.
  • the orthogonal transformation unit and inverse orthogonal transformation unit according to the present embodiment use a combination of two (if a vertical (inverse) transformation unit and a horizontal (inverse) transformation unit are shared in a time division manner) 1D orthogonal transformation units with a circuit for matrix transposition to carry out four types of two-dimensional orthogonal transformations.
  • the orthogonal transformation unit and inverse orthogonal transformation unit according to the present embodiment allow an increase in the circuit scale in the hardware implementation to be significantly suppressed.
  • the software implementation is expected to be as follows.
  • Each of the transform matrices is invoked for the corresponding one of the prediction modes, and an orthogonal transformation is implemented using a general-purpose multiplier. This may increase the size of the memory to which the transform matrices are saved and thus the costs and also increase memory bandwidth as a result of loading of a transform matrix into the memory for each transformation.
  • the orthogonal transformation unit and inverse orthogonal transformation unit according to the present embodiment combines vertical transformation and horizontal transformation using two 1D orthogonal transform matrices, to carry out four types of orthogonal transformations.
  • the orthogonal transformation unit and inverse orthogonal transformation unit according to the present embodiment allows an increase in the memory size in the software implementation to be significantly suppressed.
  • the quantized transform coefficients 121 shows the tendency to generate nonzero coefficients is biased depending on the element.
  • the tendency to generate nonzero coefficients varies depending on the prediction direction of the intra-prediction.
  • even the encoding of pixel blocks from different input images 118 leads to similar tendencies to generate nonzero coefficients.
  • the coefficient order control unit 113 transforms the elements into the respective one-dimensional quantized transform coefficient sequences 122 in order of decreasing occurrence probability of nonzero coefficients in the quantized transform coefficients 121 , the nonzero coefficients are very likely to densely occur in the quantized transform coefficient sequence 122 .
  • the coefficient order control unit 113 may fixedly utilize scan orders learned for the respective prediction modes or utilize a scan order dynamically updated during the encoding process. Utilizing the scan orders optimized for the respective prediction modes, the entropy encoding unit 114 can reduce, compared to H.264, the amount of generated code based on the quantized transform coefficient sequence 122 without significantly increasing the amount of calculation.
  • An image encoding apparatus is different from the image encoding apparatus according to the first embodiment in the details of the orthogonal transformation and the inverse orthogonal transformation.
  • the same components of the present embodiment as the corresponding components of the first embodiment are hereinafter denoted by the same reference numerals, and mainly different components will be described below.
  • An image decoding apparatus corresponding to the image encoding apparatus according to the present embodiment will be described in a fifth embodiment.
  • the image encoding apparatus includes the orthogonal transformation unit 102 illustrated in FIG. 16 .
  • the orthogonal transformation unit 102 in FIG. 16 includes a selection switch 801 , a vertical transformation unit 802 , the transposition unit 203 , a selection switch 804 , and a horizontal transformation unit 805 .
  • the vertical transformation unit 802 includes a 1D orthogonal transformation unit C 806 , a 1D orthogonal transformation unit D 807 , and a 1D orthogonal transformation unit E 808 .
  • the horizontal transformation unit 805 includes a 1D orthogonal transformation unit C 809 , a 1D orthogonal transformation unit D 810 , and a 1D orthogonal transformation unit E 811 .
  • the order of the vertical transformation unit 802 and the horizontal transformation unit 805 is illustrative and may be reversed.
  • the 1D orthogonal transformation unit C 806 and the 1D orthogonal transformation unit C 809 have common functions in that both units multiply an input matrix by a 1D transform matrix C.
  • the 1D orthogonal transformation unit D 807 and the 1D orthogonal transformation unit D 810 have common functions in that both units multiply the input matrix by a 1D transform matrix D.
  • the 1D orthogonal transformation unit E 808 and the 1D orthogonal transformation unit E 811 have common functions in that both units multiply the input matrix by a 1D transform matrix E.
  • the 1D transform matrix C, 1D transform matrix D, and 1D transform matrix E will be described below.
  • the prediction error 119 has a tendency to have an absolute value increasing with the distance from the reference pixel. The tendency is similarly exhibited regardless of the prediction direction. However, the prediction error in the DC prediction mode does not exhibit the tendency in the vertical direction or the horizontal direction.
  • the present embodiment utilizes the 1D transform matrix E described below for the DC prediction mode. For the prediction modes other then the DC prediction mode, the present embodiment adaptively utilizes the 1D transform matrix C and the 1D transform matrix D depending on whether or not the tendency is present, as is the case with the first embodiment.
  • the 1D transform matrix C can be generated using the same design technique as that for the above-described 1D transform matrix A.
  • the 1D transform matrix D can be generated using a design technique similar to the design technique for the above-described 1D transform matrix B. That is, the 1D transform matrix D can be generated by carrying out the design technique for the 1D transform matrix B with the DC prediction mode excluded.
  • the 1D transform matrix E may be a matrix for DCT.
  • the 1D transform matrix E may be generated by pre-designing a common transform basis so as to increase, compared to the 1D transform matrix D, the coefficient density after 1D orthogonal transformation (that is, to reduce the rate of nonzero coefficients in the quantized transform coefficients 121 ) in the vertical direction and the horizontal direction.
  • the image encoding apparatus includes the inverse orthogonal transformation unit 105 illustrated in FIG. 17 .
  • the inverse orthogonal transformation unit 105 in FIG. 17 includes a selection switch 901 , a vertical inverse transformation unit 902 , the transposition unit 303 , a selection switch 904 , and a horizontal inverse transformation unit 905 .
  • the vertical inverse transformation unit 902 includes a 1D inverse orthogonal transformation unit C 906 , a 1D inverse orthogonal transformation unit D 907 , and a 1D inverse orthogonal transformation unit E 908 .
  • the horizontal inverse transformation unit 905 includes a 1D inverse orthogonal transformation unit C 909 , a 1D inverse orthogonal transformation unit D 910 , and a 1D inverse orthogonal transformation unit E 911 .
  • the order of the vertical inverse transformation unit 902 and the horizontal inverse transformation unit 905 is illustrative and may be reversed.
  • the 1D inverse orthogonal transformation unit C 906 and the 1D inverse orthogonal transformation unit C 909 have common functions in that both units multiply an input matrix by the transposed matrix of the 1D transform matrix C.
  • the 1D inverse orthogonal transformation unit D 907 and the 1D inverse orthogonal transformation unit D 910 have common functions in that both units multiply the input matrix by the transposed matrix of the 1D transform matrix D.
  • the 1D inverse orthogonal transformation unit E 908 and the 1D inverse orthogonal transformation unit E 911 have common functions in that both units multiply the input matrix by the transposed matrix of the 1D transform matrix E.
  • the 1D transform matrix set information 129 according to the present embodiment generated by the 1D transform matrix set unit 112 will be described below in detail.
  • the 1D transform matrix set information 129 directly or indirectly indicates the vertical transform index for selection of a transform matrix for use in vertical orthogonal transformation and vertical inverse orthogonal transformation and the horizontal transform index for selection of a transform matrix for use in horizontal orthogonal transformation and horizontal inverse orthogonal transformation.
  • the 1D transform matrix set information 129 can be expressed by a transform index (TrasformIdx) shown in FIG. 18D .
  • Reference to the table in FIG. 18D allows the vertical transform index (Vertical Transform Idx) and the horizontal transform index (Horizontal Transform Idx) to be derived from the transform index.
  • a vertical transform index of “0” allows selection of the above-described 1D transform matrix C (1D_Transform_Matrix_C) or the transposed matrix thereof for vertical orthogonal transformation or vertical inverse orthogonal transformation.
  • a vertical transform index of “1” allows selection of the above-described 1D transform matrix D (1D_Transform_Matrix_D) or the transposed matrix thereof for vertical orthogonal transformation or vertical inverse orthogonal transformation.
  • a vertical transform index of “2” allows selection of the above-described 1D transform matrix E (1D_Transform_Matrix_E) or the transposed matrix thereof for vertical orthogonal transformation or vertical inverse orthogonal transformation.
  • a horizontal transform index of “0” allows selection of the 1D transform matrix C (1D_Transform_Matrix_C) or the transposed matrix thereof for horizontal orthogonal transformation or horizontal inverse orthogonal transformation.
  • a horizontal transform index of “1” allows selection of the 1D transform matrix D (1D_Transform_Matrix_D) or the transposed matrix thereof for horizontal orthogonal transformation or horizontal inverse orthogonal transformation.
  • a horizontal transform index of “2” allows selection of the 1D transform matrix E (1D_Transform_Matrix_E) or the transposed matrix thereof for horizontal orthogonal transformation or horizontal inverse orthogonal transformation.
  • FIG. 18A illustrates the index for each (intra-) prediction mode (IntraN ⁇ NPredModeIndex), the name thereof (Name of IntraN ⁇ NPredMode), and the corresponding vertical transform index and horizontal transform index.
  • the size of the prediction target block can be expanded to “M ⁇ N” (that is, rectangles other than squares).
  • FIG. 18E is obtained by integrating FIG. 18A and FIG. 18D together and shows the index for each prediction mode, the name of the prediction mode, and the corresponding transform index.
  • the 1D transform matrix set unit 112 detects the index of the prediction mode in the prediction mode information included in the prediction information 126 .
  • the 1D transform matrix set unit 112 then generates the corresponding 1D transform matrix set information 129 .
  • the tables shown in FIG. 18A , FIG. 18B , FIG. 18C , FIG. 18D , and FIG. 18E are illustrative.
  • the 1D transform matrix set unit 112 may generate the 1D transform matrix set information 129 while avoiding the use of some or all of the tables.
  • TransformIdx indicative of 0 means that the vertical transform index indicates 0 and that the horizontal transform index indicates 0.
  • These index values also mean that the transposed matrix of the 1D transform matrix C is used for vertical inverse orthogonal transformation and that the transposed matrix of the 1D transform matrix C is used for horizontal inverse orthogonal transformation.
  • TransformIdx indicative of 1 means that the vertical transform index indicates 0 and that the horizontal transform index indicates 1. This means that the 1D transform matrix C is used for vertical orthogonal transformation and that the 1D transform matrix D is used for horizontal orthogonal transformation. These index values also mean that the transposed matrix of the 1D transform matrix C is used for vertical inverse orthogonal transformation and that the transposed matrix of the 1D transform matrix D is used for horizontal inverse orthogonal transformation.
  • TransformIdx indicative of 2 means that the vertical transform index indicates 1 and that the horizontal transform index indicates 0. This means that the 1D transform matrix D is used for vertical orthogonal transformation and that the 1D transform matrix C is used for horizontal orthogonal transformation. These index values also mean that the transposed matrix of the 1D transform matrix D is used for vertical inverse orthogonal transformation and that the transposed matrix of the 1D transform matrix C is used for horizontal inverse orthogonal transformation.
  • TransformIdx indicative of 3 means that the vertical transform index indicates 2 and that the horizontal transform index indicates 2. This means that the 1D transform matrix E is used for vertical orthogonal transformation and that the 1D transform matrix E is used for horizontal orthogonal transformation. These index values also mean that the transposed matrix of the 1D transform matrix E is used for vertical inverse orthogonal transformation and that the transposed matrix of the 1D transform matrix E is used for horizontal inverse orthogonal transformation.
  • the size of the block to be orthogonally transformed may also be M ⁇ N.
  • the table shown in FIG. 18A assigns the 1D transform matrix set information 129 taking the above-described tendency of each intra-prediction mode into account. That is, for the DC prediction mode, 2 is assigned both to the vertical transform index and to the horizontal transform index. Hence, for the DC prediction mode, the orthogonal transformation and inverse orthogonal transformation are carried out in the vertical direction and the horizontal direction using the 1D transform matrix E and the transposed matrix thereof. As a result, a high transform efficiency is achieved.
  • 0 is assigned to the vertical transform index if the tendency is exhibited in the vertical direction of the prediction error.
  • 0 is assigned to the horizontal transform index if the tendency is exhibited in the horizontal direction of the prediction error.
  • 1 is assigned to each of the directions not exhibiting the tendency.
  • a high transform efficiency is achieved by classifying the vertical and horizontal directions of the prediction modes other than the DC prediction mode into two classes depending whether or not the tendency is present and adaptively applying the 1D transform matrix C or the 1D transform matrix D to each of the vertical and horizontal directions.
  • the image encoding apparatus utilizes the tendency of the intra-prediction to have a decreasing prediction accuracy with increasing distance from the reference pixel as in the case of the first embodiment, while applying the orthogonal transformation and the inverse orthogonal transformation to the DC prediction in a manner different from the manner for the other prediction modes.
  • the image encoding apparatus classifies the vertical and horizontal directions of the prediction modes into the two classes depending whether or not the tendency is present and adaptively applies the 1D transform matrix C or the 1D transform matrix D to each of the vertical and horizontal directions.
  • the image encoding apparatus applies the 1D transform matrix E to the DC prediction mode.
  • the 1D transform matrix C is generated by pre-designing a common transform basis so as to increase the coefficient density after 1D orthogonal transformation (that is, to reduce the rate of nonzero coefficients in the quantized transform coefficients 121 ) in the direction orthogonal to the line of the group of reference pixels (vertical direction or horizontal direction).
  • the 1D transform matrix D is generated by designing a general-purpose transform matrix having no such nature with the DC prediction mode excluded.
  • the 1D transform matrix E may be a matrix for DCT.
  • the 1D transform matrix E is generated by pre-designing a common transform basis so as to increase, for the prediction error 119 in the DC prediction mode, the coefficient density after 1D orthogonal transformation (that is, to reduce the rate of nonzero coefficients in the quantized transform coefficients 121 ) in the vertical direction and the horizontal direction.
  • the image encoding apparatus achieves a higher transform efficiency than in the case where fixed orthogonal transformation such as DCT is uniformly applied to the prediction modes.
  • An image encoding apparatus is different from the image encoding apparatuses according to the first embodiment and the second embodiment in the details of the orthogonal transformation and the inverse orthogonal transformation.
  • the same components of the present embodiment as the corresponding components of the first embodiment or the second embodiment are hereinafter denoted by the same reference numerals, and mainly different components will be described below.
  • An image decoding apparatus corresponding to the image encoding apparatus according to the present embodiment will be described in a sixth embodiment.
  • the image encoding apparatus includes the orthogonal transformation unit 102 illustrated in FIG. 19 .
  • the orthogonal transformation unit 102 in FIG. 19 includes a selection switch 1201 , a vertical transformation unit 1202 , the transposition unit 203 , a selection switch 1204 , and a horizontal transformation unit 1205 .
  • the vertical transformation unit 1202 includes a 1D orthogonal transformation unit F 1206 , a 1D orthogonal transformation unit G 1207 , and a 1D orthogonal transformation unit H 1208 .
  • the horizontal transformation unit 1205 includes a 1D orthogonal transformation unit F 1209 , a 1D orthogonal transformation unit G 1210 , and a 1D orthogonal transformation unit H 1211 .
  • the order of the vertical transformation unit 1202 and the horizontal transformation unit 1205 is illustrative and may be reversed.
  • the 1D orthogonal transformation unit F 1206 and the 1D orthogonal transformation unit F 1209 have common functions in that both units multiply an input matrix by a 1D transform matrix F.
  • the 1D orthogonal transformation unit G 1207 and the 1D orthogonal transformation unit G 1210 have common functions in that both units multiply the input matrix by a 1D transform matrix G.
  • the 1D orthogonal transformation unit H 1208 and the 1D orthogonal transformation unit H 1211 have common functions in that both units multiply the input matrix by a 1D transform matrix H.
  • the 1D transform matrix F, 1D transform matrix G, and 1D transform matrix H will be described below.
  • the prediction error 119 has a tendency to have an absolute value increasing with the distance from the reference pixel. The tendency is similarly exhibited regardless of the prediction direction.
  • the intra-prediction modes include prediction modes in which only the group of reference pixels on a line located to the left of and adjacent to the prediction target block or on a line located above and adjacent to the prediction target block is referenced (the reference pixel values are copied or interpolation is carried out using the reference pixel values) and prediction modes in which the groups of reference pixels on the line located to the left of and adjacent to the prediction target block and on the line located above and adjacent to the prediction target block are referenced.
  • the present embodiment carries out orthogonal transformation and inverse orthogonal transformation differently between the prediction modes in which only the group of reference pixels on one line is referenced and the prediction modes in which the groups of reference pixels on two lines are referenced.
  • the 1D transform matrix H described below is utilized for the prediction modes in which the groups of reference pixels on two lines are referenced.
  • the 1D transform matrix F and the 1D transform matrix G are adaptively utilized for the prediction modes in which only the group of reference pixels on one line is referenced, depending on whether or not the tendency is present as is the case with the first embodiment.
  • the 1D transform matrix F can be generated using a design technique similar to the design technique for the above-described 1D transform matrix A. Furthermore, the 1D transform matrix F can be generated by carrying out the design technique for the above-described 1D transform matrix A while excluding the prediction modes in which the groups of reference pixels on two lines are referenced (for example, the mode 4, mode 5, and mode 6 in FIG. 7A ). Furthermore, the 1D transform matrix G can be generated using the same design technique as that for the above-described 1D transform matrix B. Alternatively, the 1D transform matrix G may be a matrix for DCT.
  • the 1D transform matrix H may be generated by pre-designing a common transform basis so as to increase, for the prediction error 119 in the prediction modes in which the groups of reference pixels on two lines are referenced, the coefficient density after 1D orthogonal transformation (that is, to reduce the rate of nonzero coefficients in the quantized transform coefficients 121 ) in the vertical direction and the horizontal direction.
  • the image encoding apparatus includes the inverse orthogonal transformation unit 105 illustrated in FIG. 20 .
  • the inverse orthogonal transformation unit 105 in FIG. 20 includes a selection switch 1301 , a vertical inverse transformation unit 1302 , the transposition unit 303 , a selection switch 1304 , and a horizontal inverse transformation unit 1305 .
  • the vertical inverse transformation unit 1302 includes a 1D inverse orthogonal transformation unit F 1306 , a 1D inverse orthogonal transformation unit G 1307 , and a 1D inverse orthogonal transformation unit H 1308 .
  • the horizontal inverse transformation unit 1305 includes a 1D inverse orthogonal transformation unit F 1309 , a 1D inverse orthogonal transformation unit G 1310 , and a 1D inverse orthogonal transformation unit H 1311 .
  • the order of the vertical inverse transformation unit 1302 and the horizontal inverse transformation unit 1305 is illustrative and may be reversed.
  • the 1D inverse orthogonal transformation unit F 1306 and the 1D inverse orthogonal transformation unit F 1309 have common functions in that both units multiply an input matrix by the transposed matrix of the 1D transform matrix F.
  • the 1D inverse orthogonal transformation unit G 1307 and the 1D inverse orthogonal transformation unit G 1310 have common functions in that both units multiply the input matrix by the transposed matrix of the 1D transform matrix G.
  • the 1D inverse orthogonal transformation unit H 1308 and the 1D inverse orthogonal transformation unit H 1311 have common functions in that both units multiply the input matrix by the transposed matrix of the 1D transform matrix H.
  • the 1D transform matrix set information 129 according to the present embodiment generated by the 1D transform matrix set unit 112 will be described below in detail.
  • the 1D transform matrix set information 129 directly or indirectly indicates the vertical transform index for selection of a transform matrix for use in vertical orthogonal transformation and vertical inverse orthogonal transformation and the horizontal transform index for selection of a transform matrix for use in horizontal orthogonal transformation and horizontal inverse orthogonal transformation.
  • the 1D transform matrix set information 129 can be expressed by a transform index (TrasformIdx) shown in FIG. 21D .
  • Reference to the table in FIG. 21D allows the vertical transform index (Vertical Transform Idx) and the horizontal transform index (Horizontal Transform Idx) to be derived from the transform index.
  • a vertical transform index of “0” allows selection of the above-described 1D transform matrix F (1D_Transform_Matrix_F) or the transposed matrix thereof for vertical orthogonal transformation or vertical inverse orthogonal transformation.
  • a vertical transform index of “1” allows selection of the above-described 1D transform matrix G (1D_Transform_Matrix_G) or the transposed matrix thereof for vertical orthogonal transformation or vertical inverse orthogonal transformation.
  • a vertical transform index of “2” allows selection of the above-described 1D transform matrix H (1D_Transform_Matrix_H) or the transposed matrix thereof for vertical orthogonal transformation or vertical inverse orthogonal transformation.
  • a horizontal transform index of “0” allows selection of the 1D transform matrix F (1D_Transform_Matrix_F) or the transposed matrix thereof for horizontal orthogonal transformation or horizontal inverse orthogonal transformation.
  • a horizontal transform index of “1” allows selection of the 1D transform matrix G (1D_Transform_Matrix_G) or the transposed matrix thereof for horizontal orthogonal transformation or horizontal inverse orthogonal transformation.
  • a horizontal transform index of “2” allows selection of the 1D transform matrix H (1D_Transform_Matrix_H) or the transposed matrix thereof for horizontal orthogonal transformation or horizontal inverse orthogonal transformation.
  • FIG. 21A illustrates the index for each (intra-) prediction mode (IntraN ⁇ NPredModeIndex), the name thereof (Name of IntraN ⁇ NPredMode), and the corresponding vertical transform index and horizontal transform index.
  • the size of the prediction target block can be expanded to “M ⁇ N” (that is, rectangles other than squares).
  • FIG. 21E is obtained by integrating FIG. 21A and FIG. 21D together and shows the index for each prediction mode, the name of the index, and the corresponding transform index.
  • the 1D transform matrix set unit 112 detects the index of the prediction mode in the prediction mode information included in the prediction information 126 .
  • the 1D transform matrix set unit 112 then generates the corresponding 1D transform matrix set information 129 .
  • the tables shown in FIG. 21A , FIG. 21B , FIG. 21C , FIG. 21D , and FIG. 21E are illustrative.
  • the 1D transform matrix set unit 112 may generate the 1D transform matrix set information 129 while avoiding the use of some or all of the tables.
  • TransformIdx indicative of 0 means that the vertical transform index indicates 2 and that the horizontal transform index indicates 2.
  • These index values also mean that the transposed matrix of the 1D transform matrix H is used for vertical inverse orthogonal transformation and that the transposed matrix of the 1D transform matrix H is used for horizontal inverse orthogonal transformation.
  • TransformIdx indicative of 1 means that the vertical transform index indicates 0 and that the horizontal transform index indicates 1. This means that the 1D transform matrix F is used for vertical orthogonal transformation and that the 1D transform matrix G is used for horizontal orthogonal transformation. These index values also mean that the transposed matrix of the 1D transform matrix F is used for vertical inverse orthogonal transformation and that the transposed matrix of the 1D transform matrix G is used for horizontal inverse orthogonal transformation.
  • TransformIdx indicative of 2 means that the vertical transform index indicates 1 and that the horizontal transform index indicates 0. This means that the 1D transform matrix G is used for vertical orthogonal transformation and that the 1D transform matrix F is used for horizontal orthogonal transformation. These index values also mean that the transposed matrix of the 1D transform matrix G is used for vertical inverse orthogonal transformation and that the transposed matrix of the 1D transform matrix F is used for horizontal inverse orthogonal transformation.
  • TransformIdx indicative of 3 means that the vertical transform index indicates 1 and that the horizontal transform index indicates 1. This means that the 1D transform matrix G is used for vertical orthogonal transformation and that the 1D transform matrix G is used for horizontal orthogonal transformation. These index values also mean that the transposed matrix of the 1D transform matrix G is used for vertical inverse orthogonal transformation and that the transposed matrix of the 1D transform matrix G is used for horizontal inverse orthogonal transformation.
  • the size of the block to be orthogonally transformed may also be M ⁇ N.
  • the table shown in FIG. 21A assigns the 1D transform matrix set information 129 taking the above-described tendency of each intra-prediction mode into account. That is, for the prediction modes in which the groups of reference pixels on two lines are referenced, 2 is assigned both to the vertical transform index and to the horizontal transform index. Hence, for the prediction modes in which the groups of reference pixels on two lines are referenced, the orthogonal transformation and inverse orthogonal transformation are carried out in the vertical direction and the horizontal direction using the 1D transform matrix H and the transposed matrix thereof. As a result, a high transform efficiency is achieved.
  • 0 is assigned to the vertical transform index if the tendency is exhibited in the vertical direction of the prediction error.
  • 0 is assigned to the horizontal transform index if the tendency is exhibited in the horizontal direction of the prediction error.
  • 1 is assigned to each of the directions not exhibiting the tendency.
  • a high transform efficiency is achieved by classifying the vertical and horizontal directions of the prediction modes other than those in which the groups of reference pixels on two lines are referenced, into two classes depending whether or not the tendency is present, and adaptively applying the 1D transform matrix F or the 1D transform matrix G to each of the vertical and horizontal directions.
  • the image encoding apparatus utilizes the tendency of the intra-prediction to have a decreasing prediction accuracy with increasing distance from the reference pixel as in the case of the first embodiment, while applying the orthogonal transformation and the inverse orthogonal transformation to the prediction modes in a manner varying depending on the number of lines corresponding to the groups of reference pixels.
  • the image encoding apparatus classifies the vertical and horizontal directions of the prediction modes other than those in which the groups of reference pixels on two lines are referenced, into the two classes depending whether or not the tendency is present, and adaptively applies the 1D transform matrix F or the 1D transform matrix G to each of the vertical and horizontal directions.
  • the image encoding apparatus applies the 1D transform matrix H to the prediction modes in which the groups of reference pixels on two lines are referenced.
  • the 1D transform matrix F is generated by pre-designing a common transform basis so as to increase the coefficient density after 1D orthogonal transformation (that is, to reduce the rate of nonzero coefficients in the quantized transform coefficients 121 ) in the direction orthogonal to the line of the group of reference pixels (vertical direction or horizontal direction).
  • the 1D transform matrix G is generated by designing a general-purpose transform matrix having no such nature.
  • the 1D transform matrix H is generated by pre-designing a common transform basis so as to increase, for the prediction error 119 in each prediction mode for which the group of reference pixels on two lines are referenced, the coefficient density after 1D orthogonal transformation (that is, to reduce the rate of nonzero coefficients in the quantized transform coefficients 121 ) in the vertical direction and the horizontal direction.
  • the image encoding apparatus achieves a higher transform efficiency than in the case where fixed orthogonal transformation such as DCT is uniformly applied to the prediction modes.
  • the first to third embodiments prepare two or three types of 1D transform matrices and select one of the 1D transform matrices for vertical transformation (or vertical inverse transformation) and horizontal transformation (or horizontal inverse transformation) according to the prediction mode.
  • the above-described two or three types of 1D transform matrices are illustrative.
  • the coding efficiency can be improved by preparing more transform matrices.
  • the second embodiment and the third embodiment can be combined together to prepare four types of 1D transform matrices.
  • an increased number of types of transform matrices prepared require additional hardware and the like.
  • the balance between the coding efficiency and disadvantages resulting from an increased number of types of transform matrices desirably needs to be taken into account.
  • a fourth embodiment relates to an image decoding apparatus.
  • the image encoding apparatus corresponding to the image decoding apparatus according to the present embodiment is as described in the first embodiment. That is, the image decoding apparatus according to the present embodiment decodes encoded data generated by, for example, the image encoding apparatus according to the first embodiment.
  • the image decoding apparatus includes an input buffer 401 , an entropy decoding unit 402 , a coefficient order control unit 403 , a de-quantization unit 404 , an inverse orthogonal transformation unit 405 , an addition unit 406 , a reference image memory 407 , an intra-prediction unit 408 , an inter-prediction unit 409 , a selection switch 410 , a 1D transform matrix set unit 411 , and an output buffer 412 .
  • the image decoding apparatus in FIG. 22 decodes encoded data accumulated in the input buffer 401 , accumulates a decoded image 419 in the output buffer 412 , and outputs the decoded image 419 as an output image 425 .
  • the encoded data 414 is output by, for example, the image encoding apparatus in FIG. 1 , and temporarily accumulated in the input buffer 401 via an accumulation system or a transmission system (not shown in the drawings).
  • the entropy decoding unit 402 decodes every frame or field based on syntax.
  • the entropy decoding unit 402 subjects each code sequence of the syntax to entropy decoding to reproduce coding parameters for the coding target block such as prediction information 424 including prediction mode information 421 and a quantized transform coefficient sequence 415 .
  • the coding parameters are required for decoding and include the prediction information 424 , information on transform coefficients, and information on quantization.
  • the quantized transform coefficient sequence 415 is input to the coefficient order control unit 403 .
  • the prediction mode information 421 included in the prediction information 424 , is similarly input to the coefficient order control unit 403 .
  • the prediction information 424 is input to the 1D transform matrix set unit 411 and the selection switch 410 .
  • the coefficient order control unit 403 transforms the quantized transform coefficient sequence 415 which is a one-dimensional expression into a quantized transform coefficients 416 which are a two-dimensional expression.
  • the coefficient order control unit 403 then inputs the quantized transform coefficients 416 to the de-quantization unit 404 .
  • the coefficient order control unit 403 will be described below in detail.
  • the de-quantization unit 404 de-quantizes the quantized transform coefficients 416 from the coefficient order control unit 403 to obtain a restored transform coefficients 417 . Specifically, the de-quantization unit 404 carries out de-quantization in accordance with the information on the quantization decoded by the entropy decoding unit 402 . The d quantization unit 404 inputs the restored transform coefficients 417 to the inverse orthogonal transformation unit 405 .
  • the inverse orthogonal transformation unit 405 carries out an inverse orthogonal transformation corresponding to the orthogonal transformation performed on the encoding side, on the restored transform coefficients 417 from the de-quantization unit 404 to obtain a restored prediction error 418 .
  • the inverse orthogonal transformation unit 405 inputs the restored prediction error 418 to the addition unit 406 .
  • the inverse orthogonal transformation unit 405 according to the present embodiment is a component which is substantially the same as or similar to the inverse orthogonal transformation unit 105 in FIG. 3 and will thus not be described below in detail.
  • the inverse orthogonal transformation unit 405 according to the present embodiment utilizes the 1D transform matrix A and 1D transform matrix B which are common to the inverse orthogonal transformation unit in FIG. 3 .
  • the restored transform coefficients 122 , 1D transform matrix set information 129 , and restored prediction error 123 in FIG. 3 correspond to the restored transform coefficients 417 , 1D transform matrix set information 422 , and restored prediction error 418 according to the present embodiment, respectively.
  • the addition unit 406 adds the restored prediction error 418 and the corresponding prediction image 423 together to generate the decoded image 419 .
  • the decoded image 419 is temporarily accumulated in the output buffer 412 for the output image 425 and also saved to the reference image memory 407 for a reference image 420 .
  • the decoded image 419 saved to the reference image memory 407 is referenced in frame or field units as necessary by the intra-prediction unit 408 and the inter-prediction unit 409 as the reference image 420 .
  • the decoded image 419 temporarily accumulated in the output buffer 412 is output at an output timing managed by the decoding control unit 413 .
  • the intra-prediction unit 408 , the inter-prediction unit 409 , and the selection switch 410 are components which are substantially the same as or similar to the intra-prediction unit 108 , inter-prediction unit 109 , and selection switch 110 in FIG. 1 and will thus not be described below in detail.
  • the decoding control unit 413 controls the components of the image decoding apparatus in FIG. 22 . Specifically, the decoding control unit 413 performs various control operations for a decoding process including the above-described operations.
  • the 1D transform matrix set unit 411 generates the 1D transform matrix set information 422 based on the prediction mode information included in the prediction information 424 from the entropy decoding unit 402 .
  • the 1D transform matrix set unit 411 inputs the 1D transform matrix set information 422 to the inverse orthogonal transformation unit 405 .
  • the 1D transform matrix set unit 411 is a component which is substantially the same as or similar to the 1D transform matrix set unit 112 according to the first embodiment and will thus not be described below in detail. That is, the 1D transform matrix set unit 411 according to the present embodiment uses, for example, the tables in FIG. 4A , FIG. 4B , FIG. 4C , FIG. 4D , and FIG. 4E to generate the 1D transform matrix set information 422 .
  • the prediction information 126 and 1D transform matrix set information 129 according to the first embodiment correspond to the prediction information 424 and 1D transform matrix set information 422 according to the present embodiment, respectively.
  • the image decoding apparatus in FIG. 22 utilizes a syntax which is the same as or similar to the syntax described with reference to FIG. 11 , FIG. 12 , FIG. 13 , and FIG. 14 .
  • the detailed description of the syntax is omitted.
  • the coefficient order control unit 403 will be described below in detail.
  • the coefficient order control unit 403 transforms the quantized transform coefficient sequence 415 , which is a one-dimensional expression, into the quantized transform coefficients 416 , which are a two-dimensional expression, by arranging the elements of the quantized transform coefficient sequence 415 in a predetermined order (that is, the order corresponding to the order used on the encoding side).
  • a predetermined order that is, the order corresponding to the order used on the encoding side.
  • the coefficient order control unit 403 can perform common 1D-2D transformation regardless of the prediction mode.
  • the coefficient order control unit 403 can utilize an inverse zigzag scan as is the case with H.264.
  • the inverse zigzag scan is a 1D-2D transformation corresponding to the above-described zigzag scan.
  • the coefficient order control unit 403 can also perform individual 1D-2D transformations for the respective prediction modes.
  • the coefficient order control unit 403 performing such an operation is illustrated in FIG. 23A .
  • the coefficient order control unit 403 includes a selection switch 1001 and individual 1D-2D transformation units 1002 , . . . , and 1010 for the respective prediction modes.
  • the selection switch 1001 leads the quantized transform coefficient sequence 415 to the 1D-2D transformation unit (one of 1002 , . . .
  • a prediction mode index of 0 allows the selection switch 1001 to lead the quantized transform coefficient sequence 415 to the 1D-2D transformation unit 1002 .
  • the prediction modes and the 1D-2D transformation units are on a one-to-one correspondence.
  • the quantized transform coefficient sequence 415 is led to one 1D-2D transformation unit corresponding to the prediction mode.
  • the quantized transform coefficient sequence 415 is then transformed into the quantized transform coefficients 416 .
  • the coefficient order control unit 403 may also dynamically update the scan order in the 1D-2D transformation in a manner corresponding to the manner used on the encoding side.
  • the coefficient order control unit 403 performing such an operation is illustrated in FIG. 23B .
  • the coefficient order control unit 403 includes the selection switch 1001 , the individual 1D-2D transformation units 1002 , . . . , and 1010 for the respective nine types of prediction modes, an occurrence frequency count unit 1011 , and a coefficient order update unit 1012 .
  • the selection switch 1001 is as described with reference to FIG. 23A .
  • the occurrence frequency count unit 1011 creates a histogram of the number of occurrences of nonzero coefficients in each element of the quantized transform coefficient sequence 416 .
  • the occurrence frequency count unit 1011 inputs the created histogram 1013 to the coefficient order update unit 1012 .
  • the coefficient order update unit 1012 updates the order of coefficients at a predetermined timing based on the histogram 1013 .
  • the timing may be, for example, a timing when a decoding process carried out on a coding tree unit is finished or a timing when a decoding process carried out on one line in the coding tree unit is finished.
  • the coefficient order control unit 1012 references the histogram 1013 to update the order of coefficients for a prediction mode with an element for which the counted number of occurrences of nonzero coefficients is equal to or larger than a threshold.
  • the coefficient order control unit 1012 performs the updating for a prediction mode with an element for which the counted number of occurrences of nonzero coefficients is 16 or more. Setting a threshold for the number of occurrences allows the order of coefficients to be globally updated, thus avoiding convergence to a local optimum solution.
  • the coefficient order control unit 1012 sorts, for the updating-target prediction mode, the elements in order of decreasing occurrence frequency of nonzero coefficients.
  • the sorting can be achieved in accordance with an existing algorithm, for example, bubble sort or quick sort.
  • the coefficient order control unit 1012 inputs coefficient order update information 1014 indicative of the sorted order of the elements to the 1D-2D transformation unit corresponding to the updating-target prediction mode.
  • the 1D-2D transformation unit carries out a 1D-2D transformation in accordance with the updated scan order. If the scan order is dynamically updated, initial scan orders for the 1D-2D transform units need to be set which correspond to the scan order on the encoding side.
  • the image decoding apparatus includes an inverse orthogonal transformation unit which is the same as or similar to the inverse orthogonal transformation unit of the image encoding apparatus according to the first embodiment.
  • the image decoding apparatus according to the present embodiment exerts effects which are the same as or similar to the effects of the image encoding apparatus according to the first embodiment.
  • An image decoding apparatus is different from the image decoding apparatus according to the fourth embodiment in the details of the inverse orthogonal transformation.
  • the same components of the present embodiment as the corresponding components of the fourth embodiment are hereinafter denoted by the same reference numerals, and mainly different components will be described below.
  • the image encoding apparatus corresponding to the image decoding apparatus according to the present embodiment is as described in the second embodiment.
  • the inverse orthogonal transformation unit 405 according to the present embodiment is a component which is substantially the same as or similar to the inverse orthogonal transformation unit 105 in FIG. 17 and will thus not be described below in detail.
  • the inverse orthogonal transformation unit 405 according to the present embodiment utilizes the 1D transform matrix C, 1D transform matrix D, and 1D transform matrix E which are common to the inverse orthogonal transformation unit 105 in FIG. 17 .
  • the restored transform coefficients 122 , 1D transform matrix set information 129 , and restored prediction error 123 in FIG. 17 correspond to the restored transform coefficients 417 , the 1D transform matrix set information 422 , and a restored prediction error signal 418 according to the present embodiment, respectively.
  • the 1D transform matrix set unit 411 according to the present embodiment is a component which is substantially the same as or similar to the 1D transform matrix set unit 112 according to the second embodiment and will thus not described below in detail. That is, the 1D transform matrix set unit 411 according to the present embodiment utilizes, for example, the tables in FIG. 18A , FIG. 18B , FIG. 18C , FIG. 18D , and FIG. 18E to generate the 1D transform matrix set information 422 .
  • the prediction information 126 and 1D transform matrix set information 129 according to the second embodiment correspond to the prediction information 424 and 1D transform matrix set information 422 according to the present embodiment, respectively.
  • the image decoding apparatus includes an inverse orthogonal transformation unit which is the same as or similar to the inverse orthogonal transformation unit of the image encoding apparatus according to the second embodiment.
  • the image decoding apparatus according to the present embodiment exerts effects which are the same as or similar to the effects of the image encoding apparatus according to the second embodiment.
  • An image decoding apparatus is different from the image decoding apparatuses according to the fourth embodiment and the fifth embodiment in the details of the inverse orthogonal transformation.
  • the same components of the present embodiment as the corresponding components of the fourth embodiment or the fifth embodiment are hereinafter denoted by the same reference numerals, and mainly different components will be described below.
  • the image encoding apparatus corresponding to the image decoding apparatus according to the present embodiment is as described in the third embodiment.
  • the inverse orthogonal transformation unit 405 according to the present embodiment is a component which is substantially the same as or similar to the inverse orthogonal transformation unit 105 in FIG. 20 and will thus not be described below in detail.
  • the inverse orthogonal transformation unit 405 according to the present embodiment utilizes the 1D transform matrix F, 1D transform matrix G, and 1D transform matrix H which are common to the inverse orthogonal transformation unit 105 in FIG. 20 .
  • the restored transform coefficients 122 , 1D transform matrix set information 129 , and restored prediction error 123 in FIG. 20 correspond to the restored transform coefficients 417 , 1D transform matrix set information 422 , and restored prediction error signal 418 according to the present embodiment, respectively.
  • the 1D transform matrix set unit 411 according to the present embodiment is a component which is substantially the same as or similar to the 1D transform matrix set unit 112 according to the third embodiment and will thus not described below in detail. That is, the 1D transform matrix set unit 411 according to the present embodiment utilizes, for example, the tables in FIG. 21A , FIG. 21B , FIG. 21C , FIG. 21D , and FIG. 21E to generate the 1D transform matrix set information 422 .
  • the prediction information 126 and 1D transform matrix set information 129 according to the third embodiment correspond to the prediction information 424 and 1D transform matrix set information 422 according to the present embodiment, respectively.
  • the image decoding apparatus includes an inverse orthogonal transformation unit which is the same as or similar to the inverse orthogonal transformation unit of the image encoding apparatus according to the third embodiment.
  • the image decoding apparatus according to the present embodiment exerts effects which are the same as or similar to the effects of the image encoding apparatus according to the third embodiment.
  • the fourth to sixth embodiments prepare two or three types of 1D transform matrices and select one of the 1D transform matrices for vertical inverse transformation and horizontal inverse transformation according to the prediction mode.
  • the above-described two or three types of 1D transform matrices are illustrative.
  • the coding efficiency can be improved by preparing more transform matrices.
  • the fifth embodiment and the sixth embodiment can be combined together to prepare four types of 1D transform matrices.
  • an increased number of types of transform matrices prepared require additional hardware and the like.
  • the balance between the coding efficiency and disadvantages resulting from an increased number of types of transform matrices desirably needs to be taken into account.
  • the first to sixth embodiments describe the example in which a frame is divided into rectangular blocks with a 16 ⁇ 16 pixel size or the like and in which encoding or decoding is carried out on each block in order from upper left to lower right in the picture (see FIG. 6A ).
  • the encoding order and the decoding order are not limited to this example.
  • the encoding and decoding may be carried out in order from lower right to upper left or spirally from the center to the end of the picture.
  • the encoding and decoding may be carried out in order from upper right to lower left or spirally from the end to the center of the picture.
  • the first to sixth embodiments illustrate the prediction target block sizes such as the 4 ⁇ 4 pixel block, the 8 ⁇ 8 pixel block, and the 16 ⁇ 16 pixel size for description.
  • the prediction target block need not necessarily have a uniform block shape.
  • examples of the prediction target block size may include a 16 ⁇ 8 pixel block, an 8 ⁇ 16 pixel block, an 8 ⁇ 4 pixel block, or a 4 ⁇ 8 pixel block.
  • the block size is desirably selected taking into account the balance between the amount of code for the division information and the quality of the local decoded image or decoded image.
  • the first to sixth embodiments comprehensively describe the color signal components without distinguishing luminance signals from color difference signals.
  • the same prediction scheme or different prediction schemes may be used. If different prediction schemes are used for the luminance signal and for the color difference signal, the prediction scheme selected for the color difference signal can be encoded or decoded in a manner similar to the manner for the luminance signal.
  • the first to sixth embodiments comprehensively describe the color signal components without distinguishing the luminance signal from the color difference signal.
  • the same orthogonal transformation scheme or different orthogonal transformation schemes may be used. If different orthogonal transformation schemes are used for the luminance signal and for the color difference signal, the orthogonal transformation scheme selected for the color difference signal can be encoded or decoded in a manner similar to the manner for the luminance signal.
  • each of the embodiments achieves efficient orthogonal transformation and inverse orthogonal transformation while alleviating difficulties in the hardware implementation and the software implementation. Hence, each embodiment improves the coding efficiency and thus subjective image quality.
  • a program can be provided which is stored in a computer readable storage medium and configured to implement the processing according to each of the embodiments.
  • the storage medium may be in any storage format provided that the program can be stored in the storage medium and read from the storage medium by a computer; the storage medium may be a magnetic disk, an optical disc (CD-ROM, CD-R, DVD, or the like), a magneto-optical disk (MO or the like), a semiconductor memory, or the like.
  • the program configured to implement the processing according to each of the embodiments may be stored in a computer (server) connected to a network such as the Internet.
  • the program may be downloaded into a computer (client) via the network.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Medicines Containing Antibodies Or Antigens For Use As Internal Diagnostic Agents (AREA)
US13/740,841 2010-07-15 2013-01-14 Image encoding apparatus and image decoding apparatus Abandoned US20130128972A1 (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
US14/186,959 US9706226B2 (en) 2010-07-15 2014-02-21 Image encoding apparatus and image decoding apparatus employing intra preciction and direction transform matrix
US14/186,990 US9426467B2 (en) 2010-07-15 2014-02-21 Image decoding method using a vertical inverse transform matrix and a horizontal inverse transform matrix

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2010/062007 WO2012008039A1 (ja) 2010-07-15 2010-07-15 画像符号化方法及び画像復号化方法

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2010/062007 Continuation WO2012008039A1 (ja) 2010-07-15 2010-07-15 画像符号化方法及び画像復号化方法

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US14/186,990 Continuation US9426467B2 (en) 2010-07-15 2014-02-21 Image decoding method using a vertical inverse transform matrix and a horizontal inverse transform matrix
US14/186,959 Continuation US9706226B2 (en) 2010-07-15 2014-02-21 Image encoding apparatus and image decoding apparatus employing intra preciction and direction transform matrix

Publications (1)

Publication Number Publication Date
US20130128972A1 true US20130128972A1 (en) 2013-05-23

Family

ID=45469063

Family Applications (3)

Application Number Title Priority Date Filing Date
US13/740,841 Abandoned US20130128972A1 (en) 2010-07-15 2013-01-14 Image encoding apparatus and image decoding apparatus
US14/186,990 Active 2030-10-20 US9426467B2 (en) 2010-07-15 2014-02-21 Image decoding method using a vertical inverse transform matrix and a horizontal inverse transform matrix
US14/186,959 Active US9706226B2 (en) 2010-07-15 2014-02-21 Image encoding apparatus and image decoding apparatus employing intra preciction and direction transform matrix

Family Applications After (2)

Application Number Title Priority Date Filing Date
US14/186,990 Active 2030-10-20 US9426467B2 (en) 2010-07-15 2014-02-21 Image decoding method using a vertical inverse transform matrix and a horizontal inverse transform matrix
US14/186,959 Active US9706226B2 (en) 2010-07-15 2014-02-21 Image encoding apparatus and image decoding apparatus employing intra preciction and direction transform matrix

Country Status (10)

Country Link
US (3) US20130128972A1 (de)
EP (3) EP2595385A4 (de)
JP (1) JP5259879B2 (de)
CN (1) CN103493491A (de)
AU (1) AU2010357291B2 (de)
BR (2) BR122020008881B1 (de)
CA (2) CA2921057C (de)
MX (2) MX2013000516A (de)
RU (2) RU2528144C1 (de)
WO (1) WO2012008039A1 (de)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20140301465A1 (en) * 2013-04-05 2014-10-09 Texas Instruments Incorporated Video Coding Using Intra Block Copy
US20150139308A1 (en) * 2011-10-18 2015-05-21 Kt Corporation Method for encoding image, method for decoding image, image encoder, and image decoder
US20180091828A1 (en) * 2015-05-29 2018-03-29 SZ DJI Technology Co., Ltd. System and method for video processing
US20180302629A1 (en) * 2015-10-30 2018-10-18 Sony Corporation Image processing apparatus and method
US10368086B2 (en) * 2014-05-19 2019-07-30 Huawei Technologies Co., Ltd. Image coding/decoding method, device, and system
CN111429555A (zh) * 2020-03-24 2020-07-17 谷元(上海)文化科技有限责任公司 一种动画人物表情转换视觉捕捉方法
CN111881438A (zh) * 2020-08-14 2020-11-03 支付宝(杭州)信息技术有限公司 基于隐私保护进行生物特征识别的方法、装置及电子设备
US20210105477A1 (en) * 2017-04-13 2021-04-08 Lg Electronics Inc. Image encoding/decoding method and device therefor
US20220167018A1 (en) * 2017-08-04 2022-05-26 Lg Electronics Inc. Method and apparatus for configuring transform for video compression
US11909979B2 (en) 2019-07-10 2024-02-20 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method for colour component prediction, encoder, decoder and storage medium

Families Citing this family (17)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101452860B1 (ko) 2009-08-17 2014-10-23 삼성전자주식회사 영상의 부호화 방법 및 장치, 영상 복호화 방법 및 장치
AU2010357291B2 (en) 2010-07-15 2015-01-15 Kabushiki Kaisha Toshiba Image encoding method and image decoding method
JP5537695B2 (ja) * 2013-04-10 2014-07-02 株式会社東芝 画像復号化装置、方法およびプログラム
JP5535361B2 (ja) * 2013-04-10 2014-07-02 株式会社東芝 画像符号化装置、方法およびプログラム
US10405000B2 (en) * 2014-11-21 2019-09-03 Vid Scale, Inc. One-dimensional transform modes and coefficient scan order
CN108605129A (zh) * 2016-01-28 2018-09-28 日本放送协会 编码装置、解码装置以及程序
FR3050598B1 (fr) * 2016-04-26 2020-11-06 Bcom Procede de decodage d'une image numerique, procede de codage, dispositifs, et programmes d'ordinateurs associes
US10972733B2 (en) * 2016-07-15 2021-04-06 Qualcomm Incorporated Look-up table for enhanced multiple transform
CN106254883B (zh) * 2016-08-02 2021-01-22 海信视像科技股份有限公司 一种视频解码中的反变换方法和装置
RU2645290C1 (ru) * 2017-03-27 2018-02-19 федеральное государственное казенное военное образовательное учреждение высшего образования "Военная академия связи имени Маршала Советского Союза С.М. Буденного" Министерства обороны Российской Федерации Способ кодирования оцифрованных изображений с использованием адаптивного ортогонального преобразования
MX2020006313A (es) 2017-12-15 2020-09-18 Lg Electronics Inc Metodo de codificacion de imagenes basado en transformacion secundaria no separable y dispositivo para el mismo.
EP3800883A4 (de) * 2018-06-25 2021-07-07 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Verfahren und vorrichtung zur intraframe-vorhersage
US10904563B2 (en) * 2019-01-02 2021-01-26 Tencent America LLC Method and apparatus for improved zero out transform
CN113302934B (zh) * 2019-12-23 2022-10-18 Oppo广东移动通信有限公司 图像预测方法、编码器、解码器以及存储介质
US11418795B2 (en) * 2020-08-05 2022-08-16 University Of Electronic Science And Technology Of China Temporal domain rate distortion optimization based on video content characteristic and QP-λcorrection
CN114580649B (zh) * 2022-03-09 2024-07-19 北京百度网讯科技有限公司 消除量子泡利噪声的方法及装置、电子设备和介质
CN114580648B (zh) * 2022-03-09 2024-07-16 北京百度网讯科技有限公司 消除广义振幅阻尼噪声的方法及装置、电子设备和介质

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030128887A1 (en) * 2002-01-10 2003-07-10 Nec Corporation Two-dimensional orthogonal transformation and quantization method and its device and program
US20060008164A1 (en) * 2004-07-03 2006-01-12 Microsoft Corporation System and method for image coding employing a hybrid directional prediction and wavelet lifting
US20110170594A1 (en) * 2010-01-14 2011-07-14 Madhukar Budagavi Method and System for Intracoding in Video Encoding
US20110249724A1 (en) * 2010-04-09 2011-10-13 Sony Corporation Directional discrete wavelet transform (ddwt) for video compression applications

Family Cites Families (16)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP3050736B2 (ja) * 1993-12-13 2000-06-12 シャープ株式会社 動画像符号化装置
JP2900998B2 (ja) 1997-07-31 1999-06-02 日本ビクター株式会社 ブロック間内挿予測符号化装置、復号化装置、符号化方法及び復号化方法
US6707948B1 (en) * 1997-11-17 2004-03-16 The Regents Of The University Of California Image compression for memory-constrained decoders
US6990506B2 (en) * 2000-12-13 2006-01-24 Sharp Laboratories Of America, Inc. Integer cosine transform matrix for picture coding
RU2329615C2 (ru) * 2003-12-01 2008-07-20 Самсунг Электроникс Ко., Лтд. Способ масштабируемого кодирования и декодирования видеосигнала и устройство для его осуществления
JP4591657B2 (ja) * 2003-12-22 2010-12-01 キヤノン株式会社 動画像符号化装置及びその制御方法、プログラム
US7778327B2 (en) * 2005-02-08 2010-08-17 Texas Instruments Incorporated H.264 quantization
CA2631336A1 (en) * 2005-11-30 2007-06-07 Kabushiki Kaisha Toshiba Image encoding/image decoding method, image encoding/image decoding apparatus
US8619853B2 (en) 2007-06-15 2013-12-31 Qualcomm Incorporated Separable directional transforms
RU2479940C2 (ru) * 2007-10-15 2013-04-20 Ниппон Телеграф Энд Телефон Корпорейшн Устройство кодирования и декодирования изображения, способы кодирования и декодирования изображения, их программы и носитель записи, записанный программами
JP2009272727A (ja) * 2008-04-30 2009-11-19 Toshiba Corp 予測誤差の方向性に基づく変換方法、画像符号化方法及び画像復号化方法
WO2011083573A1 (ja) 2010-01-07 2011-07-14 株式会社 東芝 動画像符号化装置及び動画像復号化装置
WO2011112239A1 (en) * 2010-03-10 2011-09-15 Thomson Licensing Methods and apparatus for constrained transforms for video coding and decoding having transform selection
AU2010357291B2 (en) 2010-07-15 2015-01-15 Kabushiki Kaisha Toshiba Image encoding method and image decoding method
JP5535361B2 (ja) 2013-04-10 2014-07-02 株式会社東芝 画像符号化装置、方法およびプログラム
JP5622954B2 (ja) 2014-04-17 2014-11-12 株式会社東芝 画像復号化装置、方法およびプログラム

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20030128887A1 (en) * 2002-01-10 2003-07-10 Nec Corporation Two-dimensional orthogonal transformation and quantization method and its device and program
US20060008164A1 (en) * 2004-07-03 2006-01-12 Microsoft Corporation System and method for image coding employing a hybrid directional prediction and wavelet lifting
US20110170594A1 (en) * 2010-01-14 2011-07-14 Madhukar Budagavi Method and System for Intracoding in Video Encoding
US20110249724A1 (en) * 2010-04-09 2011-10-13 Sony Corporation Directional discrete wavelet transform (ddwt) for video compression applications

Cited By (21)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10264283B2 (en) 2011-10-18 2019-04-16 Kt Corporation Method and apparatus for decoding a video signal using adaptive transform
US20150139308A1 (en) * 2011-10-18 2015-05-21 Kt Corporation Method for encoding image, method for decoding image, image encoder, and image decoder
US10575015B2 (en) * 2011-10-18 2020-02-25 Kt Corporation Method and apparatus for decoding a video signal using adaptive transform
US20230085594A1 (en) * 2013-04-05 2023-03-16 Texas Instruments Incorporated Video Coding Using Intra Block Copy
US20140301465A1 (en) * 2013-04-05 2014-10-09 Texas Instruments Incorporated Video Coding Using Intra Block Copy
US12081787B2 (en) * 2013-04-05 2024-09-03 Texas Instruments Incorporated Video coding using intra block copy
US10904551B2 (en) * 2013-04-05 2021-01-26 Texas Instruments Incorporated Video coding using intra block copy
US11533503B2 (en) 2013-04-05 2022-12-20 Texas Instruments Incorporated Video coding using intra block copy
US10368086B2 (en) * 2014-05-19 2019-07-30 Huawei Technologies Co., Ltd. Image coding/decoding method, device, and system
US20180091828A1 (en) * 2015-05-29 2018-03-29 SZ DJI Technology Co., Ltd. System and method for video processing
US10893300B2 (en) * 2015-05-29 2021-01-12 SZ DJI Technology Co., Ltd. System and method for video processing
US20180302629A1 (en) * 2015-10-30 2018-10-18 Sony Corporation Image processing apparatus and method
US11729390B2 (en) * 2017-04-13 2023-08-15 Lg Electronics Inc. Image encoding/decoding method and device therefor
US20210105477A1 (en) * 2017-04-13 2021-04-08 Lg Electronics Inc. Image encoding/decoding method and device therefor
US20220167018A1 (en) * 2017-08-04 2022-05-26 Lg Electronics Inc. Method and apparatus for configuring transform for video compression
AU2021258067B2 (en) * 2017-08-04 2023-02-16 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method and apparatus for configuring transform for video compression
US11825119B2 (en) * 2017-08-04 2023-11-21 Lg Electronics Inc. Method and apparatus for configuring transform for video compression
US11909979B2 (en) 2019-07-10 2024-02-20 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method for colour component prediction, encoder, decoder and storage medium
US11930181B2 (en) * 2019-07-10 2024-03-12 Guangdong Oppo Mobile Telecommunications Corp., Ltd. Method for colour component prediction, encoder, decoder and storage medium
CN111429555A (zh) * 2020-03-24 2020-07-17 谷元(上海)文化科技有限责任公司 一种动画人物表情转换视觉捕捉方法
CN111881438A (zh) * 2020-08-14 2020-11-03 支付宝(杭州)信息技术有限公司 基于隐私保护进行生物特征识别的方法、装置及电子设备

Also Published As

Publication number Publication date
JP5259879B2 (ja) 2013-08-07
US20140169460A1 (en) 2014-06-19
US9706226B2 (en) 2017-07-11
JPWO2012008039A1 (ja) 2013-09-05
MX2013000516A (es) 2013-09-02
BR122020008881B1 (pt) 2021-08-10
CN103493491A (zh) 2014-01-01
CA2805248A1 (en) 2012-01-19
WO2012008039A1 (ja) 2012-01-19
CA2921057A1 (en) 2012-01-19
BR112013000865A2 (pt) 2016-05-17
AU2010357291A1 (en) 2013-02-14
RU2528144C1 (ru) 2014-09-10
RU2580088C2 (ru) 2016-04-10
US20140169461A1 (en) 2014-06-19
MX338867B (es) 2016-05-03
AU2010357291B2 (en) 2015-01-15
RU2014127123A (ru) 2016-02-10
CA2805248C (en) 2016-04-26
EP2595385A4 (de) 2014-06-25
EP2747435A1 (de) 2014-06-25
CA2921057C (en) 2017-10-17
EP2747436A1 (de) 2014-06-25
EP2595385A1 (de) 2013-05-22
US9426467B2 (en) 2016-08-23
BR112013000865B1 (pt) 2021-08-10

Similar Documents

Publication Publication Date Title
US9426467B2 (en) Image decoding method using a vertical inverse transform matrix and a horizontal inverse transform matrix
CN113873241B (zh) 用于对视频进行解码的方法和用于对视频进行编码的方法
US9294768B2 (en) Moving-picture encoding apparatus and moving-picture decoding apparatus
US20230353771A1 (en) Method and apparatus for processing video signal
CN108141589B (zh) 图像编译系统中对图像进行滤波的方法和装置
CN111448798A (zh) 基于块形状的视频编码和解码的方法和装置
AU2015201843B2 (en) Image encoding method and image decoding method
JP5622954B2 (ja) 画像復号化装置、方法およびプログラム
JP5535361B2 (ja) 画像符号化装置、方法およびプログラム
JP5537695B2 (ja) 画像復号化装置、方法およびプログラム
JP6310034B2 (ja) 復号装置、復号方法および復号プログラム
JP6042478B2 (ja) 画像復号化装置
WO2022218280A1 (en) Transforms and sign prediction
RU2631992C2 (ru) Способ кодирования изображения и способ декодирования изображения
JP5925855B2 (ja) 画像復号化装置、方法およびプログラム、第１のプログラムおよび第２のプログラム、サーバシステムならびにダウンロード制御方法

Legal Events

Date Code Title Description
AS Assignment

Owner name: KABUSHIKI KAISHA TOSHIBA, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:YAMAGUCHI, JUN;TANIZAWA, AKIYUKI;REEL/FRAME:029623/0919

Effective date: 20130108

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION