US20130028326A1 - Moving image encoding device and moving image decoding device - Google Patents

Moving image encoding device and moving image decoding device Download PDF

Info

Publication number
US20130028326A1
US20130028326A1 US13/639,134 US201113639134A US2013028326A1 US 20130028326 A1 US20130028326 A1 US 20130028326A1 US 201113639134 A US201113639134 A US 201113639134A US 2013028326 A1 US2013028326 A1 US 2013028326A1
Authority
US
United States
Prior art keywords
unit
prediction
image
encoding
transformation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/639,134
Other languages
English (en)
Inventor
Yoshimi Moriya
Shunichi Sekiguchi
Kazuo Sugimoto
Kohtaro Asai
Tokumichi Murakami
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Mitsubishi Electric Corp
Original Assignee
Mitsubishi Electric Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Mitsubishi Electric Corp filed Critical Mitsubishi Electric Corp
Assigned to MITSUBISHI ELECTRIC CORPORATION reassignment MITSUBISHI ELECTRIC CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ASAI, KOHTARO, MORIYA, YOSHIMI, MURAKAMI, TOKUMICHI, SEKIGUCHI, SHUNICHI, SUGIMOTO, KAZUO
Publication of US20130028326A1 publication Critical patent/US20130028326A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/109—Selection of coding mode or of prediction mode among a plurality of temporal predictive coding modes
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/119—Adaptive subdivision aspects, e.g. subdivision of a picture into rectangular or non-rectangular coding blocks
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/13—Adaptive entropy coding, e.g. adaptive variable length coding [AVLC] or context adaptive binary arithmetic coding [CABAC]
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/46—Embedding additional information in the video signal during the compression process
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/567—Motion estimation based on rate distortion criteria
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/593—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving spatial prediction techniques
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/90—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using coding techniques not provided for in groups H04N19/10-H04N19/85, e.g. fractals
    • H04N19/91—Entropy coding, e.g. variable length coding [VLC] or arithmetic coding
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/107—Selection of coding mode or of prediction mode between spatial and temporal predictive coding, e.g. picture refresh
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/12—Selection from among a plurality of transforms or standards, e.g. selection between discrete cosine transform [DCT] and sub-band transform or selection between H.263 and H.264
    • H04N19/122—Selection of transform size, e.g. 8x8 or 2x4x8 DCT; Selection of sub-band transforms of varying structure or type
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/157—Assigned coding mode, i.e. the coding mode being predefined or preselected to be further used for selection of another element or parameter
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/523—Motion estimation or motion compensation with sub-pixel accuracy

Definitions

  • the present invention relates to a moving image encoding device which divides a moving image into predetermined areas and encodes the moving image in units of one area, and a moving image decoding device which decodes an encoded moving image in units of one predetermined area.
  • a method of defining block data (referred to as “macroblock” from here on) as a unit, the block data being a combination of 16 ⁇ 16 pixels of brightness signal and 8 ⁇ 8 pixels of color difference signal corresponding to the 16 ⁇ 16 pixels of brightness signal, and compressing each frame of a video signal in units of block data in accordance with a motion compensation technique, and an orthogonal transformation/transform coefficient quantization technique is used.
  • the motion compensation technique is used to reduce the redundancy of a signal in a time direction for each macroblock by using a high correlation existing between video frames.
  • an already-encoded frame which has been previously encoded is stored in a memory as a reference image, and a block area which provides the smallest difference in electric power between the block area itself and the current macroblock which is a target block for the motion-compensated prediction is searched for through a search range predetermined in the reference image, and a spatial displacement between the spatial position of the current macroblock and the spatial position of the block area in the reference image which is determined as the result of the search is then encoded as a motion vector.
  • a differential signal which is acquired by subtracting a prediction signal acquired as the result of the above-mentioned motion-compensated prediction from the current macroblock is orthogonal transformed and quantized so that the amount of information is compressed.
  • each block which is used as a unit for motion-compensated prediction has a minimum size of 8 ⁇ 8 pixels, and DCT (discrete cosine transform) having a 8 ⁇ 8 pixel size is used also for orthogonal transformation.
  • DCT discrete cosine transform
  • MPEG-4 AVC Moving Picture Experts Group-4 Advanced Video Coding
  • a motion-compensated prediction with a block size smaller than 8 ⁇ 8 pixels is prepared in order to efficiently carry out encoding on even an area, such as a boundary between objects, having a small correlation between pixels in a spatial direction.
  • the compression and encoding can be carried out by adaptively switching between 8 ⁇ 8-pixel DCT having integer pixel accuracy and 4 ⁇ 4-pixel DCT having integer pixel accuracy on a per-macroblock basis.
  • patent reference 1 a device which switches between macroblock sizes in accordance with the resolution or the contents of an image is disclosed (for example, refer to patent reference 1).
  • the moving image encoding device disclosed by patent reference 1 can carry out compression and encoding by switching between selectable orthogonal transformation block sizes or between selectable sets of orthogonal transformation block sizes in accordance with the macroblock size.
  • a problem with the conventional international standard video image encoding methods and the invention disclosed by patent reference 1 is, however, that because transformation cannot be carried out by switching among a plurality of orthogonal transformation block sizes within each macroblock, the encoding efficiency is reduced particularly when an object having a different movement or a different pattern exists in a macroblock.
  • the present invention is made in order to solve the above-mentioned problem, and it is therefore an object of the present invention to provide a moving image encoding device which can carry out compression and encoding by adaptively switching among orthogonal transformation block sizes for each area which is a unit for motion-compensated prediction in each macroblock, and a moving image decoding device.
  • a moving image encoding device including an encoding controlling unit for selecting a certain transformation block size from a set of transformation block sizes predetermined in accordance with a block size of a block image to notify the transformation block size selected thereby to a transformation/quantization unit, in which the transformation/quantization unit divides a prediction difference signal into blocks having the transformation block size notified thereto from the encoding controlling unit, and carries out a transformation and quantization process on each of the blocks to generate compressed data.
  • an inverse quantization/inverse transformation unit determines a transformation block size on the basis of a decoded encoding mode and transformation block size information included in compression parameters, and carries out an inverse transformation and inverse quantization process on the compressed data in units of one block having the transformation block size.
  • a certain transformation block size is selected from among a set of transformation block sizes which are predetermined in accordance with the block size of a block image, and a prediction difference signal is divided into blocks having the transformation block size and a transformation and quantization process is carried out on each of the blocks to generate compressed data
  • a moving image encoding device and a moving image decoding device which can carry out compression and encoding by adaptively switching among the transformation block sizes for each area which is a unit for motion-compensated prediction in each macroblock can be provided.
  • FIG. 1 is a block diagram showing the structure of a moving image encoding device in accordance with Embodiment 1 of the present invention
  • FIG. 2A is a view showing an example of encoding modes for pictures in each of which predictive encoding in a time direction is carried out;
  • FIG. 2B is a view showing another example of the encoding modes for pictures in each of which predictive encoding in a time direction is carried out;
  • FIG. 3 is a block diagram showing the internal structure of a motion-compensated prediction unit of the moving image encoding device in accordance with Embodiment 1;
  • FIG. 4 is a view explaining a determining method of determining a predicted value of a motion vector in accordance with an encoding mode
  • FIG. 5 is a view showing an example of adaptation of a transformation block size in accordance with an encoding mode
  • FIG. 6 is a view showing another example of the adaptation of a transformation block size in accordance with an encoding mode
  • FIG. 7 is a block diagram showing the internal structure of a transformation/quantization unit of the moving image encoding device in accordance with Embodiment 1;
  • FIG. 8 is a block diagram showing the structure of a moving image decoding device in accordance with Embodiment 1 of the present invention.
  • FIG. 9 is a block diagram showing the internal structure of a variable length encoding unit of a moving image encoding device in accordance with Embodiment 2 of the present invention.
  • FIG. 10 is a view showing an example of a binarization table in a state in which the binarization table is yet to be updated
  • FIG. 11 is a view showing an example of a probability table
  • FIG. 12 is a view showing an example of a state transition table
  • FIG. 13 is a view explaining a procedure of generating context identification information
  • FIG. 13( a ) is a view showing the binarization table in binary tree representation
  • FIG. 13( b ) is a view showing a positional relationship between a macroblock to be encoded and peripheral blocks;
  • FIG. 14 is a view showing an example of the binarization table in a state in which the binarization table has been updated
  • FIG. 15 is a block diagram showing the internal structure of a variable length decoding unit of a moving image decoding device in accordance with Embodiment 2 of the present invention.
  • FIG. 16 is a block diagram showing the internal structure of an interpolated image generating unit with which a motion-compensated prediction unit of a moving image encoding device in accordance with Embodiment 3 of the present invention is provided.
  • a moving image encoding device which carries out a motion-compensated prediction between adjacent frames by using each frame image of a video image as an input and carries out a compression process using orthogonal transformation and quantization on an acquired prediction difference signal, and, after that, carries out variable length encoding to generate a bitstream
  • an moving image decoding device which decodes the bitstream
  • FIG. 1 is a block diagram showing the structure of a moving image encoding device in accordance with Embodiment 1 of the present invention.
  • the moving image encoding device shown in FIG. 1 includes a block dividing unit 2 for dividing each frame image of an inputted video signal 1 into macroblock images of a plurality of more blocks each having a macroblock size 4 and further dividing each of the macroblock images into a macro/subblock image 5 of one or more subblocks in accordance with an encoding mode 7 to output this macro/subblock image 5 , an intra-prediction unit 8 for, when receiving a macro/subblock image 5 inputted thereto, carrying out an intra-frame prediction on the macro/subblock image 5 by using an image signal stored in a memory 28 for intra prediction to generate a prediction image 11 , a motion-compensated prediction unit 9 for, when receiving a macro/subblock image 5 inputted thereto, carrying out a motion-compensated prediction on the macro/subblock image 5 by using a reference image 15 stored in a
  • An encoding controlling unit 3 outputs pieces of information required for the process carried out by each unit (the macroblock size 4 , encoding modes 7 , an optimum encoding mode 7 a , prediction parameters 10 , optimum prediction parameters 10 a or 18 a , compression parameters 20 , and optimum compression parameters 20 a ).
  • the macroblock size 4 and the encoding mode 7 will be explained.
  • the details of the other pieces of information will be mentioned later.
  • the encoding controlling unit 3 notifies the block dividing unit 2 of the macroblock size 4 of each frame image of the inputted video signal 1 , and also notifies the block dividing unit 2 of all selectable encoding modes 7 in accordance with the picture type for each macro block to be encoded.
  • the encoding controlling unit 3 can select a certain encoding mode from among a set of encoding modes, this set of encoding modes is arbitrarily set up.
  • the encoding controlling unit can select a certain encoding mode from among a set shown in FIG. 2A or 2 B which will be mentioned below.
  • FIG. 2A is a view showing an example of encoding modes for a P (Predictive) picture in each of which predictive encoding in a time direction is carried out.
  • mb_mode 0 to mb_mode 2 show modes (inter) in each of which a macroblock (M ⁇ L pixel block) is encoded by using an inter-frame prediction.
  • mb_mode 0 is the mode in which a motion vector is allocated to the whole of a macroblock
  • mc_mode 1 and mc_mode 2 are the modes in each of which a macroblock is divided into equal parts which are aligned horizontally or vertically, and different motion vectors are allocated to the subblocks into which the macroblock is divided, respectively.
  • mc_mode 3 is the mode in which a macroblock is divided into four parts, and different encoding modes (sub_mb_mode) are allocated to the four subblocks into which the macroblock is divided, respectively.
  • sub_mb_mode 0 to sub_mb_mode 4 are the modes each of which, when mb_mode 3 is selected as the encoding mode of a macroblock, can be allocated to each of the four subblocks (mxl pixel blocks) into which the macroblock is divided.
  • sub_mb_mode 0 is the mode (intra) in which a subblock is encoded by using an intra-frame prediction.
  • the other modes are the modes (inter) which a subblock is encoded by using an inter-frame prediction.
  • sub_mb_mode 1 is the mode in which one motion vector is allocated to the whole of a subblock
  • sub_mc_mode 2 and sub_mc_mode 3 are the modes in each of which a subblock is divided into equal parts which are aligned horizontally or vertically, and different motion vectors are allocated to the subblocks into which the subblock is divided, respectively
  • sub_mb_mode 4 is the mode in which a subblock is divided into four parts, and different motion vectors are allocated to the four subblocks into which the subblock is divided, respectively.
  • FIG. 2B is a view showing another example of encoding modes for a P picture in each of which predictive encoding in a time direction is carried out.
  • mb_mode 0 to 6 are the modes (inter) in each of which a macroblock (M ⁇ L pixel block) is encoded by using an inter frame prediction.
  • mb_mode 0 is the mode in which one motion vector is allocated to the whole of a macroblock
  • mb_mode 1 to mb_mode 6 are the modes in each of which a macroblock is divided into two parts which are aligned horizontally, vertically or diagonally, and different motion vectors are allocated to the two subblocks into which the macroblock is divided, respectively.
  • mb_mode 7 is the mode in which a macroblock is divided into four parts, and different encoding modes (sub_mb_mode) are allocated to the four subblocks into which the macroblock is divided, respectively.
  • sub_mb_mode 0 to sub_mb_mode 8 are the modes each of which, when mb_mode 7 is selected as the encoding mode of a macroblock, can be allocated to each of the four subblocks (mxl pixel blocks) into which the macroblock is divided.
  • sub_mb_mode 0 is the mode (intra) in which a subblock is encoded by using an intra-frame prediction.
  • the other modes are the modes (inter) which a subblock is encoded by using an inter-frame prediction.
  • sub_mb_mode 1 is the mode in which one motion vector is allocated to the whole of a subblock
  • sub_mb_mode 2 to sub_mb_mode 7 are the modes in each of which a subblock is divided into two parts which are aligned horizontally, vertically or diagonally, and different motion vectors are allocated to the two subblocks into which the subblock is divided, respectively.
  • sub_mb_mode 8 is the mode in which a subblock is divided into four parts, and different motion vectors are allocated to the four subblocks into which the subblock is divided, respectively.
  • the block dividing unit 2 divides each frame image of the inputted video signal 1 inputted to the moving image encoding device into macroblock images each having the macroblock size 4 notified thereto by the encoding controlling unit 3 .
  • an encoding mode 7 notified thereto from the encoding controlling unit 3 includes a mode (one of sub_mb_mode 1 to 4 of FIG. 2A or one of sub_mb_mode 1 to sub_mb_mode 8 of FIG. 2B ) in which different encoding modes are respectively allocated to subblocks into which a macroblock is divided
  • the block dividing unit 2 divides each macroblock image into subblock images shown by the encoding mode 7 . Therefore, a block image outputted from the block dividing unit 2 is either one of a macroblock image or a subblock image in accordance with the encoding mode 7 .
  • this block image is referred to as a macro/subblock image 5 .
  • each frame of the inputted video signal 1 has a horizontal or vertical size which is not an integral multiple of the horizontal size or vertical size of the macroblock size 4
  • a frame (extended frame) in which pixels are additionally provided in a horizontal or vertical direction in such a way that each frame of the inputted video signal 1 has a horizontal or vertical size which is an integral multiple of the horizontal size or vertical size of the macroblock size is generated for each frame of the inputted video signal 1 .
  • a generation method of generating pixels in the extended region when pixels are added to widen each frame in, for example, a vertical direction there is a method of filling the extended region by repeatedly copying a line of pixels running on a lower edge of the original frame or by repeatedly generating a line of pixels having a fixed pixel value (gray, black, white, or the like). Also when pixels are added to widen each frame in a horizontal direction, there is a method of filling the extended region by repeatedly copying a line of pixels running on a right edge of the original frame or by repeatedly generating a line of pixels having a fixed pixel value (gray, black, white, or the like).
  • the extended frame which is generated for each frame of the inputted video signal 1 and whose frame size is an integral multiple of the macroblock size, instead of each frame image of the inputted video signal 1 , is inputted to the block dividing unit 2 .
  • the macroblock size 4 and the frame size (horizontal size and vertical size) of each frame of the inputted video signal 1 are outputted to the variable length encoding unit 23 so as to be multiplexed into the bitstream in units of one sequence which consists of one or more frames or in units of one picture.
  • the value of the macroblock size can be alternatively defined by a profile or the like, instead of being multiplexed directly into the bitstream.
  • identification information for identifying the profile on a per-sequence basis is multiplexed into the bitstream.
  • the switching unit 6 is a switch for switching between the input destinations of the macro/subblock image 5 in accordance with the encoding mode 7 .
  • the encoding mode 7 is the mode in which the macro/subblock image is encoded by using an intra-frame prediction (referred to as the intra-frame prediction mode from here on)
  • this switching unit 6 inputs the macro/subblock image 5 to the intra-prediction unit 8
  • the encoding mode 7 is the mode in which the macro/subblock image is encoded by using an inter-frame prediction (referred to as the inter-frame prediction mode from here on)
  • the switching unit inputs the macro/subblock image 5 to the motion-compensated prediction unit 9 .
  • the intra-prediction unit 8 carries out an intra-frame prediction on the macro/subblock image 5 inputted thereto in units of one macro block to be encoded having a size specified by the macroblock size 4 or in units of one subblock specified by the encoding mode 7 .
  • the intra-prediction unit 8 generates a prediction image 11 by using the image signal in the frame stored in the memory 28 for intra prediction for each of all intra prediction modes included in the prediction parameters 10 notified thereto from the encoding controlling unit 3 .
  • the encoding controlling unit 3 specifies an intra prediction mode as a prediction parameter 10 corresponding to the encoding mode 7 .
  • this intra prediction mode for example, there can be a mode in which the macroblock or subblock is divided into blocks of 4 ⁇ 4 pixels, and a prediction image is generated by using pixels in the vicinity of a unit block of the image signal stored in the memory 28 for intra prediction, a mode in which the macroblock or subblock is divided into blocks of 8 ⁇ 8 pixels, and a prediction image is generated by using pixels in the vicinity of a unit block of the image signal stored in the memory 28 for intra prediction, a mode in which the macroblock or subblock is divided into blocks of 16 ⁇ 16 pixels, and a prediction image is generated by using pixels in the vicinity of a unit block of the image signal stored in the memory 28 for intra prediction, and a mode in which a prediction image is generated from an image of a reduced inside of the macroblock or subblock.
  • the motion-compensated prediction unit 9 specifies a reference image 15 which is used for the generation of a prediction image from the data about one or more frames of reference images stored in the motion-compensated prediction frame memory 14 , and carries out a motion-compensated prediction by using this reference image 15 and the macro/subblock image 5 in accordance with the encoding mode 7 notified thereto from the encoding controlling unit 3 to generate prediction parameters 18 and a prediction image 17 .
  • the motion-compensated prediction unit 9 determines motion vectors and the identification number (reference image index) or the like of the reference image indicated by each of the motion vectors as the prediction parameters 18 corresponding to the encoding mode 7 .
  • the details of a generation method of generating prediction parameters 18 will be mentioned later.
  • the subtraction unit 12 subtracts either one of the prediction image 11 and the prediction image 17 from the macro/subblock image 5 to acquire a prediction difference signal 13 .
  • the prediction difference signal 13 is generated for each of all the prediction images 11 which the intra-prediction unit 8 generates in accordance with all the intra prediction modes specified by the prediction parameters 10 .
  • the prediction difference signal 13 which is generated in accordance with each of all the intra prediction modes specified by the prediction parameters 10 is evaluated by the encoding controlling unit 3 , and optimum prediction parameters 10 a including an optimum intra prediction mode are determined.
  • the encoding controlling unit uses, for example, a method of calculating an encoding cost J 2 , which will be mentioned below, by using the compressed data 21 generated by transforming and quantizing the prediction difference signal 13 .
  • the encoding controlling unit selects the intra prediction mode which minimizes the encoding cost J 2 .
  • the encoding controlling unit 3 evaluates the prediction difference signal 13 which is generated for each of all the modes included in the encoding modes 7 by either the intra-prediction unit 8 or the motion-compensated prediction unit 9 , and determines an optimum encoding mode 7 a which provides an optimum degree of encoding efficiency from among the encoding modes 7 on the basis of the result of the evaluation.
  • the encoding controlling unit 3 further determines optimum prediction parameters 10 a or 18 a and optimum compression parameters 20 a corresponding to the optimum encoding mode 7 a from the prediction parameters 10 or 18 and the compression parameters 20 .
  • a procedure of determining the optimum prediction parameters and a procedure of determining the optimum compression parameters will be mentioned later.
  • the intra prediction mode is included in the prediction parameters 10 and in the optimum prediction parameters 10 a .
  • motion vectors, the identification number (reference image index) of the reference image indicated by each of the motion vectors, etc. are included in the prediction parameters 18 and in the optimum prediction parameters 18 a .
  • a transformation block size, a quantization step size, etc. are included in the compression parameters 20 and in the optimum compression parameters 20 a.
  • the encoding controlling unit 3 outputs the optimum encoding mode 7 a , the optimum prediction parameters 10 a or 18 a , and the optimum compression parameters 20 a for the macro block or subblock to be encoded to the variable length encoding unit 23 .
  • the encoding controlling unit 3 also outputs the optimum compression parameters 20 a of the compression parameters 20 to the transformation/quantization unit 19 and to the inverse quantization/inverse transformation unit 22 .
  • the transformation/quantization unit 19 selects the prediction difference signal 13 (referred to as the optimum prediction differential signal 13 a from here on) which corresponds to the prediction image 11 or 17 generated on the basis of the optimum encoding mode 7 a and the optimum prediction parameters 10 a or 18 a which the encoding controlling unit 3 has determined from among the plurality of prediction difference signals 13 which are respectively generated for all the modes included in the encoding modes 7 , carries out a transforming process, such as a DCT, on this optimum prediction differential signal 13 a on the basis of the transformation block size in the optimum compression parameters 20 a determined by the encoding controlling unit 3 to calculate transform coefficients and also quantizes these transform coefficients on the basis of the quantization step size in the optimum compression parameters 20 a notified thereto from the encoding controlling unit 3 , and then outputs the compressed data 21 which are the transform coefficients quantized thereby to the inverse quantization/inverse transformation unit 22 and to the variable length encoding unit 23 .
  • a transforming process such
  • the inverse quantization/inverse transformation unit 22 inverse-quantizes the compressed data 21 inputted thereto from the transformation/quantization unit 19 by using the optimum compression parameters 20 a and then carries out an inverse transformation process, such as an inverse DCT, to generate a local decoded prediction difference signal 24 of the prediction difference signal 13 a , and outputs this local decoded prediction difference signal 24 to the adding unit 25 .
  • an inverse transformation process such as an inverse DCT
  • the adding unit 25 adds the local decoded prediction difference signal 24 and the prediction image 11 or 17 to generate a local decoded image signal 26 , and outputs this local decoded image signal 26 to the loop filter unit 27 while storing the local decoded image signal in the memory 28 for intra prediction.
  • This local decoded image signal 26 serves as an image signal for intra-frame prediction.
  • the loop filter unit 27 carries out a predetermined filtering process on the local decoded image signal 26 inputted thereto from the adding unit 25 , and stores the local decoded image 29 on which the loop filter unit has carried out the filtering process in the motion-compensated prediction frame memory 14 .
  • This local decoded image 29 serves as a reference image 15 for motion-compensated prediction.
  • the filtering process by the loop filter unit 27 can be carried out in units of one macro block of the local decoded image signal 26 inputted to the loop filter unit, or can be carried out on one screenful of macro blocks after the local decoded image signal 26 corresponding to the one screenful of macro blocks are inputted to the loop filter unit.
  • the variable length encoding unit 23 entropy-encodes the compressed data 21 outputted thereto from the transformation/quantization unit 19 , the optimum encoding mode 7 a outputted thereto from the encoding controlling unit 3 , the optimum prediction parameters 10 a or 18 a , and the optimum compression parameters 20 a to generate a bitstream 30 showing the results of those encodings.
  • the optimum prediction parameters 10 a or 18 a and the optimum compression parameters 20 a are encoded in units of one element in accordance with the encoding mode indicated by the optimum encoding mode 7 a.
  • the motion-compensated prediction unit 9 and the transformation/quantization unit 19 operate in cooperation with the encoding controlling unit 3 to determine the encoding mode, the prediction parameters, and the compression parameters which provide an optimum degree of encoding efficiency (i.e. the optimum encoding mode 7 a , the optimum prediction parameters 10 a or 18 a , and the optimum compression parameters 20 a ).
  • the determining procedure which is carried out by the encoding controlling unit 3 , for determining the encoding mode which provides an optimum degree of encoding efficiency, the prediction parameters, and the compression parameters will be explained in the order of 1. the prediction parameters, 2. the compression parameters, and 3. the encoding mode.
  • the motion-compensated prediction unit 9 determines the prediction parameters 18 for each of all the encoding modes 7 (e.g. the set of encoding modes shown in FIG. 2A or 2 B) which are notified from the encoding controlling unit 3 to the motion compensation predicting unit 9 , in cooperation with the encoding controlling unit 3 .
  • all the encoding modes 7 e.g. the set of encoding modes shown in FIG. 2A or 2 B
  • the motion compensation predicting unit 9 in cooperation with the encoding controlling unit 3 .
  • FIG. 3 is a block diagram showing the internal structure of the motion-compensated prediction unit 9 .
  • the motion-compensated prediction unit 9 shown in FIG. 3 includes a motion compensation region dividing unit 40 , a motion detecting unit 42 , and an interpolated image generating unit 43 .
  • input data inputted to the motion-compensated prediction unit include the encoding mode 7 inputted thereto from the encoding controlling unit 3 , the macro/subblock image 5 inputted thereto from the switching unit 6 , and the reference image 15 inputted thereto from the motion-compensated prediction frame memory 14 .
  • the motion compensation region dividing unit 40 divides the macro/subblock image 5 inputted from the switching unit 6 into images of blocks each of which is a unit for motion compensation in accordance with the encoding mode 7 notified thereto from the encoding controlling unit 3 , and outputs this motion compensation region block image 41 to the motion detecting unit 42 .
  • the interpolated image generating unit 43 specifies the reference image 15 which is used for the generation of a prediction image from the data about the one or more frames of reference images stored in the motion-compensated prediction frame memory 14 , and the motion detecting unit 42 detects a motion vector 44 in a predetermined motion search range on the reference image 15 specified by the interpolated image generating unit.
  • the motion detecting unit carries out the detection of the motion vector by using a motion vector having virtual sample accuracy, like in the case of the MPEG-4 AVC standards or the like.
  • This detecting method includes the steps of, for pixel information (referred to as integer pixels) which the reference image has, generating virtual samples (pixels) between integer pixels by implementing an interpolation arithmetic operation on the integer pixels, and using the virtual samples as a prediction image.
  • virtual samples having 1 ⁇ 8-pixel accuracy can be generated and used.
  • virtual samples having 1 ⁇ 2 pixel accuracy are generated by implementing an interpolation arithmetic operation with a 6-tap filter using six integer pixels running in a vertical or horizontal direction.
  • Virtual samples having 1 ⁇ 4 pixel accuracy are generated by implementing an interpolation arithmetic operation using a filter for acquiring a mean value of adjacent 1 ⁇ 2 pixels or integer pixels.
  • the interpolated image generating unit 43 generates a prediction image 45 of virtual pixels in accordance with the accuracy of the motion vector 44 notified thereto from the motion detecting unit 42 .
  • a detection procedure for detecting a motion vector having virtual pixel accuracy will be shown.
  • the interpolated image generating unit 43 generates a prediction image 45 for the motion vector 44 having integer pixel accuracy in the predetermined motion search range of the motion compensation region block image 41 .
  • the prediction image 45 (prediction image 17 ) generated at integer pixel accuracy is outputted to the subtraction unit 12 and is subtracted from the motion compensation region block image 41 (macro/subblock image 5 ) by the subtraction unit 12 , so that the result of the subtraction is defined as a prediction difference signal 13 .
  • the encoding controlling unit 3 evaluates a degree of prediction efficiency for the prediction difference signal 13 and for the motion vector 44 (prediction parameter 18 ) having integer pixel accuracy. In the evaluation of the degree of prediction efficiency, a prediction cost J 1 is calculated in accordance with, for example, the following equation (1), and the motion vector 44 having integer pixel accuracy which minimizes the prediction cost J 1 in the predetermined motion search range is determined.
  • D 1 and R 1 are used as evaluated values.
  • D 1 is the sum of absolute values (SAD) in the macroblock or subblock of the prediction difference signal
  • R 1 is an estimated code amount of the motion vector and the identification number of the reference image indicated by this motion vector
  • A is a positive number.
  • the code amount of the motion vector is predicted by using the value of an adjacent motion vector as the value of the motion vector in each mode shown in FIG. 2A or 2 B, and the prediction difference value is entropy-encoded on the basis of a probability distribution.
  • the evaluated value is determined by carrying out an estimation of a code amount corresponding to the evaluated value.
  • FIG. 4 is a view explaining a determining method of determining a predicted value of the motion vector (referred to as a predicted vector from here on) in each encoding mode 7 shown in FIG. 2B .
  • a predicted vector PMV of this rectangular block is calculated in accordance with the following equation (2) by using already-encoded motion vectors MVa, MVb, and MVc of blocks located on a left side (position A), an upper side (position B), and an upper right side (position C) of the rectangular block.
  • median ( ) corresponds to a median filter process and is a function of outputting the median of the motion vectors MVa, MVb, and MVc.
  • the interpolated image generating unit 43 generates a prediction image 45 for one or more motion vectors 44 having 1 ⁇ 2 pixel accuracy located in the vicinity of the motion vector having integer pixel accuracy which is determined in accordance with the above-mentioned “motion vector detection procedure I”. After that, in the same way that the above-mentioned “motion vector detection procedure I” is carried out, the prediction image 45 (prediction image 17 ) generated at 1 ⁇ 2 pixel accuracy is subtracted from the motion compensation region block image 41 (macro/subblock image 5 ) by the subtraction unit 12 , so that a prediction difference signal 13 is acquired.
  • the encoding controlling unit 3 evaluates a degree of prediction efficiency for this prediction difference signal 13 and for the motion vector 44 (prediction parameter 18 ) having 1 ⁇ 2 pixel accuracy, and determines a motion vector 44 having 1 ⁇ 2 pixel accuracy which minimizes the prediction cost J 1 from the one or more motion vectors having 1 ⁇ 2 pixel accuracy located in the vicinity of the motion vector having integer pixel accuracy.
  • the encoding controlling unit 3 and the motion-compensated prediction unit 9 determine a motion vector 44 having 1 ⁇ 4 pixel accuracy which minimizes the prediction cost J 1 from one or more motion vectors having 1 ⁇ 4 pixel accuracy located in the vicinity of the motion vector having 1 ⁇ 2 pixel accuracy which is determined in accordance with the above-mentioned “motion vector detection procedure II”.
  • the encoding controlling unit 3 and the motion-compensated prediction unit 9 similarly detect a motion vector having virtual pixel accuracy until the motion vector detected thereby has a predetermined degree of accuracy.
  • the detection of a motion vector having virtual pixel accuracy can be aborted when, for example, a threshold for the prediction cost is predetermined and the prediction cost J 1 becomes smaller than the predetermined threshold before the motion vector detected has a predetermined degree of accuracy.
  • the motion vector can be made to refer to a pixel located outside the frame defined by the reference frame size. In this case, it is necessary to generate pixels located outside the frame.
  • a method of generating pixels located outside the frame there is a method of filling an outside region with pixels running on a screen edge of the frame.
  • the frame size of each frame of the inputted video signal 1 is not an integral multiple of macroblock size and an extended frame is inputted instead of each frame of the inputted video signal 1
  • the size which is extended to an integral multiple of the macroblock size is defined as the frame size of the reference frame.
  • the frame size of the original inputted video signal is defined as the frame size of the reference frame.
  • the motion-compensated prediction unit 9 For the motion compensation region block image 41 of each of a plurality of bocks into which the macro/subblock image 5 is divided and which is a unit for the motion compensation indicated by the encoding mode 7 , the motion-compensated prediction unit 9 outputs both the virtual pixel accurate motion vector having a predetermined degree of accuracy which is determined for the motion compensation region block image, and the identification number of the reference image indicated by the motion vector as the prediction parameters 18 .
  • the motion-compensated prediction unit 9 also outputs the prediction image 45 (prediction image 17 ) generated using the prediction parameters 18 to the subtraction unit 12 , and the prediction image is subtracted from the macro/subblock image by the subtraction unit 12 , so that a prediction difference signal 13 is acquired.
  • the prediction difference signal 13 outputted from the subtraction unit 12 is outputted to the transformation/quantization unit 19 .
  • FIG. 5 is a view showing an example of adaptation of the transformation block size in accordance with an encoding mode 7 shown in FIG. 2B .
  • a block of 32 ⁇ 32 pixels is used as an example of a block of M ⁇ L pixels.
  • the mode indicated by the encoding mode 7 is one of mb_mode 0 to mb_mode 6
  • either the size of 16 ⁇ 16 pixels or the size of 8 ⁇ 8 pixels is adaptively selectable as the transformation block size.
  • the encoding mode 7 indicates mb_mode 7
  • either the size of 8 ⁇ 8 pixels or the size of 4 ⁇ 4 pixels is adaptively selectable as the transformation block size for each of 16 ⁇ 16 pixel subblocks into which each macroblock is divided.
  • the set of selectable transformation block sizes for each encoding mode can be defined from among arbitrary rectangular block sizes each of which is equal to or smaller than the size of equal subblocks into which a macroblock is divided in accordance with the encoding mode.
  • FIG. 6 is a view showing another example of the adaptation of the transformation block size in accordance with an encoding mode 7 shown in FIG. 2B .
  • the mode indicated by the encoding mode 7 is the above-mentioned mb_mode 0 , mb_mode 5 , or mb_mode 6
  • the transformation block size in accordance with the shape of each subblock which is a unit for the motion compensation is selectable as a selectable transformation block size.
  • the transformation block size is adaptively selectable from among the size of 16 ⁇ 16 pixels, the size of 8 ⁇ 8 pixels, and the size of 32 ⁇ 32 pixels.
  • the transformation block size is adaptively selectable from among the size of 16 ⁇ 16 pixels, the size of 8 ⁇ 8 pixels, and the size of 16 ⁇ 32 pixels.
  • the transformation block size is adaptively selectable from among the size of 16 ⁇ 16 pixels, the size of 8 ⁇ 8 pixels, and the size of 32 ⁇ 16 pixels.
  • the transformation block size is adaptively selectable from among the size of 16 ⁇ 16 pixels, the size of 8 ⁇ 8 pixels, and the size of 16 ⁇ 32 pixels.
  • the adaptation can be carried out in such a way that the transformation block size is selected from the size of 16 ⁇ 16 pixels and the size of 8 ⁇ 8 pixels for a region which is not a rectangle, while the transformation block size is selected from the size of 8 ⁇ 8 pixels and the size of 4 ⁇ 4 pixels for a region which is a rectangle.
  • the encoding controlling unit 3 defines the set of transformation block sizes in accordance with the encoding mode 7 illustrated in FIGS. 5 and 6 as a compression parameter 20 .
  • the set of selectable transformation block sizes is determined in advance in accordance with the encoding mode 7 of each macroblock, and a transformation block size can be selected adaptively for each macroblock or subblock, the set of selectable transformation block sizes can be alternatively determined in advance in accordance with the encoding mode 7 (one of sub_mb_mode 1 to sub_mb_mode 8 shown in FIG.
  • the encoding controlling unit 3 can determine the set of transformation block sizes in accordance with the encoding mode 7 in advance, and can adaptively select a transformation block size from the set.
  • the transformation/quantization unit 19 determines an optimum transformation block size from the transformation block sizes in units of one macroblock having a size specified by the macroblock size 4 or in units of one of subblocks into which each macroblock is further divided in accordance with the encoding mode 7 , in cooperation with the encoding controlling unit 3 .
  • the details of a procedure for determining an optimum transformation block size will be explained.
  • FIG. 7 is a block diagram showing the internal structure of the transformation/quantization unit 19 .
  • the transformation/quantization unit 19 shown in FIG. 7 includes a transformation block size dividing unit 50 , a transforming unit 52 , and a quantizing unit 54 .
  • input data inputted to the transformation/quantization unit include the compression parameters 20 (the transformation block size, the quantization step size, etc.) inputted thereto from the encoding controlling unit 3 and the prediction difference signal 13 inputted thereto from the encoding controlling unit 3 .
  • the transformation block size dividing unit 50 converts the prediction difference signal 13 of each macroblock or subblock which is the target for determination of the transformation block size into blocks in accordance with the transformation block size in the compression parameters 20 , and outputs each of the blocks to the transforming unit 52 as a transformation target block 51 .
  • the plurality of transformation block sizes of transformation target blocks 51 are sequentially outputted to the transforming unit 52 .
  • the transforming unit 52 carries out a DCT, an integer transformation in which the transform coefficients of a DCT are approximated by integers, and a transforming process in accordance with a transforming method, such as Hadamard transform, on the transformation object block 51 inputted thereto to generate transform coefficients 53 , and outputs the transform coefficients 53 generated thereby to the quantizing unit 54 .
  • a transforming method such as Hadamard transform
  • the quantizing unit 54 quantizes the transform coefficients 53 inputted thereto in accordance with the quantization step size in the compression parameters 20 notified thereto from the encoding controlling unit 3 , and outputs compressed data 21 which are the transform coefficients quantized to the inverse quantization/inverse transformation unit 22 and to the encoding controlling unit 3 .
  • the transforming unit 52 and the quantizing unit 54 carry out the above-mentioned transformation and quantization process on all the transformation block sizes of transformation target blocks, and outputs the compressed data 21 associated with each of all the transformation block sizes.
  • the compressed data 21 outputted from the quantizing unit 54 are inputted to the encoding controlling unit 3 , and are used for the evaluation of a degree of encoding efficiency for the transformation block size in the compression parameters 20 .
  • the encoding controlling unit 3 uses the compressed data 21 acquired for each of all the selectable transformation block sizes in each encoding mode included in the encoding modes 7 to calculate an encoding cost J 2 in accordance with, for example, the following equation (3), and to select the transformation block size which minimizes the encoding cost J 2 from among the selectable transformation block sizes.
  • D 2 and R 2 are used as evaluated values.
  • D 2 the distortion sum of squared differences or the like between the local decoded image signal 26 , which is acquired by inputting the compressed data 21 acquired for the transformation block size to the inverse quantization/inverse transformation unit 22 , and adding the prediction image 17 to a local decoded prediction difference signal 24 which is acquired by carrying out an inverse transformation and inverse quantization process on the compressed data 21 , and the macro/subblock image 5 can be used.
  • R 2 the code amount (or estimated code amount) acquired by actually encoding the compressed data 21 acquired for the transformation block size, and the encoding mode 7 and the prediction parameters 10 or 18 associated with the compressed data 21 by means of the variable length encoding unit 23 is used.
  • the encoding controlling unit 3 selects the transformation block size corresponding to the determined optimum encoding mode 7 a and includes the transformation block size in the optimum compression parameters 20 a , and then outputs the optimum compression parameters to the variable length encoding unit 23 . After entropy-encoding these optimum compression parameters 20 a , the variable length encoding unit 23 multiplexes the optimum compression parameters entropy-encoded thereby into the bitstream 30 .
  • the transformation block size is selected from among the set of transformation block sizes (illustrated in FIGS. 5 and 6 ) which are defined in advance in accordance with the optimum encoding mode 7 a of the macroblock or subblock, what is necessary is just to assign identification information, such as an ID, to each transformation block size included in each set of transformation block sizes, entropy-encode the identification information as information about the transformation block size, and multiplex the identification information into the bitstream 30 .
  • the identification information of each set of transformation block sizes is set up in advance in the decoding device.
  • the decoding device can determine the transformation block size automatically from the set of transformation block sizes when only one transformation block size is included in the set of transformation block sizes, the encoding device does not have to multiplex the identification information of the transformation block size into the bitstream 30 .
  • the encoding controlling unit 3 uses the compressed data 21 which are acquired by further transforming and quantizing the prediction difference signal 13 which is acquired by using each of the encoding modes 7 , and the prediction parameters 10 or 18 and the compression parameters 20 in that encoding mode to determine the encoding mode 7 which reduces the encoding cost J 2 to a minimum in accordance with the above-mentioned equation (3), and selects the encoding mode 7 as the optimum encoding mode 7 a of the macroblock currently being processed.
  • the encoding controlling unit can determine the optimum encoding mode 7 a from among all the encoding modes including a skip mode as a mode of the macroblock or subblock in addition to the encoding modes shown in FIGS. 2A 2 B.
  • the skip mode is the mode in which a prediction image on which motion compensation is carried out by using the motion vector of an adjacent macroblock or subblock is defined as the local decoded image signal in the encoding device. Because it is not necessary to calculate the prediction parameters other than the encoding modes, and the compression parameters to multiplex them into the bitstream, the inputted image can be encoded while the code amount is suppressed.
  • the decoding device outputs the prediction image on which motion compensation is carried out by using the motion vector of an adjacent macroblock or subblock in accordance with the same procedure as that carried out by the encoding device as the decoded image signal.
  • each frame of the inputted video signal 1 is not an integral multiple of the macroblock size and an extended frame is inputted instead of each frame of the inputted video signal 1 , a control operation of selecting only the skip mode for a macroblock or subblock including an extended region can be carried out, and an encoding mode can be determined in such a way that the code amount spent on the extended region can be suppressed.
  • the encoding controlling unit 3 outputs the optimum encoding mode 7 a providing the optimum degree of encoding efficiency which is determined in accordance with the above-mentioned “1. Determining procedure for determining the prediction parameters”, “2. Determining procedure for determining the compression parameters”, and “3. Determining procedure for determining the encoding mode” to the variable length encoding unit 23 , while selecting the prediction parameters 10 or 18 corresponding to the optimum encoding mode 7 a as the optimum prediction parameters 10 a or 18 a and similarly selecting the compression parameters 20 corresponding to the optimum encoding mode 7 a as the optimum compression parameters 20 a , and then outputting these optimum prediction and compression parameters to the variable length encoding unit 23 .
  • variable length encoding unit 23 entropy-encodes the optimum encoding mode 7 a , the optimum prediction parameters 10 a or 18 a , and the optimum compression parameters 20 a , and then multiplexes them into the bitstream 30 .
  • the optimum prediction differential signal 13 a acquired from the prediction image 11 or 17 based on the optimum encoding mode 7 a , the optimum prediction parameters 10 a or 18 a , and the optimum compression parameter 20 a which are determined as above is transformed and quantized into compressed data 21 by the transformation/quantization unit 19 , as mentioned above, and these compressed data 21 are entropy-encoded by the variable length encoding unit 23 and are multiplexed into the bitstream 30 . Further, these compressed data 21 are made to pass via the inverse quantization/inverse transformation unit 22 and the adding unit 25 , and then become a local decoded image signal 26 and are inputted to the loop filter unit 27 .
  • FIG. 8 is a block diagram showing the structure of the moving image decoding device in accordance with Embodiment 1 of the present invention.
  • the moving image decoding device shown in FIG. 8 is a block diagram showing the structure of the moving image decoding device in accordance with Embodiment 1 of the present invention.
  • variable length decoding unit 61 for entropy-decoding the optimum encoding mode 62 multiplexed into the bitstream 60 in units of one macroblock to while entropy-decoding the optimum prediction parameters 63 , the compressed data 64 , and the optimum compression parameters 65 from the bitstream 60 in units of one macroblock or subblock divided in accordance with the decoded optimum encoding mode 62 , an intra-prediction unit 69 for, when the optimum prediction parameters 63 are inputted, generating a prediction image 71 by using the intra prediction mode included in the optimum prediction parameters 63 , and a decoded image 74 a stored in a memory 77 for intra prediction, a motion-compensated prediction unit 70 for, when the optimum prediction parameters 63 are inputted, carrying out a motion-compensated prediction by using the motion vector included in the optimum prediction parameters 63 , and the reference image 76 in a motion-compensated prediction frame memory 75 which is specified by the reference image index included in the
  • the variable length decoding unit 61 carries out an entropy decoding process on the bitstream 60 to acquire the macroblock size and the frame size in units of one sequence which consists of one or more frames of pictures or in units of one picture.
  • the macroblock size is determined on the basis of the identification information of the profile which is decoded from the bitstream in units of one sequence.
  • the number of macroblocks included in each frame is determined on the basis of the decoded macroblock size of each frame and the decoded frame size, and the optimum encoding mode 62 , the optimum prediction parameters 63 , the compressed data 64 (i.e. quantized and transformed coefficient data), the optimum compression parameters 65 (the transformation block size information and the quantization step size), etc. of each macroblock included in the frame are decoded.
  • the optimum encoding mode 62 , the optimum prediction parameters 63 , the compressed data 64 , and the optimum compression parameters 65 which are decoded by the decoding device correspond to the optimum encoding mode 7 a , the optimum prediction parameters 10 a or 18 a , the compressed data 21 , and the optimum compression parameters 20 a which are encoded by the encoding device, respectively.
  • the decoding device specifies the transformation block size of the macroblock or subblock from the optimum encoding mode 62 and the transformation block size information in the optimum compression parameters 65 .
  • the inverse quantization/inverse transformation unit 66 carries out an inverse quantization and inverse transformation process by using the compressed data 64 and the optimum compression parameters 65 which are inputted from the variable length decoding unit 61 in units of one block whose size is specified by the transformation block size information to calculate prediction difference signal decoded values 67 .
  • variable length decoding unit 61 refers to the motion vectors of already-decoded peripheral blocks, and determines a predicted vector by carrying out a process shown in FIG. 4 to acquire a decoded value of the motion vector by adding the prediction difference values decoded from the bitstream 60 to the predicted vector.
  • the variable length decoding unit 61 includes the decoded value of this motion vector in the optimum prediction parameters 63 , and outputs these optimum prediction parameters to the switching unit 68 .
  • the switching unit 68 is a switch for switching between the input destinations of the optimum prediction parameters 63 in accordance with the optimum encoding mode 62 .
  • this switching unit 68 outputs the optimum prediction parameters 63 (intra prediction mode) similarly inputted from the variable length decoding unit 61 to the intra-prediction unit 69 , whereas when the optimum encoding mode 62 shows the inter frame prediction mode, the switching unit outputs the optimum prediction parameters 63 (the motion vectors, the identification number (reference image index) of the reference image indicated by each of the motion vectors, etc.) to the motion-compensated prediction unit 70 .
  • the intra-prediction unit 69 refers to the decoded image 74 a in the frame stored in the memory 77 for intra prediction (decoded image signal in the frame), and generates and outputs a prediction image 71 corresponding to the intra prediction mode indicated by the optimum prediction parameters 63 .
  • the intra-prediction unit 8 generates a prediction image 11 corresponding to each of all the intra prediction modes indicated by the encoding modes 7 , while the intra-prediction unit 69 differs from the intra-prediction unit 8 in that the intra-prediction unit 69 generates only a prediction image 71 corresponding to the intra prediction mode indicated by the optimum encoding mode 62 .
  • the motion-compensated prediction unit 70 generates a prediction image 72 from the one or more frames of reference images 76 stored in the motion-compensated prediction frame memory 75 on the basis of the motion vector, the reference image index, and so on which are indicated by the inputted optimum prediction parameters 63 , and outputs the prediction image 72 .
  • a generation method of generating a prediction image 72 which is implemented by the motion-compensated prediction unit 70 corresponds to the operation of the motion-compensated prediction unit 9 in the encoding device from which the process of searching through a plurality of reference images for motion vectors (corresponding to the operations of the motion detecting unit 42 and the interpolated image generating unit 43 shown in FIG. 3 ) is excluded.
  • the motion-compensated prediction unit carries out only the process of generating a prediction image 72 in accordance with the optimum prediction parameters 63 provided thereto from the variable length decoding unit 61 .
  • the motion-compensated prediction unit 70 When the motion vector is made to refer to a pixel located outside the frame which is defined by the reference frame size, the motion-compensated prediction unit 70 generates a prediction image 72 by using, for example, a method of filling a pixel region located outside the frame with pixels running on a screen edge of the frame, like that of the encoding device.
  • the reference frame size can be defined by the decoded frame size which is extended in such a way as to be an integral multiple of the decoded macroblock size, or can be defined by the decoded frame size, and the decoding device determines the reference frame size in accordance with the same procedure as that carried out by the encoding device.
  • the adding unit 73 adds either one of the prediction image 71 and the prediction image 72 and the prediction difference signal decoded values 67 outputted thereto from the inverse quantization/inverse transformation unit 66 to generate a decoded image 74 .
  • decoded image 74 is stored in the memory 77 for intra prediction in order to use the decoded image as a reference image (decoded image 74 a ) for generation of an intra prediction image for a subsequent macroblock, the decoded image 74 is inputted to the loop filter unit 78 .
  • the loop filter unit 78 carries out the same operation as that of the loop filter unit 27 in the encoding device to generate a reproduced image 79 , and outputs this reproduced image to outside the moving image decoding device. Further, this reproduced image 79 is stored in the motion-compensated prediction frame memory 75 in order to use the reproduced image as a reference image 76 for subsequent generation of a prediction image.
  • the size of the reproduced image acquired after decoding all the macroblocks in the frame is an integral multiple of the macroblock size.
  • the decoded image of the extended region of the reproduced image stored in the motion-compensated prediction frame memory 75 is not referred to for subsequent generation of a prediction image. Therefore, the decoded image in which the decoded image of the extended region is removed from the reproduced image can be stored in the motion-compensated prediction frame memory 75 .
  • the moving image encoding device in accordance with Embodiment 1 is constructed in such a way that for each of macro/subblock images 5 into which an inputted moving image is divided in accordance with the encoding mode 7 of each macroblock, the moving image encoding device predetermines a set of transformation blocks including a plurality of transformation block sizes in accordance with the size of a macroblock or subblock, the encoding controlling unit 3 selects one transformation block size which provides an optimum degree of encoding efficiency from the set of transformation block sizes and includes the transformation block size selected thereby in optimum compression parameters 20 a , and then notifies these optimum compression parameters to the transformation/quantization unit 19 , and the transformation/quantization unit 19 divides an optimum prediction differential signal 13 a into blocks each having the transformation block size included in the optimum compression parameters 20 a , and carries out a transformation and quantization process on each of the blocks to generate compressed data 21 , the moving image encoding device can improve the quality of the encoded video with a similar code
  • variable length encoding unit 23 is constructed in such a way as to multiplex the transformation block size which is adaptively selected in accordance with the encoding mode 7 from the set of transformation block sizes into the bitstream 30
  • the moving image decoding device in accordance with Embodiment 1 is constructed in such a way that the variable length decoding unit 61 decodes the optimum compression parameters 65 from the bitstream 60 in units of one macroblock or subblock (or on a per-macroblock or per-subblock basis), and the inverse quantization/inverse transformation unit 66 determines a transformation block size on the basis of the transformation block size information included in these optimum compression parameters 65 and carries out an inverse transformation and inverse quantization process on the compressed data 64 in units of one block having the transformation block size.
  • the moving image decoding device can select the transformation block size which has been used by the encoding device from the set of transformation block sizes which is defined in the same way that the set of transformation block sizes is defined by the moving image encoding device to decode the compressed data, the moving image decoding device can correctly decode the bitstream encoded by the moving image encoding device in accordance with Embodiment 1.
  • FIG. 9 is a block diagram showing the internal structure of the variable length encoding unit 23 of the moving image encoding device in accordance with Embodiment 2 of the present invention.
  • the same components as those shown in FIG. 1 or like components are designated by the same reference numerals as those shown in the figure, and the explanation of the components will be omitted hereafter.
  • Embodiment 2 because the structure of the moving image encoding device in accordance with this Embodiment 2 is the same as that in accordance with above-mentioned Embodiment 1, and the operation of each component except the variable length encoding unit 23 is the same as that in accordance with above-mentioned Embodiment 1, an explanation will be made by using FIGS. 1 to 8 . Further, for the sake of simplicity, although it is assumed hereafter that the moving image encoding device in accordance with this Embodiment 2 has a structure and a processing method based on the use of the set of encoding modes shown in FIG. 2A , it is needless to say that Embodiment 2 can also be applied to a structure and a processing method based on the use of the set of encoding modes shown in FIG. 2B .
  • the variable length encoding unit 23 shown in FIG. 9 includes a binarization table memory 105 for storing a binarization table indicating a correspondence between index values which a multi valued signal showing an encoding mode 7 (or an optimum prediction parameter 10 a or 18 a , or an optimum compression parameter 20 a ) can have, and binary signals, a binarizing unit 92 for using this binarization table to convert an optimum encoding mode 7 a (or an optimum prediction parameter 10 a or 18 a , or an optimum compression parameter 20 a ) shown by the multi valued signal which is selected by an encoding controlling unit 3 into a binary signal 103 , an arithmetic encoding processing operation unit 104 for referring to context identification information 102 which is generated by a context generating unit 99 , a context information memory 96 , a probability table memory 97 , and a state transition table memory 98 to carry out arithmetic encoding on the binary signal 103 into which the optimum en
  • variable length encoding procedure carried out by the variable length encoding unit 23 will be explained by taking the optimum encoding mode 7 a of a macroblock outputted from the encoding controlling unit 3 as an example of a parameter to be entropy-encoded.
  • An optimum prediction parameter 10 a or 18 a , or an optimum compression parameter 20 a which is similarly a parameter to be encoded, can be variable-length-encoded in accordance with the same procedure as that in accordance with which the variable length encoding unit encodes the optimum encoding mode 7 a , the explanation of the procedure carried out on an optimum prediction parameter or an optimum compression parameter will be omitted hereafter.
  • the encoding controlling unit 3 in accordance with this Embodiment 2 outputs a context information initialization flag 91 , a type indicating signal 100 , peripheral block information 101 , and a binarization table update flag 113 .
  • the details of each of the pieces of information will be mentioned later.
  • An initializing unit 90 initializes context information 106 stored in the context information memory 96 in accordance with the context information initialization flag 91 notified thereto from the encoding controlling unit 3 to place the context information 106 in its initial state. The details of the initialization process carried out by the initializing unit 90 will be mentioned later.
  • the binarizing unit 92 refers to the binarization table stored in the binarization table memory 105 to convert the index value of the multi valued signal showing the type of the optimum encoding mode 7 a inputted thereto from the encoding controlling unit 3 into a binary signal 103 , and outputs this binary signal to the arithmetic encoding processing operation unit 104 .
  • FIG. 10 is a view showing an example of the binarization table held by the binarization table memory 105 .
  • an “encoding mode” column shown in FIG. 10 there are five types of encoding modes 7 including a skip mode (mb_skip: a mode in which the decoding device uses a prediction image, on which the motion compensation has been carried out by using the motion vectors of adjacent macroblocks by the encoding device, for a decoded image) in addition to the encoding modes (mb_mode 0 to mb_mode 3 ) shown in FIG. 2A .
  • An “index” value corresponding to each of the encoding modes is stored in the binarization table.
  • each of these encoding modes is binarized into a binary number having one or three bits, and is stored as a “binary signal”.
  • each bit of the binary signal is referred to as a “bin” number.
  • a smaller index value is assigned to an encoding mode having a higher frequency of occurrence, and a corresponding binary signal is also set to be short in length, i.e. 1 bit in length.
  • the optimum encoding mode 7 a outputted from the encoding controlling unit 3 is inputted to the binarizing unit 92 , and is also inputted to the frequency information generating unit 93 .
  • the frequency information generating unit 93 counts the frequency of occurrence of the index value of the encoding mode included in this optimum encoding mode 7 a (the frequency of selection of the encoding mode which the encoding controlling unit has selected) to generate frequency information 94 , and outputs this frequency information to the binarization table updating unit 95 which will be mentioned later.
  • the probability table memory 97 holds a table for storing two or more sets of one symbol (MPS: Most Probable Symbol) having a higher probability of occurrence of the symbol values “0” and “1” in each bin included in the binary signal 103 , and the probability of occurrence of the symbol.
  • MPS Most Probable Symbol
  • FIG. 11 is a view showing an example of a probability table held by the probability table memory 97 .
  • a “probability table number” is assigned to each of discrete probability values ranging from 0.5 to 1.0 (“probabilities of occurrence”).
  • the state transition table memory 98 holds a table for storing a plurality of sets each having a “probability table number” stored in the probability table memory 97 , and a state transition from a probability state in which the MPS of “0” or “1”, which is shown by the probability table number, has not been encoded to a probability state in which the MPS of “0” or “1” has been encoded.
  • FIG. 12 is a view showing an example of a state transition table held by the state transition table memory 98 .
  • Each set of a “probability table number”, a “probability transition after LPS is encoded”, and a “probability transition after MPS is encoded”, which are shown in FIG. 12 corresponds to a probability table number shown in FIG. 11 .
  • this figure shows that at the time of a probability state having the “probability table number 1 ” enclosed by a box shown in FIG. 12 (at the time that the probability of occurrence of the MPS is 0.527, as shown in FIG.
  • the encoding of one symbol having a lower probability of occurrence causes the probability state to make a transition to the one having the probability table number 0 (the probability of occurrence of the MPS is 0.500, as shown in FIG. 11 ), as can be seen from the “probability transition after LPS is encoded”. More specifically, because the LPS occurs, the probability of occurrence of the MPS becomes small. In contrast with this, the figure shows that the encoding of the MPS causes the probability state to make a transition to the one having the probability table number 2 (the probability of occurrence of the MPS is 0.550, as shown in FIG. 11 ), as can be seen from the “probability transition after MPS is encoded”. More specifically, because the MPS occurs, the probability of occurrence of the MPS becomes large.
  • LPS Least Probable Symbol
  • the context generating unit 99 refers to the type indicating signal 100 indicating the type of the parameter to be encoded (the optimum encoding mode 7 a , an optimum prediction parameter 10 a or 18 a , or an optimum compression parameter 20 a ) which is inputted from the encoding controlling unit 3 , and the peripheral block information 101 to generate context identification information 102 for each bin of the binary signal 103 acquired by carrying out binarization on the parameter to be encoded.
  • the type indicating signal 100 indicates the optimum encoding mode 7 a of the macroblock to be encoded.
  • the peripheral block information 101 indicates the optimum encoding modes 7 a of macroblocks adjacent to the macroblock to be encoded.
  • a generation procedure for generating the context identification information which is carried out the context generating unit 99 will be explained.
  • FIG. 13( a ) is a view showing the binarization table shown in FIG. 10 in binary tree representation.
  • a macroblock to be encoded denoted by a thick-bordered box shown in FIG. 13( b ) and peripheral blocks A and B which are adjacent to this macroblock to be encoded as an example.
  • each black dot is referred to as a node, and each line which connects between nodes is referred to as a path.
  • the indexes of the multi valued signal to be binarized are assigned to the terminal nodes of the binary tree, respectively.
  • each node in a direction of the depth of the binary tree which is extending downwardly from an upper part to a lower part on the page, corresponds to its bin number, and a bit sequence which is acquired by connecting the symbols (each 0 or 1) respectively assigned to paths extending from the root node to each terminal node shows a binary signal 103 corresponding to the index of the multi valued signal assigned to the terminal node.
  • a parent node a node which is not any terminal node
  • one or more pieces of context identification information are prepared in accordance with the information about the peripheral blocks A and B.
  • the context generating unit 99 refers to the pieces of peripheral block information 101 about the adjacent peripheral blocks A and B to select either one of the three pieces of context identification information C 0 , C 1 , and C 2 in accordance with the following equation (4).
  • the context generating unit 99 outputs the selected context identification information as the context identification information 102 .
  • the above equation (4) is prepared on the assumption that when each of the peripheral blocks A and B is defined as a macroblock X, there is a high probability that the encoding mode of the macroblock to be encoded is “0” (mb_skip) when the encoding mode of each of the peripheral blocks A and B is “0” (mb_skip). Therefore, the context identification information 102 selected in accordance with the above equation (4) is based on the same assumption.
  • One context identification information (C 3 , C 4 , or C 5 ) is assigned to each of the parent nodes other than the root node.
  • the context information identified by the context identification information 102 holds the value (0 or 1) of the MPS and the probability table number which approximates the probability of occurrence of the value. Now, the context information is placed in its initial state.
  • the context information memory 96 stores this context information.
  • the arithmetic encoding processing operation unit 104 carries out arithmetic encoding on each bin of the binary signal 103 with one or three bits inputted from the binarizing unit 92 to generate an encoded bit sequence 111 , and multiplexes this encoded bit sequence into the bitstream 30 .
  • an arithmetic encoding procedure based on the context information will be explained.
  • the arithmetic encoding processing operation unit 104 refers to the context information memory 96 first to acquire the context information 106 based on the context identification information 102 corresponding to the bin 0 of the binary signal 103 . Next, the arithmetic encoding processing operation unit 104 refers to the probability table memory 97 to specify the probability 108 of occurrence of the MPS of the bin 0 corresponding to the probability table number 107 held by the context information 106 .
  • the arithmetic encoding processing operation unit 104 carries out arithmetic encoding on the symbol value 109 (0 or 1) of the bin 0 on the basis of the value (0 or 1) of the MPS held by the context information 106 and the specified probability 108 of occurrence of the MPS.
  • the arithmetic encoding processing operation unit 104 refers to the state transition table memory 98 and acquires the probability table number 110 at a time after the symbol of the bin 0 is encoded on the basis of both the probability table number 107 held by the context information 106 , and the symbol value 109 of the bin 0 on which arithmetic encoding has been carried out previously.
  • the arithmetic encoding processing operation unit 104 updates the value of the probability table number (i.e. the probability table number 107 ) of the context information 106 of the bin 0 stored in the context information memory 96 to the probability table number at a time after the state transition (i.e. the probability table number 110 at a time after the symbol of the bin 0 is encoded, which has been previously acquired from the state transition table memory 98 ).
  • the arithmetic encoding processing operation unit 104 also carries out arithmetic encoding based on the context information 106 identified by the context identification information 102 on the symbol of each of the bins 1 and 2 , and then updates the context information 106 after encoding the symbol of each of the bins, like in the case of carrying out arithmetic encoding on the symbol of the bin 0 .
  • the arithmetic encoding processing operation unit 104 outputs an encoded bit sequence 111 which the arithmetic encoding processing operation unit has acquired by carrying out arithmetic encoding on the symbols of all the bins, and the variable length encoding unit 23 multiplexes the encoded bit sequence into the bitstream 30 .
  • the context information 106 identified by the context identification information 102 is updated every time when arithmetic encoding is carried out on the symbol of each bin. More specifically, this update means that the probability state of each node makes a transition every time when the symbol of each bin is encoded.
  • Initialization of the context information 106 i.e. reset of the probability state is carried out by the above-mentioned initializing unit 90 . While the initializing unit 90 initializes the context information in accordance with an instruction shown by the context information initialization flag 91 of the encoding controlling unit 3 , the initializing unit 90 carries out this initialization at the head of each slice or the like.
  • a plurality of sets can be prepared in advance for the initial state of each context information 106 (the value of the MPS, and the initial value of the probability table number which approximates the probability of occurrence of the value), and the encoding controlling unit 3 can include information showing which initial state is to be selected from the plurality of sets in the context information initialization flag 91 and notify this context information initialization flag to the initializing unit 90 .
  • the binarization table updating unit 95 refers to the frequency information 94 showing the frequency of occurrence of the index value of the parameter to be encoded (in this case, the optimum encoding mode 7 a ) which is generated by the frequency information generating unit 93 on the basis of the binarization table update flag 113 notified thereto from the encoding controlling unit 3 to update the binarization table memory 105 .
  • the binarization table updating unit 95 a procedure for updating the binarization table which is carried out by the binarization table updating unit 95 will be explained.
  • the binarization table updating unit updates a correspondence between the encoding modes in the binarization table and the indexes in accordance with the frequency of occurrence of the encoding mode specified by the optimum encoding mode 7 a which is the parameter to be encoded in such a way as to be able to binarize the encoding mode having the highest frequency of occurrence into a short codeword, thereby reducing the code amount.
  • FIG. 14 is a view showing an example of the binarization table updated. Assuming that the binarization table which is yet to be updated is placed in a state shown in FIG. 10 , FIG. 14 shows a state in which the binarization table has been updated. For example, when the frequency of occurrence of mb_mode 3 is the highest, the binarization table updating unit 95 assigns the smallest index value to mb_mode 3 in accordance with the frequency information 94 in such a way that a binary signal having a short codeword is assigned to mb_mode 3 .
  • the binarization table updating unit 95 needs to generate binarization table update identification information 112 for enabling the decoding device to identify the updated binarization table and multiplex the binarization table update identification information into the bitstream 30 .
  • an ID for enabling the identification of each parameter to be encoded can be provided in advance for both the encoding device and the decoding device, and the binarization table updating unit 95 can be constructed in such a way as to output the ID of the binarization table updated as binarization table updating identification information 112 , and multiplex this binarization table update identification information into the bitstream 30 .
  • the encoding controlling unit 3 carries out control of an update time by referring to the frequency information 94 of the parameter to be encoded at the head of each slice, and, when determining that the distribution of the frequency of occurrence of the parameter to be encoded has changed and deviated from a predetermined permissible range, outputting a binarization table update flag 113 .
  • the variable length encoding unit 23 should just multiplex the binarization table update flag 113 into the slice header of the bitstream 30 .
  • variable length encoding unit 23 multiplexes the binarization table updating identification information 112 showing which binarization table of the binarization tables of the encoding mode, a compression parameter, and a prediction parameter has been updated into the bitstream 30 .
  • the encoding controlling unit 3 can notify the update of the binarization table at a time other than the time that the encoding controlling unit processes the head of each slice. For example, the encoding controlling unit can output the binarization table update flag 113 at the time that the encoding controlling unit processes the head of an arbitrary macroblock to instruct an update of the binarization table.
  • the binarization table updating unit 95 needs to output information for identifying the position of the macroblock for which the binarization table has been updated, and the variable length encoding unit 23 also needs to multiplex the information into the bitstream 30 .
  • the encoding controlling unit 3 When outputting the binarization table update flag 113 to the binarization table updating unit 95 to cause this binarization table updating unit to update the binarization table, the encoding controlling unit 3 needs to output the context information initialization flag 91 to the initializing unit 90 to cause the initializing unit to initialize the context information memory 96 .
  • FIG. 15 is a block diagram showing the internal structure of the variable length decoding unit 61 of the moving image decoding device in accordance with Embodiment 2 of the present invention. Further, because the structure of the moving image decoding device in accordance with this Embodiment 2 is the same as that in accordance with above-mentioned Embodiment 1, and the operation of each component except the variable length decoding unit 61 is the same as that in accordance with above-mentioned Embodiment 1, an explanation will be made by using FIGS. 1 to 8 .
  • the variable length decoding unit 61 shown in FIG. 15 includes an arithmetic decoding processing operation unit 127 for referring to context identification information 126 which a context generating unit 122 generates, a context information memory 128 , a probability table memory 131 , and a state transition table memory 135 to carry out arithmetic decoding on an encoded bit sequence 133 showing an optimum encoding mode (or an optimum prediction parameter 63 or an optimum compression parameter 65 ) multiplexed into a bitstream 60 to generate a binary signal 137 , a binarization table memory 143 for storing a binarization table 139 indicating a correspondence between the optimum encoding mode 62 expressed by a binary signal (or an optimum prediction parameter 63 or an optimum compression parameter 65 ) and a multi valued signal, and an inverse binarizing unit 138 for converting the binary signal 137 which the arithmetic decoding processing operation unit 127 has generated into a decoded value 140 of a multi valued signal by
  • variable length decoding procedure carried out by the variable length decoding unit 61 will be explained by taking the optimum encoding mode 62 of a macroblock included in the bitstream 60 as an example of a parameter to be entropy-decoded. Because the variable length decoding unit 61 can also variable-length-decode an optimum prediction parameter 63 or an optimum compression parameter 65 which is similarly a parameter to be decoded in accordance with the same procedure as that in accordance with which the variable length decoding unit variable-length-decodes the optimum encoding mode 62 , the explanation of the variable length decoding procedure carried out on an optimum prediction parameter or an optimum compression parameter will be omitted.
  • Context initialization information 121 an encoded bit sequence 133 , a binarization table update flag 142 , and binarization table updating identification information 144 which are multiplexed into the bitstream 60 by the encoding device are included in the bitstream 60 in accordance with this Embodiment 2. The details of each of these pieces of information will be mentioned later.
  • An initializing unit 120 initializes the context information stored in the context information memory 128 at the head of each slice or the like.
  • a plurality of sets can be prepared for an initial state of the context information (the value of the MPS, and an initial value of the probability table number which approximates the probability of occurrence of the MPS value) in the initializing unit 120 , and an initial state corresponding to the decoded value of the context initialization information 121 can be selected from the plurality of sets.
  • the context generating unit 122 refers to both a type indicating signal 123 showing the type of the parameter to be decoded (the optimum encoding mode 62 , an optimum prediction parameter 63 , or an optimum compression parameter 65 ) and peripheral block information 124 to generate context identification information 126 .
  • the type indicating signal 123 shows the type of the parameter to be decoded, and the decoding device determines what the parameter to be decoded is in accordance with a syntax held by the variable length decoding unit 61 . Therefore, the encoding device and the decoding device need to hold the same syntax, and, in this embodiment, it is assumed that the encoding controlling unit 3 of the encoding device holds the syntax.
  • the encoding device sequentially outputs the type of the parameter to be encoded next and the value (index value) of the parameter, i.e. the type indicating signal 100 to the variable length encoding unit 23 in accordance with the syntax held by the encoding controlling unit 3 .
  • the peripheral block information 124 includes the encoding mode which is acquired by decoding each macroblock or subblock, and is stored in a memory (not shown) in the variable length decoding unit 61 in order to use the peripheral block information 124 as information used for subsequent decoding of a macroblock or subblock and is outputted to the context generating unit 122 as needed.
  • a generation procedure for generating context identification information 126 which is carried out by the context generating unit 122 is the same as the operation of the context generating unit 99 disposed in the encoding device. Also the context generating unit 122 in the decoding device generates context identification information 126 for each bin of the binarization table 139 which is to be referred to by the inverse binarizing unit 138 .
  • the value (0 or 1) of the MPS, and the probability table number for specifying the probability of occurrence of the value of the MPS are held as probability information used for carrying out arithmetic decoding on the bin.
  • the probability table memory 131 and the state transition table memory 135 store the same probability table ( FIG. 11 ) as the probability table memory 97 of the encoding device and the same state transition table ( FIG. 12 ) as the state transition table memory 98 of the encoding device, respectively.
  • the arithmetic decoding processing operation unit 127 carries out arithmetic decoding on the encoded bit sequence 133 multiplexed into the bitstream 60 on a per-bin basis to generate a binary signal 137 , and outputs this binary signal to the inverse binarizing unit 138 .
  • the arithmetic decoding processing operation unit 127 refers to the context information memory 128 first to acquire the context information 129 based on the context identification information 126 corresponding to each bin of the encoded bit sequence 133 . Next, the arithmetic decoding processing operation unit 127 refers to the probability table memory 131 to specify the probability 132 of occurrence of the MPS of each bin corresponding to the probability table number 130 held by the context information 129 .
  • the arithmetic decoding processing operation unit 127 then carries out arithmetic decoding on the encoded bit sequence 133 inputted to the arithmetic decoding processing operation unit 127 on the basis of the value (0 or 1) of the MPS held by the context information 129 and the specified probability 132 of occurrence of the MPS to acquire the symbol value 134 (0 or 1) of each bin.
  • the arithmetic decoding processing operation unit 127 After acquiring the symbol value of each bin through the decoding, the arithmetic decoding processing operation unit 127 refers to the state transition table memory 135 , and acquires the probability table number 136 at a time after the symbol of each bin is decoded (at a time after a state transition is made) on the basis of the symbol value 134 of each decoded bin and the probability table number 130 held by the context information 129 and in accordance with the same procedure as that carried out by the arithmetic encoding processing operation unit 104 of the encoding device.
  • the arithmetic decoding processing operation unit 127 updates the value of the probability table number (i.e. the probability table number 130 ) of the context information 129 of each bin which is stored in the context information memory 128 to the probability table number at a time after a state transition is made (i.e. the probability table number 136 at a time after the symbol of each bin is decoded which has been previously acquired from the state transition table memory 135 ).
  • the arithmetic decoding processing operation unit 127 outputs a binary signal 137 in which the symbols of the bins acquired as results of performing the above-mentioned arithmetic decoding on the encoded bit sequence are connected to one another to the inverse binarizing unit 138 .
  • the inverse binarizing unit 138 selects the same binarization table 139 as that used at the time of the encoding from among the binarization tables stored in the binarization table memory 143 and prepared for all the types of parameters to be decoded and refers to the binarization table selected thereby, and selectively outputs the decoded value 140 of the parameter to be decoded from the binary signal 137 inputted thereto from the arithmetic decoding processing operation unit 127 .
  • the binarization table 139 is the same as the binarization table in the encoding device shown in FIG. 10 .
  • the binarization table updating unit 141 updates the binarization table stored in the binarization table memory 143 on the basis of the binarization table update flag 142 and the binarization table updating identification information 144 which are decoded from the bitstream 60 .
  • the binarization table update flag 142 is information which corresponds to the binarization table update flag 113 in the encoding device, and which is included in the header information or the like of the bitstream 60 and shows whether or not there is an update to the binarization table.
  • the binarization table updating identification information 144 is decoded from the bitstream 60 .
  • the binarization table updating identification information 144 is information which corresponds to the binarization table updating identification information 112 in the encoding device, and which is used for identifying the binarization table of a parameter updated by the encoding device. For example, when a plurality of binarization tables are provided in advance for each parameter to be encoded, as mentioned above, an ID which enables each parameter to be encoded to be identified and an ID of each of the binarization tables are provided in advance both in the encoding device side and in the decoding device, and the binarization table updating unit 141 updates the binarization table corresponding to the ID value in the binarization table updating identification information 144 which is decoded from the bitstream 60 . In this example, two types of binarization tables shown in FIGS.
  • the binarization table updating unit 141 must necessarily select the binarization table corresponding to the ID included in the binarization table updating identification information 144 by simply carrying out an update process in accordance with the binarization table update flag 142 and the binarization table updating identification information 144 . Therefore, the binarization table updated enters a state shown in FIG. 14 , and becomes the same as the binarization table which has been updated in the encoding device.
  • the moving image encoding device in accordance with Embodiment 2 is constructed in such a way that the encoding controlling unit 3 selects and outputs a parameter to be encoded, such as an optimum encoding mode 7 a which provides an optimum degree of encoding efficiency, an optimum prediction parameter 10 a or 18 a , or an optimum compression parameter 20 a , the binarizing unit 92 of the variable length encoding unit 23 converts the parameter to be encoded expressed by a multi valued signal into a binary signal 103 by using the binarization table stored in the binarization table memory 105 , the arithmetic encoding processing operation unit 104 carries out arithmetic encoding on the binary signal 103 to output an encoded bit sequence 111 , the frequency information generating unit 93 generates frequency information 94 of the parameter to be encoded, and the binarization table updating unit 95 updates the correspondence between the multi valued signal in the binarization table and the binary signal on the basis of the frequency information 94
  • the moving image decoding device in accordance with Embodiment 2 is constructed in accordance with the structure of the binarization table updating unit in such a way that the arithmetic decoding processing operation unit 127 of the variable length decoding unit 61 carries out arithmetic decoding on the encoded bit sequence 133 multiplexed into the bitstream 60 to generate a binary signal 137 , the inverse binarizing unit 138 uses the binarization table 139 of the binarization table memory 143 to convert the binary signal 137 into a multi valued signal and acquire a decoded value 140 , and the binarization table updating unit 141 updates a predetermined binarization table stored in the binarization table memory 143 on the basis of the
  • the moving image decoding device can update the binarization table in accordance with the same procedure as that carried out by the moving image encoding device and can carryout inverse binarization on the parameter to be encoded, the moving image encoding device in accordance with Embodiment 2 can decode the encoded bitstream correctly.
  • Embodiment 3 a variant of the generating process of generating a prediction image by using a motion-compensated prediction which is made by the motion-compensated prediction unit 9 in the moving image encoding device and the moving image decoding device in accordance with any one of above-mentioned Embodiments 1 and 2 will be explained
  • a motion-compensated prediction unit 9 of a moving image encoding device in accordance with this Embodiment 3 will be explained. Further, because the structure of the moving image encoding device in accordance with this Embodiment 3 is the same as that in accordance with above-mentioned Embodiment 1 or 2, and the operation of each component except the motion-compensated prediction unit 9 is the same as that in accordance with above-mentioned Embodiment 1 or 2, an explanation will be made by using FIGS. 1 to 15 .
  • the motion-compensated prediction unit 9 in accordance with this Embodiment 3 has the same structure as and operates in the same way as that in accordance with above-mentioned Embodiment 1 or 2, with the exception that a structure and an operation associated with a prediction image generating process having virtual sample accuracy differ from those in accordance with any one of above-mentioned Embodiments 1 and 2. More specifically, in accordance with above-mentioned Embodiments 1 and 2 , as shown in FIG.
  • the interpolated image generating unit 43 of the motion-compensated prediction unit 9 generates reference image data having virtual pixel accuracy, such as half-pixel or 1 ⁇ 4 pixel accuracy, and, when generating a prediction image 45 on the basis of this reference image data having virtual pixel accuracy, generates virtual pixels by implementing an interpolation arithmetic operation with a 6-tap filter using six integer pixels running in a vertical or horizontal direction to generate a prediction image, like in the case of the MPEG-4 AVC standards.
  • the motion-compensated prediction unit 9 in accordance with this Embodiment 3 enlarges a reference image 15 having integer pixel accuracy stored in a motion-compensated prediction frame memory 14 by carrying out a super resolution process on the reference image 15 to generate a reference image 207 having virtual pixel accuracy, and then generates a prediction image on the basis of this reference image 207 having virtual pixel accuracy.
  • an interpolated image generating unit 43 in accordance with this Embodiment 3 specifies one or more frames of reference images 15 from the motion-compensated prediction frame memory 14 , and a motion detecting unit 42 detects a motion vector 44 in a predetermined motion search range on the reference image 15 specified by the interpolated image generating unit.
  • the detection of the motion vector is implemented by using a motion vector having virtual pixel accuracy, like in the case of the MPEG-4 AVC standards or the like.
  • an interpolation arithmetic operation is performed on pixel information (referred to as integer pixels) which the reference image has to generate virtual samples (pixels) between the integer pixels, and these virtual samples are used as a reference image.
  • the interpolated image generating unit 43 in accordance with this Embodiment 3 uses a super resolution technique disclosed by “W. T. Freeman, E. C. Pasztor and O. T. Carmichael, “Learning Low-Level Vision”, International Journal of Computer Vision, vol. 40, No. 1, 2000′′ to generate a reference image having virtual pixel accuracy.
  • the motion-compensated prediction unit 9 carries out a super resolution image generating process to generate a reference image 207 having virtual pixel accuracy from the reference image data stored in the motion-compensated prediction frame memory 14 , and the motion detecting unit 42 carries out a motion vector search process using the reference image will be mentioned hereafter.
  • FIG. 16 is a block diagram showing the internal structure of the interpolated image generating unit 43 of the motion-compensated prediction unit 9 of the moving image encoding device in accordance with Embodiment 3 of the present invention.
  • 16 includes an image enlargement processing unit 205 for carrying out an enlarging process on the reference image 15 stored in the motion-compensated prediction frame memory 14 , an image reduction processing unit 200 for carrying out a reducing process on the reference image 15 , a high frequency feature extracting unit 201 a for extracting a feature quantity of a high frequency region component from the image reduction processing unit 200 , a high frequency feature extracting unit 201 b for extracting a feature quantity of a high frequency region component from the reference image 15 , a correlation calculating unit 202 for calculating the value of a correlation between the feature quantities, a high frequency component estimating unit 203 for estimating a high frequency component from both the value of the correlation and prior learned data stored in a high frequency component pattern memory 204 , and an adding unit 206 for correcting a high frequency component of the enlarged image by using the estimated high frequency component to generate a reference image 207 having virtual pixel accuracy.
  • the interpolated image generating unit shown in FIG. 16 when the reference image 15 in the range used for a motion search process is inputted from the reference image data stored in the motion-compensated prediction frame memory 14 to the interpolated image generating unit 43 , this reference image 15 is inputted to the image reduction processing unit 200 , to the high frequency feature extracting unit 201 b , and to the image enlargement processing unit 205 .
  • the image reduction processing unit 200 generates a reduced image whose height and width are respectively reduced to 1/N times (N is a power of 2, such as 2 or 4) the original height and width from the reference image 15 , and outputs the reduced image to the high frequency feature extracting unit 201 a .
  • N is a power of 2, such as 2 or 4
  • a typical image reduction filter implements this reducing process.
  • the high frequency feature extracting unit 201 a extracts a first feature quantity associated with a high frequency component, such as an edge component, from the reduced image which is generated by the image reduction processing unit 200 .
  • a first feature quantity for example, a parameter showing a DCT in a local block or a Wavelet transform coefficient distribution can be used.
  • the high frequency feature extracting unit 201 b carries out a high frequency feature extracting process similar to that carried out by the high frequency feature extracting unit 201 a , and extracts a second feature quantity having a frequency component region different from that of the first feature quantity from the reference image 15 .
  • the second feature quantity is outputted to the correlation calculating unit 202 , and is also outputted to the high frequency component estimating unit 203 .
  • the correlation calculating unit 202 calculates the value of a feature-quantity-based correlation in a high frequency component region between the reference image 15 and the reduced image in units of one local block. As this value of the correlation, for example, a distance between the first feature quantity and the second feature quantity is calculated.
  • the high frequency component estimating unit 203 specifies a prior learned pattern of the high frequency component from the high frequency component pattern memory 204 on the basis of both the second feature quantity inputted thereto from the high frequency feature extracting unit 201 b , and the value of the correlation inputted thereto from the correlation calculating unit 202 , and estimates and generates a high frequency component which the reference image 207 having virtual pixel accuracy should have.
  • the generated high frequency component is outputted to the adding unit 206 .
  • the image enlargement processing unit 205 performs either an interpolation arithmetic operation with a 6-tap filter using six integer pixels running in a vertical or horizontal direction or enlargement filtering, such as bilinear filtering, on the inputted reference image 15 , like in the case of performing a generating process of generating samples having half pixel accuracy in accordance with the MPEG-4 AVC standards, to enlarge each of the height and width of the reference image 15 by N times to generate an enlarged image.
  • the adding unit 206 adds the high frequency component inputted thereto from the high frequency component estimating unit 203 to the enlarged image inputted thereto from the image enlargement processing unit 205 to generate an enlarged reference image. More specifically, the adding unit corrects the high frequency component of the enlarged image to generate an enlarged reference image whose height and width are respectively enlarged to N times the original height and width.
  • the interpolated image generating unit 43 uses this enlarged reference image data as a reference image 207 having virtual pixel accuracy in which 1/N is set to 1.
  • the interpolated image generating unit 43 can be alternatively constructed in such away as to, after generating a reference image 207 having half pixel (1 ⁇ 2 pixel) accuracy by setting N to 2, generate virtual samples (pixels) having 1 ⁇ 4 pixel accuracy by performing an interpolation arithmetic operation using a filter for acquiring a mean value of adjacent 1 ⁇ 2 pixels or integer pixels.
  • the interpolated image generating unit 43 can be constructed in such a way as to include a unit for switching whether or not to add the high frequency component outputted by the high frequency component estimating unit 203 to the enlarged image outputted by the image enlargement processing unit 205 to control the result of the generation of the reference image 207 having virtual pixel accuracy in addition to the structure shown in FIG. 16 .
  • the interpolated image generating unit is constructed in this way, there is provided an advantage of suppressing a bad influence upon the encoding efficiency when the estimation accuracy of the high frequency component estimating unit 203 is bad for some reason such as an exceptional image pattern.
  • the moving image encoding device When the adding unit 206 selectively determines whether or not to add the high frequency component outputted by the high frequency component estimating unit 203 to the enlarged image, the moving image encoding device generates a prediction image 45 for both the case of adding them and the case of not adding them and then carries out a motion-compensated prediction, and encodes the results of the motion-compensated prediction and determines one of the prediction images which provides a higher degree of efficiency. The moving image encoding device then multiplexes adding process information showing whether the adding unit has added the high frequency component to the enlarged image into the bitstream 30 as control information.
  • the interpolated image generating unit 43 can uniquely determine whether or not to add the high frequency component to the enlarged image from another parameter to be multiplexed into the bitstream 30 to control the adding process carried out by the adding unit 206 .
  • determining whether or not to add the high frequency component to the enlarged image from another parameter for example, there can be provided a method of using the type of an encoding mode 7 as shown in FIG. 2A or 2 B. When an encoding mode showing that the division of a macroblock into motion compensation region blocks is fine is selected, there is a high probability that the image pattern has a violent movement.
  • the interpolated image generating unit 43 controls the adding unit 206 to cause this adding unit not to add the high frequency component outputted by the high frequency component estimating unit 203 to the enlarged image.
  • the interpolated image generating unit 43 controls the adding unit 206 to cause this adding unit to add the high frequency component outputted by the high frequency component estimating unit 203 to the enlarged image.
  • a parameter such as the size of the motion vector or a variation in the motion vector field in consideration of adjacent areas
  • the interpolated image generating unit 43 of the motion-compensated prediction unit 9 shares the type of the parameter with the decoding device to determine whether or not to add the high frequency component to the enlarged image.
  • the moving image encoding device does not have to multiplex the control information about the adding process directly into the bitstream 30 , thereby being able to improve the compression efficiency.
  • the moving image encoding device can be constructed in such a way as to perform the above-mentioned super resolution process on the reference image 15 , which is to be stored in the motion-compensated prediction frame memory 14 , to convert the reference image into a reference image 207 having virtual pixel accuracy before storing the reference image in the motion-compensated prediction frame memory 14 , and, after that, store the reference image 207 in the motion-compensated prediction frame memory.
  • the moving image encoding device does not have to sequentially carry out the super resolution process during the motion vector search and during the prediction image generation and the processing load on the motion-compensated prediction process itself can be reduced, and the moving image encoding device becomes able to carry out the frame encoding process and the generating process of generating a reference image 207 having virtual pixel accuracy in parallel, and can speed up the processes.
  • FIG. 3 An example of a motion vector detection procedure for detecting a motion vector having virtual pixel accuracy using the reference image 207 having virtual pixel accuracy will be shown by using FIG. 3 .
  • the interpolated image generating unit 43 generates a prediction image 45 for the motion vector 44 having integer pixel accuracy in the predetermined motion search range of a motion compensation region block image 41 .
  • the prediction image 45 (prediction image 17 ) generated at integer pixel accuracy is outputted to the subtraction unit 12 , and is subtracted from the motion compensation region block image 41 (macro/subblock image 5 ) by the subtraction unit 12 , so that the result of the subtraction is defined as a prediction difference signal 13 .
  • the encoding controlling unit 3 evaluates a degree of prediction efficiency for the prediction difference signal 13 and for the motion vector 44 (prediction parameter 18 ) having integer pixel accuracy. Because the evaluation of this prediction efficiency can be carried out in accordance with the above equation (1) explained in above-mentioned Embodiment 1, the explanation of the evaluation will be omitted hereafter.
  • the interpolated image generating unit 43 generates a prediction image 45 by using the reference image 207 having virtual pixel accuracy generated within the interpolated image generating unit 43 shown in FIG. 16 for a motion vector 44 having 1 ⁇ 2 pixel accuracy located in the vicinity of the motion vector having integer pixel accuracy which is determined in accordance with the above-mentioned “motion vector detection procedure I”. After that, like in the case of the above-mentioned “motion vector detection procedure I”, the prediction image 45 (prediction image 17 ) generated at 1 ⁇ 2 pixel accuracy is subtracted from the motion compensation region block image 41 (macro/subblock image 5 ) by the subtraction unit 12 to acquire a prediction difference signal 13 .
  • the encoding controlling unit 3 evaluates a degree of prediction efficiency for this prediction difference signal 13 and for the motion vector 44 (prediction parameter 18 ) having 1 ⁇ 2 pixel accuracy, and selectively determines a motion vector 44 having 1 ⁇ 2 pixel accuracy which minimizes the prediction cost J 1 from among one or more motion vectors having 1 ⁇ 2 pixel accuracy located in the vicinity of the motion vector having integer pixel accuracy.
  • the encoding controlling unit 3 and the motion-compensated prediction unit 9 selectively determine a motion vector 44 having 1 ⁇ 4 pixel accuracy which minimizes the prediction cost J 1 from one or more motion vectors having 1 ⁇ 4 pixel accuracy located in the vicinity of the motion vector having 1 ⁇ 2 pixel accuracy which is determined in accordance with the above-mentioned “motion vector detection procedure II”.
  • the encoding controlling unit 3 and the motion-compensated prediction unit 9 similarly detect a motion vector having virtual pixel accuracy until the motion vector detected thereby has a predetermined degree of accuracy.
  • the motion-compensated prediction unit 9 outputs the virtual-pixel-accuracy motion vector having the predetermined accuracy, which is determined for each motion compensation region block image 41 which is one of a plurality of blocks into which the macro/subblock image 5 is divided and each of which is a unit for the motion compensation shown by the encoding mode 7 , and the identification number of the reference image specified by the motion vector as prediction parameters 18 .
  • the motion-compensated prediction unit 9 also outputs the prediction image 45 (prediction image 17 ) which is generated by using the prediction parameters 18 to the subtraction unit 12 , and the subtraction unit 12 subtracts the prediction image 45 from the macro/subblock image 5 to acquire a prediction difference signal 13 .
  • the prediction difference signal 13 outputted from the subtraction unit 12 is outputted to the transformation/quantization unit 19 . Because subsequent processes carried out after that are the same as those explained in above-mentioned Embodiment 1, the explanation of the processes will be omitted hereafter.
  • the moving image decoding device in accordance with this Embodiment 3 has the same structure as the moving image decoding device in accordance with any one of above-mentioned Embodiments 1 and 2, with the exception that the moving image decoding device in accordance with this Embodiment 3 has a structure different from the structure associated with the prediction image generating process having virtual pixel accuracy carried out by the motion-compensated prediction unit 70 in accordance with any one of above-mentioned Embodiments 1 and 2, and the moving image decoding device in accordance with this Embodiment 3 operates in a different way from that in accordance with any one of above-mentioned Embodiments 1 and 2 when carrying out the prediction image generating process, the structure and operation of the moving image decoding device in accordance with this Embodiment 3 will be explained by using FIGS. 1 to 16 .
  • the motion-compensated prediction unit 70 when generating a prediction image on the basis of a reference image having virtual pixel accuracy, such as half-pixel or 1 ⁇ 4 pixel accuracy, the motion-compensated prediction unit 70 generates virtual pixels by implementing an interpolation arithmetic operation with a 6-tap filter using six integer pixels running in a vertical or horizontal direction, or the like to generate a prediction image, like in the case of the MPEG-4 AVC standards.
  • a motion-compensated prediction unit 70 in accordance with this Embodiment 3 enlarges a reference image 76 having integer pixel accuracy stored in a motion-compensated prediction frame memory 75 by carrying out a super resolution process on the reference image. As a result, the motion-compensated prediction unit generates a reference image having virtual pixel accuracy.
  • the motion-compensated prediction unit 70 in accordance with this Embodiment 3 generates a prediction image 72 from the reference image 76 stored in the motion-compensated prediction frame memory 75 on the basis of motion vectors included in the inputted optimum prediction parameters 63 , the identification number (reference image index) of the reference image specified by each of the motion vectors, and so on, and outputs the prediction image, like that in accordance with any one of above-mentioned Embodiments 1 and 2.
  • An adding unit 73 adds the prediction image 72 inputted from the motion-compensated prediction unit 70 to prediction difference signal decoded values 67 inputted from an inverse quantization/inverse transformation unit 66 to generate a decoded image 74 .
  • a generation method of generating the prediction image 72 which is implemented by the motion-compensated prediction unit 70 corresponds to the operation of the motion-compensated prediction unit 9 in the encoding device from which the process of searching through a plurality of reference images for motion vectors (corresponding to the operations of the motion detecting unit 42 and the interpolated image generating unit 43 shown in FIG. 3 ) is excluded.
  • the motion-compensated prediction unit carries out only the process of generating the prediction image 72 in accordance with optimum prediction parameters 63 provided thereto from a variable length decoding unit 61 .
  • the motion-compensated prediction unit 70 When generating a prediction image 72 at virtual pixel accuracy, the motion-compensated prediction unit 70 carries out the same process as that shown in FIG. 16 on the reference image 76 stored in the motion-compensated prediction frame memory 75 and specified by the identification number (reference image index) of the reference image to generate a reference image having virtual pixel accuracy, and then generates a prediction image 72 by using the decoded motion vector.
  • the encoding device has selectively determined whether or not to add the high frequency component outputted by the high frequency component estimating unit 203 shown in FIG.
  • the decoding device extracts the control information showing whether the encoding device has carries out the adding process from the bitstream 60 , or uniquely determines whether the encoding device has added the high frequency component to the enlarged image from another parameter to control an adding process in the motion-compensated prediction unit 70 .
  • a parameter such as the size of the motion vector or a variation in the motion vector field in consideration of adjacent areas, can be used.
  • the motion-compensated prediction unit 70 shares the type of the parameter with the encoding device and determines whether the encoding device has added the high frequency component to the enlarged image. As a result, the moving image encoding device does not have to multiplex the control information about the adding process directly into the bitstream 30 , thereby being able to improve the compression efficiency.
  • the motion-compensated prediction unit 70 can carry out the process of generating a reference image having virtual pixel accuracy only when the motion vectors included in the optimum prediction parameters 18 a which are outputted from the encoding device (i.e. optimum prediction parameters 63 in the decoding device) indicate the virtual pixel accuracy.
  • the motion-compensated prediction unit 9 switches between the use of the reference image 15 stored in the motion-compensated prediction frame memory 14 , and the generation and use of a reference image 207 having virtual pixel accuracy by means of the interpolated image generating unit 43 in accordance with the motion vector, and generates a prediction image 17 from either the reference image 15 or the reference image 207 having virtual pixel accuracy.
  • the motion-compensated prediction unit can be constructed in such a way as to carryout the process shown in FIG. 16 on the reference image which is yet to be stored in the motion-compensated prediction frame memory 75 and store the reference image having virtual pixel accuracy on which the enlargement process has been carried out and in which a high frequency component has been corrected in the motion-compensated prediction frame memory 75 .
  • the size of a memory which should be prepared as the motion-compensated prediction frame memory 75 increases, it is not necessary to duplicately carry out the process shown in FIG. 16 when the number of times that the motion vector points to a pixel at the same virtual sample position. Therefore, the amount of computation can be reduced.
  • the motion-compensated prediction unit 70 can be constructed in such a way as to carry out the process shown in FIG. 16 on the target region while limiting this region only to the range. What is necessary is just to make the decoding device know the range of displacement which is pointed to by the motion vector by, for example, multiplexing a value range showing the range of displacement which is pointed to by the motion vector into the bitstream 60 to transmit the value range to the decoding device, or making both the encoding device and the decoding device mutually determine and set the value range in their operations.
  • the moving image encoding device in accordance with Embodiment 3 is constructed in such a way that the motion-compensated prediction unit 9 has the interpolated image generating unit 43 for carrying out an enlarging process on the reference image 15 stored in the motion-compensated prediction frame memory 14 , and also correcting a high frequency component to generate a reference image 207 having virtual pixel accuracy, and switches between the use of the reference image 15 or the generation and use of the reference image 207 having virtual pixel accuracy in accordance with the motion vector to generate a prediction image 17 , even when carrying out high compression on the inputted video signal 1 including many high frequency components, such as fine edges, the moving image encoding device can generate the prediction image 17 which is to be generated by using a motion-compensated prediction from the reference image including many high frequency components, thereby being able to compression-encode the inputted video signal efficiently.
  • the moving image decoding device in accordance with Embodiment 3 is constructed in such away that the motion-compensated prediction unit 70 has the interpolated image generating unit for generating a reference image having virtual pixel accuracy in accordance with the same procedure as that carried out by the moving image encoding device, and switches between the use of the reference image 76 stored in the motion-compensated prediction frame memory 75 or the generation and use of the reference image having virtual pixel accuracy in accordance with the motion vector multiplexed into the bitstream 60 to generate a prediction image 72 , the moving image decoding device can correctly decode the bitstream encoded by the moving image encoding device in accordance with Embodiment 3.
  • the interpolated image generating unit 43 in accordance with above-mentioned Embodiment 3 carries out the super resolution process based on the above-mentioned technique disclosed by W. T. Freeman et al. (2000) to generate a reference image 207 having virtual pixel accuracy.
  • the super resolution process is not limited the one based on the above-mentioned technique, and the interpolated image generating unit can be constructed in such a way as to use another arbitrary super resolution technique to generate a reference image 207 having virtual pixel accuracy.
  • a moving image encoding program in which the processes carried out by the block dividing unit 2 , the encoding controlling unit 3 , the switching unit 6 , the intra-prediction unit 8 , the motion-compensated prediction unit 9 , the motion-compensated prediction frame memory 14 , the transformation/quantization unit 19 , the inverse quantization/inverse transformation unit 22 , the variable length encoding unit 23 , the loop filter unit 27 , and the memory 28 for intra prediction are described can be stored in a memory of the computer, and a CPU of the computer can be made to execute the moving image encoding program stored in the memory.
  • a moving image decoding program in which the processes carried out by the variable length decoding unit 61 , the inverse quantization/inverse transformation unit 66 , the switching unit 68 , the intra-prediction unit 69 , the motion-compensated prediction unit 70 , the motion-compensated prediction frame memory 75 , the memory 77 for intra prediction, and the loop filter unit 78 are described can be stored in a memory of the computer, and a CPU of the computer can be made to execute the moving image decoding program stored in the memory.
  • the moving image encoding device and the moving image decoding device in accordance with the present invention can switch among transformation block sizes adaptively to compression-encode an inputted moving image for each region which serves as a unit for motion-compensated prediction in each macroblock
  • the moving image encoding device and the moving image decoding device in accordance with the present invention are suitable for use as a moving image encoding device which divides a moving image into predetermined regions to encode the moving image on a per-region basis and as a moving image decoding device which decodes an encoded moving image on a per-predetermined-region basis.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Physics & Mathematics (AREA)
  • Discrete Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
US13/639,134 2010-04-09 2011-03-31 Moving image encoding device and moving image decoding device Abandoned US20130028326A1 (en)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
JP2010090534 2010-04-09
JP2010-090534 2010-04-09
PCT/JP2011/001953 WO2011125313A1 (ja) 2010-04-09 2011-03-31 動画像符号化装置および動画像復号装置

Related Parent Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2011/001953 A-371-Of-International WO2011125313A1 (ja) 2010-04-09 2011-03-31 動画像符号化装置および動画像復号装置

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US15/443,431 Continuation US9973753B2 (en) 2010-04-09 2017-02-27 Moving image encoding device and moving image decoding device based on adaptive switching among transformation block sizes

Publications (1)

Publication Number Publication Date
US20130028326A1 true US20130028326A1 (en) 2013-01-31

Family

ID=44762284

Family Applications (6)

Application Number Title Priority Date Filing Date
US13/639,134 Abandoned US20130028326A1 (en) 2010-04-09 2011-03-31 Moving image encoding device and moving image decoding device
US15/443,431 Active US9973753B2 (en) 2010-04-09 2017-02-27 Moving image encoding device and moving image decoding device based on adaptive switching among transformation block sizes
US15/920,105 Active US10412385B2 (en) 2010-04-09 2018-03-13 Moving image encoding device and moving image decoding device based on adaptive switching among transformation block sizes
US15/996,293 Active US10390011B2 (en) 2010-04-09 2018-06-01 Moving image encoding device and moving image decoding device based on adaptive switching among transformation block sizes
US16/444,922 Active US10554970B2 (en) 2010-04-09 2019-06-18 Moving image encoding device and moving image decoding device based on adaptive switching among transformation block sizes
US16/444,898 Active US10469839B2 (en) 2010-04-09 2019-06-18 Moving image encoding device and moving image decoding device based on adaptive switching among transformation block sizes

Family Applications After (5)

Application Number Title Priority Date Filing Date
US15/443,431 Active US9973753B2 (en) 2010-04-09 2017-02-27 Moving image encoding device and moving image decoding device based on adaptive switching among transformation block sizes
US15/920,105 Active US10412385B2 (en) 2010-04-09 2018-03-13 Moving image encoding device and moving image decoding device based on adaptive switching among transformation block sizes
US15/996,293 Active US10390011B2 (en) 2010-04-09 2018-06-01 Moving image encoding device and moving image decoding device based on adaptive switching among transformation block sizes
US16/444,922 Active US10554970B2 (en) 2010-04-09 2019-06-18 Moving image encoding device and moving image decoding device based on adaptive switching among transformation block sizes
US16/444,898 Active US10469839B2 (en) 2010-04-09 2019-06-18 Moving image encoding device and moving image decoding device based on adaptive switching among transformation block sizes

Country Status (15)

Country Link
US (6) US20130028326A1 (ja)
EP (2) EP2557792A4 (ja)
JP (8) JPWO2011125313A1 (ja)
KR (6) KR101540899B1 (ja)
CN (4) CN107046644B (ja)
BR (1) BR112012025206B1 (ja)
CA (1) CA2795425C (ja)
ES (1) ES2899780T3 (ja)
HK (1) HK1257212A1 (ja)
MX (3) MX353109B (ja)
PL (1) PL3101897T3 (ja)
RU (7) RU2573222C2 (ja)
SG (3) SG10201502226SA (ja)
TW (4) TWI520617B (ja)
WO (1) WO2011125313A1 (ja)

Cited By (23)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130094542A1 (en) * 2011-10-14 2013-04-18 Texas Instruments Incorporated Interconnect coding method and apparatus
US20140079122A1 (en) * 2011-06-30 2014-03-20 Sony Corporation Image processing apparatus and image processing method
EP2768220A4 (en) * 2011-11-04 2014-10-29 Huawei Tech Co Ltd METHOD AND DEVICE FOR ENCODING AND DECODING BASED ON TRANSFORMATION MODE
US9277233B1 (en) * 2012-04-15 2016-03-01 Samsung Electronics Co., Ltd. Parameter update method for entropy coding and decoding of conversion coefficient level, and entropy coding device and entropy decoding device of conversion coefficient level using same
US20160110882A1 (en) * 2013-06-25 2016-04-21 Chung-Ang University Industry-Academy Cooperation Foundation Apparatus and method for detecting multiple objects using adaptive block partitioning
US20160301941A1 (en) * 2013-09-09 2016-10-13 Nec Corporation Video-encoding device, video-encoding method, and program
US20160330468A1 (en) * 2014-02-03 2016-11-10 Mitsubishi Electric Corporation Image encoding device, image decoding device, encoded stream conversion device, image encoding method, and image decoding method
US9534469B2 (en) 2013-09-27 2017-01-03 Baker Hughes Incorporated Stacked tray ball dropper for subterranean fracking operations
US20170026663A1 (en) * 2015-07-24 2017-01-26 Qualcomm Incorporated Modification to block size for transform mode in display stream compression
KR20170076345A (ko) * 2015-12-24 2017-07-04 삼성전자주식회사 프레임 레이트 변환 장치 및 그 프레임 레이트 변환 방법
WO2017127168A1 (en) * 2016-01-22 2017-07-27 Intel Corporation Transform unit size determination for video coding
US20170324984A1 (en) * 2014-11-27 2017-11-09 Orange Segmentation method and method for signaling segmentation of a coding tree unit
US9959586B1 (en) * 2016-12-13 2018-05-01 GoAnimate, Inc. System, method, and computer program for encoding and decoding a unique signature in a video file as a set of watermarks
US10271056B2 (en) * 2011-10-24 2019-04-23 Sony Corporation Encoding apparatus, encoding method and program
US10356437B2 (en) 2013-07-22 2019-07-16 Renesas Electronics Corporation Moving image encoding apparatus including padding processor for adding padding data to moving image and operation method thereof
US10368279B2 (en) * 2016-01-25 2019-07-30 Kyocera Corporation Communication method, radio terminal, and base station
US20190320188A1 (en) * 2018-04-16 2019-10-17 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
US20190364271A1 (en) * 2016-09-12 2019-11-28 Nec Corporation Intra-prediction mode determination method, intra-prediction mode determination device, and storage medium for storing intra-prediction mode determination program
US10931946B2 (en) * 2011-01-12 2021-02-23 Mitsubishi Electric Corporation Image encoding device, image decoding device, image encoding method, and image decoding method for generating a prediction image
US20210104019A1 (en) * 2019-10-04 2021-04-08 Sharp Kabushiki Kaisha Video converting apparatus and method
US11039138B1 (en) * 2012-03-08 2021-06-15 Google Llc Adaptive coding of prediction modes using probability distributions
US11107253B2 (en) * 2016-10-12 2021-08-31 Kaonmedia Co., Ltd. Image processing method, and image decoding and encoding method using same
US11170481B2 (en) * 2018-08-14 2021-11-09 Etron Technology, Inc. Digital filter for filtering signals

Families Citing this family (58)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN103947206B (zh) 2011-10-14 2018-01-23 超威半导体公司 基于区域的图像压缩的方法和装置
KR20140092861A (ko) * 2011-10-31 2014-07-24 미쓰비시덴키 가부시키가이샤 동화상 부호화 장치, 동화상 복호 장치, 동화상 부호화 방법 및 동화상 복호 방법
CN103108177B (zh) * 2011-11-09 2016-11-23 华为技术有限公司 图像编码方法及图像编码装置
TWI658725B (zh) * 2011-12-28 2019-05-01 日商Ｊｖｃ建伍股份有限公司 動態影像解碼裝置、動態影像解碼方法及動態影像解碼程式
RU2616598C1 (ru) * 2012-01-19 2017-04-18 Мицубиси Электрик Корпорейшн Устройство декодирования изображений, устройство кодирования изображений, способ декодирования изображений и способ кодирования изображений
EP2842325A4 (en) * 2012-04-24 2015-10-14 Lyrical Labs Video Compression Technology Llc MACROBLOCK PARTITIONING AND MOTION ESTIMATION USING AN OBJECT ANALYSIS FOR VIDEO COMPRESSION
JP5719410B2 (ja) 2013-07-25 2015-05-20 日本電信電話株式会社 画像符号化方法、画像符号化装置及び画像符号化プログラム
KR102264840B1 (ko) * 2014-11-27 2021-06-15 삼성전자주식회사 비디오 프레임 인코딩 회로, 그것의 인코딩 방법 및 그것을 포함하는 비디오 데이터 송수신 장치
CN106331722B (zh) 2015-07-03 2019-04-26 华为技术有限公司 图像预测方法和相关设备
US10142635B2 (en) 2015-12-18 2018-11-27 Blackberry Limited Adaptive binarizer selection for image and video coding
US20170180757A1 (en) * 2015-12-18 2017-06-22 Blackberry Limited Binarizer selection for image and video coding
WO2018058622A1 (zh) * 2016-09-30 2018-04-05 华为技术有限公司 一种帧间预测的图像编解码方法和装置
WO2018092869A1 (ja) * 2016-11-21 2018-05-24 パナソニック インテレクチュアル プロパティ コーポレーション オブ アメリカ 符号化装置、復号装置、符号化方法及び復号方法
CA3057445A1 (en) * 2017-03-21 2018-09-27 Lg Electronics Inc. Transform method in image coding system and apparatus for same
CN116708780A (zh) 2017-08-11 2023-09-05 华为技术有限公司 视频图像编码和解码的方法、装置及设备
MX2020002143A (es) 2017-12-06 2020-07-20 Fujitsu Ltd Metodos y aparatos para codificar y decodificar informacion de modo y dispositivo electronico.
CN110035287B (zh) * 2018-01-12 2023-05-09 富士通株式会社 对统一转换单元模式进行分组标识的方法、装置和电子设备
WO2019160133A1 (ja) * 2018-02-19 2019-08-22 日本電信電話株式会社 情報管理装置、情報管理方法及び情報管理プログラム
KR102516233B1 (ko) 2018-04-02 2023-03-30 엘지전자 주식회사 움직임 벡터에 기반한 영상 코딩 방법 및 그 장치
WO2020156454A1 (en) * 2019-01-31 2020-08-06 Mediatek Inc. Method and apparatus of transform type assignment for intra sub-partition in video coding
EP3915265A4 (en) 2019-03-01 2022-06-22 Beijing Bytedance Network Technology Co., Ltd. DIRECTION-BASED PREDICTION FOR INTRA BLOCK COPY IN VIDEO CODING
US10939107B2 (en) 2019-03-01 2021-03-02 Sony Corporation Embedded codec circuitry for sub-block based allocation of refinement bits
WO2020182091A1 (en) 2019-03-08 2020-09-17 Beijing Bytedance Network Technology Co., Ltd. Reshaping model in video processing
JP2022068378A (ja) * 2019-03-08 2022-05-10 ソニーグループ株式会社 画像符号化装置、画像符号化方法、画像復号装置、および画像復号方法
SG11202110936PA (en) 2019-04-12 2021-11-29 Beijing Bytedance Network Technology Co Ltd Chroma coding mode determination based on matrix-based intra prediction
CN113678462B (zh) 2019-04-15 2023-01-10 北京字节跳动网络技术有限公司 自适应环路滤波器中的裁剪参数推导
CN113767623B (zh) 2019-04-16 2024-04-02 北京字节跳动网络技术有限公司 用于视频编解码的自适应环路滤波
CN113711610B (zh) 2019-04-23 2024-04-16 北京字节跳动网络技术有限公司 降低跨分量依赖性的方法
BR112021022868A2 (pt) 2019-05-22 2022-01-04 Beijing Bytedance Network Tech Co Ltd Método de processamento de vídeos, aparelho para processar dados de vídeo e meios de armazenamento e gravação não transitórios legíveis por computador
CN113924775B (zh) 2019-05-31 2023-11-14 北京字节跳动网络技术有限公司 基于矩阵的帧内预测中的限制的上采样
KR20220013938A (ko) 2019-05-31 2022-02-04 바이트댄스 아이엔씨 인트라 블록 복사 예측을 갖는 팔레트 모드
WO2020244610A1 (en) 2019-06-05 2020-12-10 Beijing Bytedance Network Technology Co., Ltd. Context determination for matrix-based intra prediction
WO2020259427A1 (en) 2019-06-22 2020-12-30 Beijing Bytedance Network Technology Co., Ltd. Syntax element for chroma residual scaling
KR20220023338A (ko) 2019-06-25 2022-03-02 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 모션 벡터 차이에 대한 제약
JP7460748B2 (ja) 2019-07-07 2024-04-02 北京字節跳動網絡技術有限公司 クロマ残差スケーリングのシグナリング
CN114175633B (zh) 2019-07-10 2023-12-29 北京字节跳动网络技术有限公司 用于视频编解码中的帧内块复制的样点标识
KR20220030995A (ko) 2019-07-14 2022-03-11 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 비디오 코딩에서 변환 블록 크기 제한
JP7481430B2 (ja) 2019-08-13 2024-05-10 北京字節跳動網絡技術有限公司 サブブロックに基づくインター予測における動き精度
CN114342400A (zh) 2019-09-02 2022-04-12 北京字节跳动网络技术有限公司 基于色彩格式的编解码模式确定
MX2022002617A (es) 2019-09-12 2022-03-25 Bytedance Inc Uso de predictor de paleta en codificacion de video.
WO2021051047A1 (en) 2019-09-14 2021-03-18 Bytedance Inc. Chroma quantization parameter in video coding
WO2021057996A1 (en) 2019-09-28 2021-04-01 Beijing Bytedance Network Technology Co., Ltd. Geometric partitioning mode in video coding
KR20220078600A (ko) 2019-10-18 2022-06-10 베이징 바이트댄스 네트워크 테크놀로지 컴퍼니, 리미티드 서브픽처들의 파라미터 세트 시그널링에서의 신택스 제약들
JP7395727B2 (ja) 2019-10-23 2023-12-11 北京字節跳動網絡技術有限公司 ビデオ・データを処理する方法、装置及び記憶方法
EP4042689A4 (en) 2019-10-28 2023-06-07 Beijing Bytedance Network Technology Co., Ltd. SIGNALING AND SYNTAX ANALYSIS BASED ON A COLOR COMPONENT
CN114788279A (zh) 2019-12-09 2022-07-22 字节跳动有限公司 视频编解码中使用的量化组
MX2022006734A (es) 2019-12-11 2022-06-09 Beijing Bytedance Network Tech Co Ltd Relleno de muestra para filtrado de circuito adaptivo de componente cruzado.
JP2023513518A (ja) 2020-02-05 2023-03-31 北京字節跳動網絡技術有限公司 ローカルデュアルツリー向けのパレットモード
EP4088453A4 (en) 2020-02-14 2023-05-10 Beijing Bytedance Network Technology Co., Ltd. COLOCATED IMAGE INDICATION IN VIDEO BITSTREAMS
KR20220155293A (ko) 2020-03-17 2022-11-22 바이트댄스 아이엔씨 비디오 코딩에서 픽처 출력 플래그 표시
CN115362674A (zh) 2020-03-18 2022-11-18 抖音视界有限公司 帧内块复制缓冲区和调色板预测值更新
BR112022018206A2 (pt) 2020-03-19 2022-10-25 Bytedance Inc Método e aparelho de processamento de vídeo, método de armazenamento de um fluxo contínuo de bits de um vídeo, e, mídia legível por computador
BR112022018884A2 (pt) 2020-03-20 2022-11-08 Bytedance Inc Método de processamento de vídeo, aparelho para processamento de dados de vídeo, meios de armazenamento e de gravação não transitórios legíveis por computador
JP7464742B2 (ja) 2020-03-21 2024-04-09 北京字節跳動網絡技術有限公司 参照ピクチャリサンプリング
JP2023531223A (ja) 2020-06-30 2023-07-21 北京字節跳動網絡技術有限公司 適応ループフィルタリングのための境界位置
CN111968151B (zh) * 2020-07-03 2022-04-05 北京博雅慧视智能技术研究院有限公司 一种运动估计精细搜索方法及装置
US11770498B2 (en) 2020-09-29 2023-09-26 Lemon Inc. Supplemental enhancement information for multi-layer video streams
US11750778B2 (en) 2021-09-30 2023-09-05 Coretronic Corporation Method for adjusting pixel values of blending image and projection system thereof

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040126028A1 (en) * 2002-10-08 2004-07-01 Ntt Docomo, Inc. Image encoding method, image decoding method, image encoding apparatus, image decoding apparatus, image encoding program, and image decoding program
JP2009005413A (ja) * 2008-09-30 2009-01-08 Toshiba Corp 画像符号化装置

Family Cites Families (49)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5107345A (en) 1990-02-27 1992-04-21 Qualcomm Incorporated Adaptive block size image compression method and system
FR2756399B1 (fr) * 1996-11-28 1999-06-25 Thomson Multimedia Sa Procede et dispositif de compression video pour images de synthese
US6600836B1 (en) 2000-01-28 2003-07-29 Qualcomm, Incorporated Quality based image compression
DE10022331A1 (de) * 2000-05-10 2001-11-15 Bosch Gmbh Robert Verfahren zur Transformationscodierung von Bewegtbildsequenzen
US20030053637A1 (en) 2001-09-14 2003-03-20 Michael Rodemer Audio distributor
US7336711B2 (en) 2001-11-16 2008-02-26 Ntt Docomo, Inc. Image encoding method, image decoding method, image encoder, image decode, program, computer data signal, and image transmission system
JP2003189313A (ja) * 2001-12-20 2003-07-04 Matsushita Electric Ind Co Ltd 画面間予測符号化方法および画面間予測復号化方法
JP2003348598A (ja) * 2002-04-12 2003-12-05 Seiko Epson Corp メモリ効率のいい圧縮領域ビデオ処理のための且つ因数分解及び整数近似法を用いる高速逆動き補償のための方法並びに装置
JP4090862B2 (ja) * 2002-04-26 2008-05-28 松下電器産業株式会社 可変長符号化方法および可変長復号化方法
JP2003318740A (ja) * 2002-04-23 2003-11-07 Matsushita Electric Ind Co Ltd 可変長符号化方法および可変長復号化方法
JP2003324731A (ja) * 2002-04-26 2003-11-14 Sony Corp 符号化装置、復号装置、画像処理装置、それらの方法およびプログラム
JP2003319400A (ja) * 2002-04-26 2003-11-07 Sony Corp 符号化装置、復号装置、画像処理装置、それらの方法およびプログラム
US6975773B1 (en) * 2002-07-30 2005-12-13 Qualcomm, Incorporated Parameter selection in data compression and decompression
WO2004030369A1 (en) * 2002-09-27 2004-04-08 Videosoft, Inc. Real-time video coding/decoding
JP4238553B2 (ja) 2002-10-08 2009-03-18 日本電気株式会社 携帯電話装置および表示装置の画像表示方法
JP2004135252A (ja) * 2002-10-09 2004-04-30 Sony Corp 符号化処理方法、符号化装置及び復号化装置
JP4702059B2 (ja) * 2003-12-22 2011-06-15 日本電気株式会社 動画像を符号化する方法及び装置
US20050249278A1 (en) * 2004-04-28 2005-11-10 Matsushita Electric Industrial Co., Ltd. Moving image coding method, moving image decoding method, moving image coding device, moving image decoding device, moving image coding program and program product of the same
US7894530B2 (en) * 2004-05-07 2011-02-22 Broadcom Corporation Method and system for dynamic selection of transform size in a video decoder based on signal content
CN100401780C (zh) * 2004-05-07 2008-07-09 美国博通公司 在视频解码器中动态选择变换尺寸的方法和系统
JP4421940B2 (ja) * 2004-05-13 2010-02-24 株式会社エヌ・ティ・ティ・ドコモ 動画像符号化装置および方法、並びに動画像復号化装置および方法
KR100813958B1 (ko) * 2004-06-07 2008-03-14 세종대학교산학협력단 동영상의 무손실 인코딩 및 디코딩 방법, 그 장치
WO2006028088A1 (ja) * 2004-09-08 2006-03-16 Matsushita Electric Industrial Co., Ltd. 動画像符号化方法および動画像復号化方法
JP4542107B2 (ja) * 2005-07-22 2010-09-08 三菱電機株式会社 画像復号装置及び画像復号方法
EP2720466A1 (en) 2005-09-26 2014-04-16 Mitsubishi Electric Corporation Moving image coding method
JP2007243427A (ja) * 2006-03-07 2007-09-20 Nippon Hoso Kyokai <Nhk> 符号化装置及び復号化装置
US20070286277A1 (en) * 2006-06-13 2007-12-13 Chen Xuemin Sherman Method and system for video compression using an iterative encoding algorithm
JP2008022383A (ja) * 2006-07-13 2008-01-31 Matsushita Electric Ind Co Ltd 画像符号化装置
AU2006346583B2 (en) * 2006-07-28 2011-04-28 Kabushiki Kaisha Toshiba Image encoding and decoding method and apparatus
US8565314B2 (en) * 2006-10-12 2013-10-22 Qualcomm Incorporated Variable length coding table selection based on block type statistics for refinement coefficient coding
US20080170793A1 (en) * 2007-01-12 2008-07-17 Mitsubishi Electric Corporation Image encoding device and image encoding method
JP4635016B2 (ja) * 2007-02-16 2011-02-16 株式会社東芝 情報処理装置およびインター予測モード判定方法
JP2010135864A (ja) * 2007-03-29 2010-06-17 Toshiba Corp 画像符号化方法及び装置並びに画像復号化方法及び装置
JP4364919B2 (ja) * 2007-04-20 2009-11-18 三菱電機株式会社 動画像復号化装置
US8422803B2 (en) * 2007-06-28 2013-04-16 Mitsubishi Electric Corporation Image encoding device, image decoding device, image encoding method and image decoding method
JP4325708B2 (ja) * 2007-07-05 2009-09-02 ソニー株式会社 データ処理装置、データ処理方法およびデータ処理プログラム、符号化装置、符号化方法および符号化プログラム、ならびに、復号装置、復号方法および復号プログラム
JP4821723B2 (ja) * 2007-07-13 2011-11-24 富士通株式会社 動画像符号化装置及びプログラム
JP4922101B2 (ja) * 2007-08-21 2012-04-25 株式会社東芝 情報処理装置およびインター予測モード判定方法
JP2009055236A (ja) * 2007-08-24 2009-03-12 Canon Inc 映像符号化装置及び方法
US20090274213A1 (en) 2008-04-30 2009-11-05 Omnivision Technologies, Inc. Apparatus and method for computationally efficient intra prediction in a video coder
US8908763B2 (en) * 2008-06-25 2014-12-09 Qualcomm Incorporated Fragmented reference in temporal compression for video coding
KR101549823B1 (ko) * 2008-09-02 2015-09-04 삼성전자주식회사 적응적 이진화를 이용한 영상 부호화, 복호화 방법 및 장치
US8619856B2 (en) * 2008-10-03 2013-12-31 Qualcomm Incorporated Video coding with large macroblocks
KR101247923B1 (ko) * 2008-10-03 2013-03-26 퀄컴 인코포레이티드 4×4 및 8×8 보다 큰 변환을 이용한 비디오 코딩
US8879637B2 (en) * 2008-10-06 2014-11-04 Lg Electronics Inc. Method and an apparatus for processing a video signal by which coding efficiency of a video signal can be raised by using a mixed prediction mode in predicting different macroblock sizes
KR101690291B1 (ko) * 2009-02-05 2016-12-27 톰슨 라이센싱 적응형 모드 비디오 인코딩 및 디코딩 방법 및 장치
CN102308518B (zh) 2009-02-09 2014-12-10 罗伯特·博世有限公司 使用计算机网络的方法
US20120040366A1 (en) 2009-02-10 2012-02-16 Tissue Genetics, Inc. Compositions, methods and uses for disease diagnosis
EA201690320A3 (ru) * 2009-04-08 2016-10-31 Шарп Кабусики Кайся Устройство кодирования видеокадров и устройство декодирования видеокадров

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20040126028A1 (en) * 2002-10-08 2004-07-01 Ntt Docomo, Inc. Image encoding method, image decoding method, image encoding apparatus, image decoding apparatus, image encoding program, and image decoding program
JP2009005413A (ja) * 2008-09-30 2009-01-08 Toshiba Corp 画像符号化装置

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
Wien, "Variable Block-Size Transforms for H.264/AVC," IEEE Transactions on Circuits and Systems for Video Technology, Vol. 13, No. 7, July 2003, pp. 604-613. *

Cited By (37)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10931946B2 (en) * 2011-01-12 2021-02-23 Mitsubishi Electric Corporation Image encoding device, image decoding device, image encoding method, and image decoding method for generating a prediction image
US20140079122A1 (en) * 2011-06-30 2014-03-20 Sony Corporation Image processing apparatus and image processing method
US10104372B2 (en) * 2011-06-30 2018-10-16 Sony Corporation Image processing apparatus and image processing method
US20130094542A1 (en) * 2011-10-14 2013-04-18 Texas Instruments Incorporated Interconnect coding method and apparatus
US8571092B2 (en) * 2011-10-14 2013-10-29 Texas Instruments Incorporated Interconnect coding method and apparatus
US10271056B2 (en) * 2011-10-24 2019-04-23 Sony Corporation Encoding apparatus, encoding method and program
EP2768220A4 (en) * 2011-11-04 2014-10-29 Huawei Tech Co Ltd METHOD AND DEVICE FOR ENCODING AND DECODING BASED ON TRANSFORMATION MODE
US9462274B2 (en) 2011-11-04 2016-10-04 Huawei Technologies Co., Ltd. Transformation mode encoding and decoding method and apparatus
US11627321B2 (en) 2012-03-08 2023-04-11 Google Llc Adaptive coding of prediction modes using probability distributions
US11039138B1 (en) * 2012-03-08 2021-06-15 Google Llc Adaptive coding of prediction modes using probability distributions
US9277233B1 (en) * 2012-04-15 2016-03-01 Samsung Electronics Co., Ltd. Parameter update method for entropy coding and decoding of conversion coefficient level, and entropy coding device and entropy decoding device of conversion coefficient level using same
US20160110882A1 (en) * 2013-06-25 2016-04-21 Chung-Ang University Industry-Academy Cooperation Foundation Apparatus and method for detecting multiple objects using adaptive block partitioning
US9836851B2 (en) * 2013-06-25 2017-12-05 Chung-Ang University Industry-Academy Cooperation Foundation Apparatus and method for detecting multiple objects using adaptive block partitioning
US10356437B2 (en) 2013-07-22 2019-07-16 Renesas Electronics Corporation Moving image encoding apparatus including padding processor for adding padding data to moving image and operation method thereof
US20160301941A1 (en) * 2013-09-09 2016-10-13 Nec Corporation Video-encoding device, video-encoding method, and program
US9534469B2 (en) 2013-09-27 2017-01-03 Baker Hughes Incorporated Stacked tray ball dropper for subterranean fracking operations
US10075725B2 (en) * 2014-02-03 2018-09-11 Mitsubishi Electric Corporation Device and method for image encoding and decoding
US20160330468A1 (en) * 2014-02-03 2016-11-10 Mitsubishi Electric Corporation Image encoding device, image decoding device, encoded stream conversion device, image encoding method, and image decoding method
US10924769B2 (en) * 2014-11-27 2021-02-16 Orange Segmentation method and method for signaling segmentation of a coding tree unit
US20170324984A1 (en) * 2014-11-27 2017-11-09 Orange Segmentation method and method for signaling segmentation of a coding tree unit
US11825129B2 (en) * 2014-11-27 2023-11-21 Orange Segmentation method and method for signaling segmentation of a coding tree unit
US10123045B2 (en) * 2015-07-24 2018-11-06 Qualcomm Incorporated Modification to block size for transform mode in display stream compression
US20170026663A1 (en) * 2015-07-24 2017-01-26 Qualcomm Incorporated Modification to block size for transform mode in display stream compression
US9749579B2 (en) * 2015-12-24 2017-08-29 Samsung Electronics Co., Ltd. Apparatus and method for frame rate conversion
KR102411911B1 (ko) * 2015-12-24 2022-06-22 삼성전자주식회사 프레임 레이트 변환 장치 및 그 프레임 레이트 변환 방법
KR20170076345A (ko) * 2015-12-24 2017-07-04 삼성전자주식회사 프레임 레이트 변환 장치 및 그 프레임 레이트 변환 방법
WO2017127168A1 (en) * 2016-01-22 2017-07-27 Intel Corporation Transform unit size determination for video coding
US10560702B2 (en) 2016-01-22 2020-02-11 Intel Corporation Transform unit size determination for video coding
US10368279B2 (en) * 2016-01-25 2019-07-30 Kyocera Corporation Communication method, radio terminal, and base station
US10721468B2 (en) * 2016-09-12 2020-07-21 Nec Corporation Intra-prediction mode determination method, intra-prediction mode determination device, and storage medium for storing intra-prediction mode determination program
US20190364271A1 (en) * 2016-09-12 2019-11-28 Nec Corporation Intra-prediction mode determination method, intra-prediction mode determination device, and storage medium for storing intra-prediction mode determination program
US11107253B2 (en) * 2016-10-12 2021-08-31 Kaonmedia Co., Ltd. Image processing method, and image decoding and encoding method using same
US9959586B1 (en) * 2016-12-13 2018-05-01 GoAnimate, Inc. System, method, and computer program for encoding and decoding a unique signature in a video file as a set of watermarks
US10986354B2 (en) * 2018-04-16 2021-04-20 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
US20190320188A1 (en) * 2018-04-16 2019-10-17 Panasonic Intellectual Property Corporation Of America Encoder, decoder, encoding method, and decoding method
US11170481B2 (en) * 2018-08-14 2021-11-09 Etron Technology, Inc. Digital filter for filtering signals
US20210104019A1 (en) * 2019-10-04 2021-04-08 Sharp Kabushiki Kaisha Video converting apparatus and method

Also Published As

Publication number Publication date
CN106998473A (zh) 2017-08-01
RU2663374C1 (ru) 2018-08-03
US20180255297A1 (en) 2018-09-06
RU2573222C2 (ru) 2016-01-20
CN102934438A (zh) 2013-02-13
JP2020017971A (ja) 2020-01-30
JP7126309B2 (ja) 2022-08-26
US10469839B2 (en) 2019-11-05
SG10201502226SA (en) 2015-05-28
TWI688267B (zh) 2020-03-11
KR20130028092A (ko) 2013-03-18
RU2699049C1 (ru) 2019-09-03
CN107046644B (zh) 2020-03-24
JP6605063B2 (ja) 2019-11-13
US20180139442A9 (en) 2018-05-17
MX353107B (es) 2017-12-19
CN106998473B (zh) 2020-03-24
CN108462874A (zh) 2018-08-28
KR20140010192A (ko) 2014-01-23
JP2015029348A (ja) 2015-02-12
US9973753B2 (en) 2018-05-15
RU2014116111A (ru) 2015-10-27
TW202025764A (zh) 2020-07-01
PL3101897T3 (pl) 2022-02-21
BR112012025206A2 (pt) 2017-06-27
JP7126310B2 (ja) 2022-08-26
EP2557792A4 (en) 2014-08-13
KR101540899B1 (ko) 2015-07-30
US20180278931A1 (en) 2018-09-27
TW201613355A (en) 2016-04-01
KR101817481B1 (ko) 2018-02-21
JP2020017970A (ja) 2020-01-30
RU2627101C2 (ru) 2017-08-03
SG184528A1 (en) 2012-11-29
RU2012147595A (ru) 2014-06-10
JPWO2011125313A1 (ja) 2013-07-08
EP3101897B1 (en) 2021-10-20
KR20150013776A (ko) 2015-02-05
SG10202001623RA (en) 2020-04-29
US20190306496A1 (en) 2019-10-03
RU2523071C1 (ru) 2014-07-20
RU2716032C1 (ru) 2020-03-05
US10412385B2 (en) 2019-09-10
CA2795425A1 (en) 2011-10-13
TW201739254A (zh) 2017-11-01
KR101389163B1 (ko) 2014-04-28
JP2020017973A (ja) 2020-01-30
EP2557792A1 (en) 2013-02-13
JP2016129405A (ja) 2016-07-14
EP3101897A1 (en) 2016-12-07
US20170171543A1 (en) 2017-06-15
WO2011125313A1 (ja) 2011-10-13
MX353109B (es) 2017-12-19
US10390011B2 (en) 2019-08-20
TWI520617B (zh) 2016-02-01
US10554970B2 (en) 2020-02-04
KR20140049092A (ko) 2014-04-24
US20190306497A1 (en) 2019-10-03
TW201143458A (en) 2011-12-01
ES2899780T3 (es) 2022-03-14
KR20170013397A (ko) 2017-02-06
BR112012025206B1 (pt) 2022-04-12
RU2734871C1 (ru) 2020-10-23
HK1257212A1 (zh) 2019-10-18
TWI765223B (zh) 2022-05-21
CN108462874B (zh) 2022-06-07
JP2020017972A (ja) 2020-01-30
RU2015151397A (ru) 2017-06-06
JP7129958B2 (ja) 2022-09-02
KR101500914B1 (ko) 2015-03-10
MX2012011695A (es) 2012-11-16
CA2795425C (en) 2018-01-09
JP7126308B2 (ja) 2022-08-26
CN107046644A (zh) 2017-08-15
KR20140110074A (ko) 2014-09-16
TWI601415B (zh) 2017-10-01
JP2018137773A (ja) 2018-08-30

Similar Documents

Publication Publication Date Title
US10469839B2 (en) Moving image encoding device and moving image decoding device based on adaptive switching among transformation block sizes
KR20110040893A (ko) 화상 부호화 장치, 화상 복호 장치, 화상 부호화 방법 및 화상 복호 방법
JP2011223319A (ja) 動画像符号化装置および動画像復号装置

Legal Events

Date Code Title Description
AS Assignment

Owner name: MITSUBISHI ELECTRIC CORPORATION, JAPAN

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:MORIYA, YOSHIMI;SEKIGUCHI, SHUNICHI;SUGIMOTO, KAZUO;AND OTHERS;REEL/FRAME:029153/0662

Effective date: 20121004

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION