USRE47004E1 - Moving image coding device and method - Google Patents

Moving image coding device and method Download PDF

Info

Publication number
USRE47004E1
USRE47004E1 US14/811,157 US200914811157A USRE47004E US RE47004 E1 USRE47004 E1 US RE47004E1 US 200914811157 A US200914811157 A US 200914811157A US RE47004 E USRE47004 E US RE47004E
Authority
US
United States
Prior art keywords
unit
image
segment
coding
pixel
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active, expires
Application number
US14/811,157
Inventor
Tae Meon Bae
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
SK Planet Co Ltd
Original Assignee
SK Planet Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by SK Planet Co Ltd filed Critical SK Planet Co Ltd
Priority to US14/811,157 priority Critical patent/USRE47004E1/en
Application granted granted Critical
Publication of USRE47004E1 publication Critical patent/USRE47004E1/en
Active legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/176—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a block, e.g. a macroblock
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/523—Motion estimation or motion compensation with sub-pixel accuracy
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/43—Hardware specially adapted for motion estimation or compensation
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/42—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation
    • H04N19/436—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals characterised by implementation details or hardware specially adapted for video compression or decompression, e.g. dedicated software implementation using parallelised computational arrangements
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation

Definitions

  • digital image data is compressed by an efficient compression method instead of being used as it is because the amount of data is greatly increased owing to its inherent characteristic and in a process of digitalizing an analog signal.
  • Digital image data is compressed using, chiefly, three kinds of methods.
  • a method of reducing temporal redundancy, a method of reducing spatial redundancy, and a method of reducing generated codes based on a statistical characteristic of the generated codes are chiefly used.
  • a representative method of reducing temporal redundancy is a motion estimation and compensation method, which is being used in most of moving image compression standards, such as MPEG or H.263.
  • FIG. 1 is a block diagram showing a conventional H.264 moving image coding apparatus.
  • the H.264 moving image coding apparatus includes a motion estimation unit ME 100 , a motion compensation unit MC 110 , an intra-mode estimation unit 120 , a transform/quantization unit 130 , an inverse transform/inverse quantization unit 140 , an entropy coding unit 150 , a deblocking filter 160 , and a reference image storage unit 170 .
  • the motion estimation unit 100 performs a function of finding a motion vector from several sheets of reference images and a macroblock mode determination function of finding an optimal macroblock type with the least bit rate and error.
  • the motion estimation unit 100 estimates an integer pel motion in units of macroblocks and then perform half pel and quarter pel motion estimation.
  • the motion compensation unit 110 performs a function of acquiring a compensation image from the reference images based on the motion vector and the macroblock mode type found by the motion estimation unit 100 .
  • the motion compensation unit 110 is limited to a structure for acquiring a difference between two images, and a process of acquiring subsequent compensation images is continued by the transform/quantization unit 130 .
  • the intra-mode estimation unit 120 performs a function of selecting an optimal intra-estimation mode by performing estimation from adjacent and surrounding blocks when coding a macroblock in an intra mode and performs a function of acquiring a compensation image, subjected to intra-mode estimation, from previously-coded and adjacent blocks using the selected intra-estimation mode.
  • the transform/quantization unit 130 performs a discrete cosine transform and performs a function of quantizing the transformed coefficients.
  • the inverse transform/inverse quantization unit 140 performs an opposite function to the transform/quantization unit 130 .
  • the entropy coding unit 150 performs entropy coding using bit allocation according to a probability that quantized DCT coefficients will occur.
  • the deblocking filter 160 performs a function of improving a restoration image, acquired through the inverse transform/inverse quantization unit 140 , to an image of a better picture quality.
  • the improved image is stored in the reference image storage unit 170 , and it becomes a reference image for subsequently inputted images.
  • FIG. 2 is a flowchart illustrating a method of the conventional H.264 moving image coding apparatus coding a moving image.
  • the H.264 moving image coding apparatus performs coding on the first macroblock to n th macroblock of an input image frame in units of macroblocks by estimating integer pel, half pel, and quarter pel motions at step S 200 .
  • the H.264 moving image coding apparatus decodes the coded image frame at step S 204 .
  • the H.264 moving image coding apparatus After estimating the integer pel motion, the H.264 moving image coding apparatus estimates the half/quarter pel motions with respect to the integer pel motion-estimation image. Next, the H.264 moving image coding apparatus determines a mode and, and transforms, quantizes, and codes the half/quarter pel motion-estimated images using the determined mode.
  • the H.264 moving image coding apparatus generates a half/quarter pel image by interpolating the decoded image at step S 206 .
  • the generated half/quarter pel image is used to estimate the motion of a next image.
  • all the functions other than the function of generating the half/quarter pel image through interpolation are sequentially performed in order to code one macroblock.
  • the coding of a next macroblock is started after the coding of a previous macroblock is finished.
  • an image decoded from the coded image is obtained.
  • the half/quarter pel image is generated by interpolating the decoded image and is prepared to be used to estimate the motion of a next image.
  • the above-described H.264 moving image coding apparatus consumes a lot of time in order to perform coding because of a heavy computational load.
  • some processes such as motion estimation, the generation of a quarter pel image, and deblocking, require a heavy computational load, thus causing to delay the coding time.
  • a block-based coding apparatus such as H.264, divides an image frame into macroblocks and performs coding in units of macroblocks.
  • the block-based coding apparatus is configured to code a next macroblock after coding a previous macroblock.
  • This phenomenon is greatly influenced in terms of performance in the case of a coder having, in particular, a pipeline structure.
  • An object of the present invention is to provide a moving image coding apparatus and method, which is capable of performing parallel coding on an image frame in units of segments and in units of macroblocks at the same time, thereby enabling high-speed coding.
  • Another object of the present invention is to provide a moving image coding apparatus and method, which can reduce the delay of the coding speed due to a difference in the processing speed between modules and enables high-speed encoding by applying different coding units to modules and using parallel processing and pipelining schemes.
  • a system simultaneously utilizing a Graphic Processing Unit (GPU) and a CPU may have improved performance because the GPU and the CPU are executed in parallel.
  • the units of a task, processed by the CPU and the GPU, and the processing speeds of the CPU and the GPU are quite different because the GPU includes several tens to several hundreds of processing units and the CPU has one processing unit with very high performance.
  • An object of the present invention is to make parallel two devices having different units of a task and different processing speeds, such as a GPU and a CPU, by taking the above problem in view of moving image coding so that one device (for example, GPU) of the two devices processes the input of a segment unit (i.e., a great unit) and the other device (for example, CPU) of the two devices processes the input of a macroblock unit. Furthermore, an object of the present invention is to enable high-speed coding by dividing an image coding process into a process of performing processing in units of segments at once and a process of performing processing in units of macroblocks in order to be applied to the parallelized structure and by suggesting a coder structure suitable for the processes.
  • a moving image coding apparatus comprising a segment-unit-coding module for receiving a segment-unit image, estimating an integer pel motion and, at the same time, generating an image, coded by a macroblock-unit-coding module, as a 1/n pixel image; and a macroblock-unit-coding module for receiving then image, having the estimated integer pel motion, from the segment-unit-coding module in units of macroblocks and coding each of the macroblocks through 1/n pixel motion estimation for each macroblock.
  • the segment-unit-coding module estimates an integer pixel motion for a next segment in advance, before the macroblock-unit-coding module codes the image of the macroblock units having the estimated integer pixel motion.
  • the segment-unit-coding module codes the image of the macroblock units having the estimated integer pixel motion
  • the segment-unit-coding module estimates an integer pixel motion for a next segment image and generates a 1/n pixel image for a previous segment image independently and at the same time.
  • the segment-unit-coding module comprise an integer pel motion estimation unit for estimating a motion vector of an integer pixel unit for a current image frame of segment units; a motion estimation information storage unit for storing information estimated by the integer pel motion estimation unit; and a 1/n pixel image generation unit for generating an image, coded by the macroblock-unit-coding module, as the 1/n pixel image.
  • the 1/n pixel image generation unit generates the 1/n pixel image by enlarging a reference image frame n times in order to estimate a motion for the image, coded by the macroblock-unit-coding module, at an interval of a 1/n pixel.
  • the macroblock-unit-coding module comprises a 1/n pixel motion estimation unit for estimating a 1/n pixel motion for the current macroblock image using the integer pel motion estimation information, stored in the motion estimation information storage unit, and the 1/n pixel image generated by the 1/n pixel image generation unit; a transform/quantization unit for transforming and quantizing the macroblock image estimated by the 1/n pixel motion estimation unit; an entropy coding unit for coding the image transformed and quantized by the transform/quantization unit; and an inverse transform/inverse quantization unit for inverse-transforming and inverse-quantizing the image transformed and quantized by the transform/quantization unit.
  • the macroblock-unit-coding module further comprises an intra-mode estimation unit for selecting an optimal intra-estimation mode by performing estimation from adjacent and surrounding blocks when coding the macroblock image in an intra mode and for acquiring a compensation image, subjected to intra-mode estimation, from previously coded and adjacent macroblocks using the selected intra-estimation mode.
  • an intra-mode estimation unit for selecting an optimal intra-estimation mode by performing estimation from adjacent and surrounding blocks when coding the macroblock image in an intra mode and for acquiring a compensation image, subjected to intra-mode estimation, from previously coded and adjacent macroblocks using the selected intra-estimation mode.
  • segment-unit-coding module and the macroblock-unit-coding module are performed in parallel.
  • a method of a moving image coding apparatus coding an image frame comprising the steps of (a) estimating an integer pel motion for segments of the image frame using a segment-unit-coding module; (b) coding the segments of the image frame, having the estimated integer pixel motion, in units of macroblocks through 1/n pixel motion estimation for each of the macroblocks within the segment using a macroblock-unit-coding module; and (c) generating a 1/n pixel image for the segments of the image frame, coded in units of the macroblocks by the segment-unit-coding module.
  • steps (a) to (c) are performed on the segments of an image frame, sequentially inputted, in series.
  • steps (a), (b), and (c) are performed on the different segments of the image frame in parallel.
  • the method further comprises the step of dividing the image frame into an N number of segments before the step (a).
  • the step (b) comprises steps of estimating a motion vector of a 1/n pixel unit using the 1/n pixel image generated through the 1/n pixel motion estimation for each of the macroblocks within the segment having the integer pixel motion estimated by the segment-unit-coding module; quantizing each of the macroblocks having a motion estimated in unit of the 1/n pixel, having the motion estimated; and performing variable length coding on the quantized macroblocks.
  • generating a 1/n pixel image for a previous segment of the image frame in the step (c) comprises generating an image obtained by enlarging a reference image frame of the segments within the image frame, generated by the macroblock-unit-coding module, n times.
  • a method of a moving image coding apparatus coding an image frame comprising the steps of (a) estimating an integer pel motion for the first segment of a first input image frame; (b) coding each of macroblocks within the first segment, having the estimated integer pixel motion, through 1/n pixel motion estimation and simultaneously estimating an integer pixel motion for the second segment of the image frame; and (c) coding each of macroblocks within the second segment, having the estimated integer pixel motion, through 1/n pixel motion estimation, estimates an integer pixel motion for the third segment of the image frame, and generates a 1/n pixel image for the coded first segment simultaneously.
  • generating the 1/n pixel image for the coded first segment in the step (a) refers to generating an image obtained by enlarging a reference image frame of the first segment, coded in units of the macroblocks, n times.
  • a recording medium on which a program for coding an image frame is recorded, wherein the program executes the steps of (a) a segment-unit-coding module estimating an integer pel motion for the third segment of a current image frame inputted in units of segments and, at the same time, generating a 1/n pixel image for the first segment of the current image frame and (b) a macroblock-unit-coding module coding each of macroblocks within the second segment of the current frame, having an integer pixel motion estimated by the segment-unit-coding module, through 1/n pixel motion estimation.
  • a recording medium on which a program for coding an image frame is recorded, wherein the program executes the steps of (a) estimating an integer pel motion and a 1/n pixel motion for the third segment of a current image frame inputted in units of segments and, at the same time, generating a 1/n pixel image for the coded first segment of the current image frame and (b) coding the second segment of the current image frame, having an integer pixel motion and a 1/n motion estimated, in units of macroblocks.
  • a moving image coding apparatus and method which enables high-speed coding using parallel coding in encoding an image frame.
  • encoding can be performed at high speed by applying different coding units to modules and reducing the delay of the coding speed due to a difference in the processing speed between the modules.
  • an image is processed in parallel in units of segments and in units of macroblocks at the same time, and thus the time taken for coding is reduced. Accordingly, moving image coding apparatus and method capable of improving the coding speed can be provided.
  • FIG. 1 is a block diagram showing a conventional H.264 moving image coding apparatus
  • FIG. 2 is a flowchart illustrating a method of the conventional H.264 moving image coding apparatus coding a moving image
  • FIG. 3 is a block diagram schematically showing the construction of a moving image coding apparatus according to the present invention.
  • FIG. 4 is a flowchart illustrating a method of coding an image frame according to the present invention.
  • FIG. 5 is a flowchart illustrating a coding method in the case where an image frame is divided into three segments according to the present invention.
  • FIG. 3 is a block diagram schematically showing the construction of a moving image coding apparatus according to the present invention.
  • the moving image coding apparatus includes a segment-unit-coding module 300 for dividing an input image frame in units of segments and estimating an integer pixel (integer pel) motion and a macroblock-unit-coding module 350 for coding each of macroblocks within the segment having its integer pel motion estimated by the segment-unit-coding module 300 .
  • the segment-unit-coding module 300 functions to receive the image frame in units of segments and to estimate the integer pel motion and, at the same time, to generate a 1/n pixel image for the macroblocks coded by the macroblock-unit-coding module 350 .
  • the segment refers to an image frame vertically divided into a predetermined number.
  • the 1/n pixel image refers to an image obtained by enlarging a reference image frame, generated by the macroblock-unit-coding module 350 , two or four times in order to estimate motion information at an interval of 1 ⁇ 2 or 1 ⁇ 4 pixels.
  • n is an integer, and n may become a multiple of 2, such as 2 or 4 in the case of H.264.
  • the segment-unit-coding module 300 includes an integer pel motion estimation unit 302 , a motion estimation information storage unit 304 , and a 1/n pixel image generation unit 306 .
  • the integer pel motion estimation unit 302 divides a current input image frame into a predetermined number of segments and estimates motion vectors of the current image frame, having the units of segments, in unit of an integer pel (integer pixel).
  • the segments should be understood as a concept to temporally divide the segments in order to process an image rather than a concept to divide temporally consecutive image data. That is, even in the case of image data constituting one scene, in the case where the image data is divided into specific parts and the parts are processed with precedence relationships temporally, each of the parts may be understood as a segment.
  • the integer pel motion estimation unit 302 calculates an integer pel cost value for a segment, having a predetermined size, of a reference frame previous to or after a segment divided with a specific size and performs integer pixel search.
  • the cost value may include the Sum of Absolute Difference (SAD) value of an absolute difference.
  • the motion information is vector information in the x, y form and is information indicating how much has a block within an image frame been moved in 2-D from a previous reference image frame. Accordingly, the integer pel motion estimation unit estimates motion information as pixel interval resolution.
  • the motion estimation information storage unit 304 stores the motion estimation information estimated by the integer pel motion estimation unit 302 .
  • the 1/n pixel image generation unit 306 generates the 1/n pixel image for the macroblocks coded by the macroblock-unit-coding module 350 .
  • the macroblocks coded by the macroblock-unit-coding module 350 may be macroblocks within a segment previous to a segment on which an integer pel motion is being estimated by the integer pel motion estimation unit 302 .
  • the n is an integer, and the n may be a multiple of 2, such as 2 or 4, in the case of H.264.
  • the 1/n pixel image generation unit 306 is hereinafter referred to as a half/quarter pel image generation unit, for convenience of description.
  • the half/quarter pel image generation unit 306 functions to generate an image obtained by enlarging the reference image frame, generated by the macroblock-unit-coding module 350 , two or four times in order to estimate motion information at an interval of a 1 ⁇ 2 or 1 ⁇ 4 pixel.
  • the half/quarter pel image generation unit 306 performs a function of improving a restoration image, acquired through the inverse transform/inverse quantization unit 360 of the macroblock-unit-coding module 350 , to an image having a better picture quality.
  • the improved image becomes a reference image for images subsequently inputted.
  • the macroblock-unit-coding module 350 performs coding in units of macroblocks through 1/n pixel motion estimation for each of the macroblocks within the segment having its integer pel motion estimated by the segment-unit-coding module 300 .
  • the 1/n pixel motion estimation includes half pel (1 ⁇ 2 pixel) motion estimation, quarter pel (1 ⁇ 4 pixel) motion estimation, and so on.
  • the macroblock-unit-coding module 350 includes a 1/n pixel motion estimation unit 352 , a transform/quantization unit 354 , an intra-mode estimation unit 356 , the inverse transform/inverse quantization unit 360 , and an entropy coding unit 358 .
  • the 1/n pixel motion estimation unit 352 estimates the 1/n pixel motion of a current macroblock using the integer pel motion estimation information, stored in the motion estimation information storage unit 304 of the segment-unit-coding module 300 , and the 1/n pixel image generated by the 1/n pixel image generation unit 306 .
  • the 1/n pixel motion estimation unit 352 is hereinafter referred to as a half/quarter pel motion estimation unit, for convenience of description.
  • the half/quarter pel motion estimation unit 352 estimates motion information at an interval of 1 ⁇ 2 and 1 ⁇ 4 pixels of the current macroblock using the integer pel motion estimation information and the half/quarter pel image.
  • the transform/quantization unit 354 transforms and quantizes the macroblocks estimated by the 1/n pixel motion estimation unit 352 . That is, the transform/quantization unit 354 functions to perform discrete cosine transform and quantize transformed coefficients.
  • the inverse transform/inverse quantization unit 360 performs an opposite function to the transform/quantization unit 354 .
  • the entropy coding unit 358 performs entropy coding bit allocation according to a probability that DCT coefficients quantized by the transform/quantization unit 354 will occur.
  • the intra-mode estimation unit 356 selects an optimal intra-estimation mode by performing estimation from adjacent and surrounding blocks when coding macroblocks in an intra mode and acquires a compensation image, subjected to intra-mode estimation, from previously coded and adjacent macroblocks using the selected intra-estimation mode.
  • segment-unit-coding module 300 and the macroblock-unit-coding module 350 configured as described above are performed in parallel.
  • the segment-unit-coding module 300 and the macroblock-unit-coding module 350 preferably operate to process different segments at the same time.
  • the macroblock-unit-coding module performs half/quarter pel motion estimation and the segment-unit-coding module generates a half/quarter pel image for segments coded by the macroblock-unit-coding module is described below as an example.
  • FIG. 4 is a flowchart illustrating a method of coding an image frame according to the present invention.
  • a case where an image frame is divided into three segments is described as an example.
  • the segment-unit-coding module estimates an integer pel motion for the first segment of an image frame inputted in units of segments at step S 400 .
  • the macroblock-unit-coding module codes each of macroblocks within the first segment having the estimated integer pel motion through half/quarter pel motion estimation at step S 402 b.
  • step S 402 b the segment-unit-coding module estimates an integer pel motion for the second segment of the image frame at step S 402 a. Steps S 402 a and step S 402 b are performed in parallel at the same time.
  • the macroblock-unit-coding module codes each of macroblocks within the second segment, having the estimated integer pel motion, through half/quarter pel motion estimation at step S 404 b.
  • step S 404 b the segment-unit-coding module estimates an integer pel motion for the third segment of the image frame and, at the same time, generates a half/quarter pel image for the first segment, coded at step S 402 b, at step S 404 a.
  • Steps S 404 a and S 404 b are performed in parallel, preferably at the same time.
  • the segment-unit-coding module divides a next input image frame into three segments, estimates an integer pel motion for the first segment of the current image frame and, at the same time, generates a half/quarter pel image for the first segment, coded at step S 404 b, at step S 408 a.
  • step S 406 the macroblock-unit-coding module codes each of macroblocks within the third segment, having the estimated integer pel motion, through half/quarter pel motion estimation at step S 408 b.
  • the segment-unit-coding module and the macroblock-unit-coding module perform the above process until the input of a corresponding image signal is completed.
  • FIG. 5 is a flowchart illustrating a coding method in the case where an image frame is divided into three segments according to the present invention.
  • one image frame consists of an M number of macroblocks, and one segment consists of an M/3 number of macroblocks.
  • the moving image coding apparatus is divided into the macroblock-unit-coding module 500 and the segment-unit-coding module 550 , and coding is executed in the two coding modules in parallel.
  • the macroblock-unit-coding module 500 performs coding
  • the segment-unit-coding module 550 estimates an integer pel motion 560 and generates a half/quarter pel image 570 independently, preferably at the same time.
  • the macroblock-unit-coding module 500 sequentially codes ( 510 ) a first macroblock to an (M/3) th macroblock included in the first segment and, after coding each of the macroblocks included in the first segment, codes ( 520 ) macroblocks included in the second segment.
  • the integer pel motion estimation unit estimates motion vectors for the respective macroblocks of the first segment, having the estimated integer pel motion, in units of half/quarter pels.
  • the macroblock-unit-coding module transforms and quantizes each of the macroblocks having its motions estimated in units of half/quarter pels and performs variable length coding on the quantized macroblocks.
  • the integer pel motion estimation unit estimates an integer pel motion for a next segment which is temporally by one segment earlier than a segment coded in the macroblock-unit-coding module.
  • the half/quarter pel image generation unit since interpolation has to be applied to an image with respect to coded segments, the half/quarter pel image generation unit generates a half/quarter pel image for a segment coded immediately before a segment now being coded.
  • the macroblock-unit-coding module codes ( 500 ) the first segment of the current image frame in units of macroblocks
  • the integer pel motion estimation unit estimates ( 562 ) an integer pel motion for the second segment of the current image frame and the half/quarter pel image generation unit generates ( 572 ) a half/quarter pel image for the third segment of the previous image frame.
  • the macroblock-unit-coding module codes ( 520 ) each of the macroblocks within the second segment of the current image frame, the integer pel motion estimation unit estimates ( 564 ) an integer pel motion for the third segment of the current image frame, and the half/quarter pel image generation unit generates ( 574 ) a half/quarter pel image for the first segment of the current image frame.
  • the macroblock-unit-coding module codes ( 530 ) each of macroblocks within the third segment of the current image frame, the integer pel motion estimation unit estimates ( 566 ) an integer pel motion for the first segment of a next image frame, and the half/quarter pel image generation unit generates ( 576 ) a half/quarter pel image for the second segment of the current image frame.
  • the estimation of an integer pel motion of the current image frame for the third segment and the generation of the half/quarter pel image for the first segment of the current image frame are performed at the same time.
  • the time taken for coding is a maximum value from among “a macroblock-unit coding time”, “an integer pel motion estimation time”, and “a half/quarter pel image generation time”.
  • the processes of estimating an integer pel motion, coding each of macroblocks, and generating a half/quarter pel image are sequentially performed on each of the segments. Meanwhile, all the processes are sequentially performed on each of the segments, but are divided and processed. Accordingly, a process for a next segment can be initiated before all the processes are performed on one segment, and thus operations for a plurality of segments are performed in parallel at the same time.
  • the moving image coding apparatus and method according to the present invention is suitable for moving image coding techniques, enabling high-speed coding because it uses parallel coding in coding an image frame.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Computing Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)

Abstract

The present invention relates to a A moving image coding apparatus and method involving the simultaneous parallel running of respective modules, namely: a segment-unit-coding module in which an input image frame is divided into segment units and the integer pel (integer pixel) motion is estimated while at the same time the 1/n pixel images are generated with respect to the segments of the image coded in the coding module; and a macroblock-unit-coding module in which the segments of the image frame whose the integer pel motion has been estimated in the segment-unit-coding module are received as input in the form of macroblock units and are coded in macroblock units through the 1/n pixel motion estimation. By separating the coding units into different modules and employing the techniques of parallel processing and pipelining, the present invention makes it possible to effect high-speed encoding with reduced delay in the coding rate due to differences in the processing speeds of the respective modules.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This application is a reissue of U.S. Pat. No. 8,705,609 issued on Apr. 22, 2014. This application is related to copending reissue applications of U.S. Pat. No. 8,705,609 including U.S. application Ser. No. 14/811,416 filed on Jul. 28, 2015; U.S. application Ser. No. 14/811,450 filed on Jul. 28, 2015; U.S. application Ser. No. 14/811,468 filed on Jul. 28, 2015; U.S. application Ser. No. 14/811,485 filed on Jul. 28, 2015; U.S. application Ser. No. 14/811,511 filed on Jul. 28, 2015; U.S. application Ser. No. 14/811,535 filed on Jul. 28, 2015; U.S. application Ser. No. 14/811,554 filed on Jul. 28, 2015; and U.S. application Ser. No. 14/811,570 filed on Jul. 28, 2015.
TECHNICAL FIELD
The present invention relates to a moving image coding apparatus and method and, more particularly, to a moving image coding apparatus and method, in which an image frame is simultaneously inputted to a segment-unit-coding module in units of segments and to a macroblock-unit-coding module in units of macroblocks, and the segment-unit-coding module estimates an integer pel (integer pixel) motion for the segments of the image frame and simultaneously generates a 1/n pixel image for the segments of a coded image frame and, at the same time, the macroblock-unit-coding module codes each of the macroblocks within the segment having the estimated integer pel (integer pixel) motion by estimating a 1/n pixel motion.
BACKGROUND ART
In general, digital image data is compressed by an efficient compression method instead of being used as it is because the amount of data is greatly increased owing to its inherent characteristic and in a process of digitalizing an analog signal.
Digital image data is compressed using, chiefly, three kinds of methods. A method of reducing temporal redundancy, a method of reducing spatial redundancy, and a method of reducing generated codes based on a statistical characteristic of the generated codes are chiefly used. A representative method of reducing temporal redundancy is a motion estimation and compensation method, which is being used in most of moving image compression standards, such as MPEG or H.263.
FIG. 1 is a block diagram showing a conventional H.264 moving image coding apparatus.
Referring to FIG. 1, the H.264 moving image coding apparatus includes a motion estimation unit ME 100, a motion compensation unit MC 110, an intra-mode estimation unit 120, a transform/quantization unit 130, an inverse transform/inverse quantization unit 140, an entropy coding unit 150, a deblocking filter 160, and a reference image storage unit 170.
The motion estimation unit 100 performs a function of finding a motion vector from several sheets of reference images and a macroblock mode determination function of finding an optimal macroblock type with the least bit rate and error.
Furthermore, the motion estimation unit 100 estimates an integer pel motion in units of macroblocks and then perform half pel and quarter pel motion estimation.
The motion compensation unit 110 performs a function of acquiring a compensation image from the reference images based on the motion vector and the macroblock mode type found by the motion estimation unit 100. Here, the motion compensation unit 110 is limited to a structure for acquiring a difference between two images, and a process of acquiring subsequent compensation images is continued by the transform/quantization unit 130.
The intra-mode estimation unit 120 performs a function of selecting an optimal intra-estimation mode by performing estimation from adjacent and surrounding blocks when coding a macroblock in an intra mode and performs a function of acquiring a compensation image, subjected to intra-mode estimation, from previously-coded and adjacent blocks using the selected intra-estimation mode.
The transform/quantization unit 130 performs a discrete cosine transform and performs a function of quantizing the transformed coefficients. The inverse transform/inverse quantization unit 140 performs an opposite function to the transform/quantization unit 130.
The entropy coding unit 150 performs entropy coding using bit allocation according to a probability that quantized DCT coefficients will occur.
The deblocking filter 160 performs a function of improving a restoration image, acquired through the inverse transform/inverse quantization unit 140, to an image of a better picture quality. The improved image is stored in the reference image storage unit 170, and it becomes a reference image for subsequently inputted images.
FIG. 2 is a flowchart illustrating a method of the conventional H.264 moving image coding apparatus coding a moving image.
Referring to FIG. 2, the H.264 moving image coding apparatus performs coding on the first macroblock to nth macroblock of an input image frame in units of macroblocks by estimating integer pel, half pel, and quarter pel motions at step S200. After the coding of the image frame is completed at step S202, the H.264 moving image coding apparatus decodes the coded image frame at step S204.
The method of the H.264 moving image coding apparatus coding the macroblocks is described below. After estimating the integer pel motion, the H.264 moving image coding apparatus estimates the half/quarter pel motions with respect to the integer pel motion-estimation image. Next, the H.264 moving image coding apparatus determines a mode and, and transforms, quantizes, and codes the half/quarter pel motion-estimated images using the determined mode.
Next, the H.264 moving image coding apparatus generates a half/quarter pel image by interpolating the decoded image at step S206. The generated half/quarter pel image is used to estimate the motion of a next image.
As described above, all the functions other than the function of generating the half/quarter pel image through interpolation are sequentially performed in order to code one macroblock. The coding of a next macroblock is started after the coding of a previous macroblock is finished. After all the macroblocks are coded, an image decoded from the coded image is obtained. The half/quarter pel image is generated by interpolating the decoded image and is prepared to be used to estimate the motion of a next image.
The above-described H.264 moving image coding apparatus consumes a lot of time in order to perform coding because of a heavy computational load. In particular, some processes, such as motion estimation, the generation of a quarter pel image, and deblocking, require a heavy computational load, thus causing to delay the coding time.
In particular, a block-based coding apparatus, such as H.264, divides an image frame into macroblocks and performs coding in units of macroblocks. The block-based coding apparatus is configured to code a next macroblock after coding a previous macroblock.
In the case where a specific process constituting a step of coding the macroblocks consumes a greater execution time than other processes, a phenomenon in which the entire coding time is delayed by the specific process is generated.
This phenomenon is greatly influenced in terms of performance in the case of a coder having, in particular, a pipeline structure. In order to solve the problem, there is a problem in that the time that it takes to execute processes, requiring a lot of time, is reduced similarly to other processes or the coder structure has to be changed.
DISCLOSURE Technical Problem
An object of the present invention is to provide a moving image coding apparatus and method, which is capable of performing parallel coding on an image frame in units of segments and in units of macroblocks at the same time, thereby enabling high-speed coding.
Another object of the present invention is to provide a moving image coding apparatus and method, which can reduce the delay of the coding speed due to a difference in the processing speed between modules and enables high-speed encoding by applying different coding units to modules and using parallel processing and pipelining schemes.
For example, a system simultaneously utilizing a Graphic Processing Unit (GPU) and a CPU may have improved performance because the GPU and the CPU are executed in parallel. However, the units of a task, processed by the CPU and the GPU, and the processing speeds of the CPU and the GPU are quite different because the GPU includes several tens to several hundreds of processing units and the CPU has one processing unit with very high performance. An object of the present invention is to make parallel two devices having different units of a task and different processing speeds, such as a GPU and a CPU, by taking the above problem in view of moving image coding so that one device (for example, GPU) of the two devices processes the input of a segment unit (i.e., a great unit) and the other device (for example, CPU) of the two devices processes the input of a macroblock unit. Furthermore, an object of the present invention is to enable high-speed coding by dividing an image coding process into a process of performing processing in units of segments at once and a process of performing processing in units of macroblocks in order to be applied to the parallelized structure and by suggesting a coder structure suitable for the processes.
Technical Solution
To achieve the above objects, according to an aspect of the present invention, there is provided a moving image coding apparatus, comprising a segment-unit-coding module for receiving a segment-unit image, estimating an integer pel motion and, at the same time, generating an image, coded by a macroblock-unit-coding module, as a 1/n pixel image; and a macroblock-unit-coding module for receiving then image, having the estimated integer pel motion, from the segment-unit-coding module in units of macroblocks and coding each of the macroblocks through 1/n pixel motion estimation for each macroblock.
Furthermore, the segment-unit-coding module estimates an integer pixel motion for a next segment in advance, before the macroblock-unit-coding module codes the image of the macroblock units having the estimated integer pixel motion.
Furthermore, while the macroblock-unit-coding module codes the image of the macroblock units having the estimated integer pixel motion, the segment-unit-coding module estimates an integer pixel motion for a next segment image and generates a 1/n pixel image for a previous segment image independently and at the same time.
Furthermore, the segment-unit-coding module comprise an integer pel motion estimation unit for estimating a motion vector of an integer pixel unit for a current image frame of segment units; a motion estimation information storage unit for storing information estimated by the integer pel motion estimation unit; and a 1/n pixel image generation unit for generating an image, coded by the macroblock-unit-coding module, as the 1/n pixel image.
Furthermore, the 1/n pixel image generation unit generates the 1/n pixel image by enlarging a reference image frame n times in order to estimate a motion for the image, coded by the macroblock-unit-coding module, at an interval of a 1/n pixel.
Furthermore, the macroblock-unit-coding module comprises a 1/n pixel motion estimation unit for estimating a 1/n pixel motion for the current macroblock image using the integer pel motion estimation information, stored in the motion estimation information storage unit, and the 1/n pixel image generated by the 1/n pixel image generation unit; a transform/quantization unit for transforming and quantizing the macroblock image estimated by the 1/n pixel motion estimation unit; an entropy coding unit for coding the image transformed and quantized by the transform/quantization unit; and an inverse transform/inverse quantization unit for inverse-transforming and inverse-quantizing the image transformed and quantized by the transform/quantization unit.
Furthermore, the macroblock-unit-coding module further comprises an intra-mode estimation unit for selecting an optimal intra-estimation mode by performing estimation from adjacent and surrounding blocks when coding the macroblock image in an intra mode and for acquiring a compensation image, subjected to intra-mode estimation, from previously coded and adjacent macroblocks using the selected intra-estimation mode.
Furthermore, the operations of the segment-unit-coding module and the macroblock-unit-coding module are performed in parallel.
Meanwhile, according to another aspect of the present invention, there is provided a method of a moving image coding apparatus coding an image frame, comprising the steps of (a) estimating an integer pel motion for segments of the image frame using a segment-unit-coding module; (b) coding the segments of the image frame, having the estimated integer pixel motion, in units of macroblocks through 1/n pixel motion estimation for each of the macroblocks within the segment using a macroblock-unit-coding module; and (c) generating a 1/n pixel image for the segments of the image frame, coded in units of the macroblocks by the segment-unit-coding module.
The steps (a) to (c) are performed on the segments of an image frame, sequentially inputted, in series.
Furthermore, the steps (a), (b), and (c) are performed on the different segments of the image frame in parallel.
Furthermore, in the case where an integer pixel motion for a nth inputted segment is estimated in the step (a) and a 1/n pixel image for segments, inputted by a predetermined number of times earlier than the nth inputted segment, is generated in the step (c), the steps (a) and (c) are performed by the segment-unit-coding module at the same time.
Furthermore, the method further comprises the step of dividing the image frame into an N number of segments before the step (a).
Furthermore, the step (b) comprises steps of estimating a motion vector of a 1/n pixel unit using the 1/n pixel image generated through the 1/n pixel motion estimation for each of the macroblocks within the segment having the integer pixel motion estimated by the segment-unit-coding module; quantizing each of the macroblocks having a motion estimated in unit of the 1/n pixel, having the motion estimated; and performing variable length coding on the quantized macroblocks.
Furthermore, generating a 1/n pixel image for a previous segment of the image frame in the step (c) comprises generating an image obtained by enlarging a reference image frame of the segments within the image frame, generated by the macroblock-unit-coding module, n times.
Meanwhile, according to yet another aspect of the present invention, there is provided a method of a moving image coding apparatus coding an image frame, comprising the steps of (a) estimating an integer pel motion for the first segment of a first input image frame; (b) coding each of macroblocks within the first segment, having the estimated integer pixel motion, through 1/n pixel motion estimation and simultaneously estimating an integer pixel motion for the second segment of the image frame; and (c) coding each of macroblocks within the second segment, having the estimated integer pixel motion, through 1/n pixel motion estimation, estimates an integer pixel motion for the third segment of the image frame, and generates a 1/n pixel image for the coded first segment simultaneously.
Here, generating the 1/n pixel image for the coded first segment in the step (a) refers to generating an image obtained by enlarging a reference image frame of the first segment, coded in units of the macroblocks, n times.
According to further yet another aspect of the present invention, there is provided a recording medium on which a program for coding an image frame is recorded, wherein the program executes the steps of (a) a segment-unit-coding module estimating an integer pel motion for the third segment of a current image frame inputted in units of segments and, at the same time, generating a 1/n pixel image for the first segment of the current image frame and (b) a macroblock-unit-coding module coding each of macroblocks within the second segment of the current frame, having an integer pixel motion estimated by the segment-unit-coding module, through 1/n pixel motion estimation.
According to further yet another aspect of the present invention, there is provided a recording medium on which a program for coding an image frame is recorded, wherein the program executes the steps of (a) estimating an integer pel motion and a 1/n pixel motion for the third segment of a current image frame inputted in units of segments and, at the same time, generating a 1/n pixel image for the coded first segment of the current image frame and (b) coding the second segment of the current image frame, having an integer pixel motion and a 1/n motion estimated, in units of macroblocks.
Advantageous Effects
According to the present invention, there can be provided a moving image coding apparatus and method, which enables high-speed coding using parallel coding in encoding an image frame.
Furthermore, encoding can be performed at high speed by applying different coding units to modules and reducing the delay of the coding speed due to a difference in the processing speed between the modules.
Furthermore, an image is processed in parallel in units of segments and in units of macroblocks at the same time, and thus the time taken for coding is reduced. Accordingly, moving image coding apparatus and method capable of improving the coding speed can be provided.
DESCRIPTION OF DRAWINGS
FIG. 1 is a block diagram showing a conventional H.264 moving image coding apparatus;
FIG. 2 is a flowchart illustrating a method of the conventional H.264 moving image coding apparatus coding a moving image;
FIG. 3 is a block diagram schematically showing the construction of a moving image coding apparatus according to the present invention;
FIG. 4 is a flowchart illustrating a method of coding an image frame according to the present invention; and
FIG. 5 is a flowchart illustrating a coding method in the case where an image frame is divided into three segments according to the present invention.
 MODE FOR INVENTION
Hereinafter, detailed contents about the above-described objects, technical configurations, and operational effects thereof of the present invention will be more clearly understood from the following detailed description of the present invention, which will be described with reference to the attached drawings.
FIG. 3 is a block diagram schematically showing the construction of a moving image coding apparatus according to the present invention.
Referring to FIG. 3, the moving image coding apparatus includes a segment-unit-coding module 300 for dividing an input image frame in units of segments and estimating an integer pixel (integer pel) motion and a macroblock-unit-coding module 350 for coding each of macroblocks within the segment having its integer pel motion estimated by the segment-unit-coding module 300.
The segment-unit-coding module 300 functions to receive the image frame in units of segments and to estimate the integer pel motion and, at the same time, to generate a 1/n pixel image for the macroblocks coded by the macroblock-unit-coding module 350. Here, the segment refers to an image frame vertically divided into a predetermined number. The 1/n pixel image refers to an image obtained by enlarging a reference image frame, generated by the macroblock-unit-coding module 350, two or four times in order to estimate motion information at an interval of ½ or ¼ pixels. Here, n is an integer, and n may become a multiple of 2, such as 2 or 4 in the case of H.264.
The segment-unit-coding module 300 includes an integer pel motion estimation unit 302, a motion estimation information storage unit 304, and a 1/n pixel image generation unit 306.
The integer pel motion estimation unit 302 divides a current input image frame into a predetermined number of segments and estimates motion vectors of the current image frame, having the units of segments, in unit of an integer pel (integer pixel).
Here, the segments should be understood as a concept to temporally divide the segments in order to process an image rather than a concept to divide temporally consecutive image data. That is, even in the case of image data constituting one scene, in the case where the image data is divided into specific parts and the parts are processed with precedence relationships temporally, each of the parts may be understood as a segment.
That is, the integer pel motion estimation unit 302 calculates an integer pel cost value for a segment, having a predetermined size, of a reference frame previous to or after a segment divided with a specific size and performs integer pixel search. Here, the cost value may include the Sum of Absolute Difference (SAD) value of an absolute difference.
Here, the motion information is vector information in the x, y form and is information indicating how much has a block within an image frame been moved in 2-D from a previous reference image frame. Accordingly, the integer pel motion estimation unit estimates motion information as pixel interval resolution.
The motion estimation information storage unit 304 stores the motion estimation information estimated by the integer pel motion estimation unit 302.
The 1/n pixel image generation unit 306 generates the 1/n pixel image for the macroblocks coded by the macroblock-unit-coding module 350. Here, the macroblocks coded by the macroblock-unit-coding module 350 may be macroblocks within a segment previous to a segment on which an integer pel motion is being estimated by the integer pel motion estimation unit 302. Furthermore, the n is an integer, and the n may be a multiple of 2, such as 2 or 4, in the case of H.264.
The 1/n pixel image generation unit 306 is hereinafter referred to as a half/quarter pel image generation unit, for convenience of description.
The half/quarter pel image generation unit 306 functions to generate an image obtained by enlarging the reference image frame, generated by the macroblock-unit-coding module 350, two or four times in order to estimate motion information at an interval of a ½ or ¼ pixel.
The half/quarter pel image generation unit 306 performs a function of improving a restoration image, acquired through the inverse transform/inverse quantization unit 360 of the macroblock-unit-coding module 350, to an image having a better picture quality. The improved image becomes a reference image for images subsequently inputted.
The macroblock-unit-coding module 350 performs coding in units of macroblocks through 1/n pixel motion estimation for each of the macroblocks within the segment having its integer pel motion estimated by the segment-unit-coding module 300. Here, the 1/n pixel motion estimation includes half pel (½ pixel) motion estimation, quarter pel (¼ pixel) motion estimation, and so on.
The macroblock-unit-coding module 350 includes a 1/n pixel motion estimation unit 352, a transform/quantization unit 354, an intra-mode estimation unit 356, the inverse transform/inverse quantization unit 360, and an entropy coding unit 358.
The 1/n pixel motion estimation unit 352 estimates the 1/n pixel motion of a current macroblock using the integer pel motion estimation information, stored in the motion estimation information storage unit 304 of the segment-unit-coding module 300, and the 1/n pixel image generated by the 1/n pixel image generation unit 306.
The 1/n pixel motion estimation unit 352 is hereinafter referred to as a half/quarter pel motion estimation unit, for convenience of description.
The half/quarter pel motion estimation unit 352 estimates motion information at an interval of ½ and ¼ pixels of the current macroblock using the integer pel motion estimation information and the half/quarter pel image.
The transform/quantization unit 354 transforms and quantizes the macroblocks estimated by the 1/n pixel motion estimation unit 352. That is, the transform/quantization unit 354 functions to perform discrete cosine transform and quantize transformed coefficients.
The inverse transform/inverse quantization unit 360 performs an opposite function to the transform/quantization unit 354.
The entropy coding unit 358 performs entropy coding bit allocation according to a probability that DCT coefficients quantized by the transform/quantization unit 354 will occur.
The intra-mode estimation unit 356 selects an optimal intra-estimation mode by performing estimation from adjacent and surrounding blocks when coding macroblocks in an intra mode and acquires a compensation image, subjected to intra-mode estimation, from previously coded and adjacent macroblocks using the selected intra-estimation mode.
The operations of the segment-unit-coding module 300 and the macroblock-unit-coding module 350 configured as described above are performed in parallel. The segment-unit-coding module 300 and the macroblock-unit-coding module 350 preferably operate to process different segments at the same time.
Hereinafter, the operation of the coding apparatus is described with reference to FIGS. 4 and 5.
For convenience of description, an example in which the macroblock-unit-coding module performs half/quarter pel motion estimation and the segment-unit-coding module generates a half/quarter pel image for segments coded by the macroblock-unit-coding module is described below as an example.
FIG. 4 is a flowchart illustrating a method of coding an image frame according to the present invention. Hereinafter, a case where an image frame is divided into three segments is described as an example.
Referring to FIG. 4, the segment-unit-coding module estimates an integer pel motion for the first segment of an image frame inputted in units of segments at step S400.
The macroblock-unit-coding module codes each of macroblocks within the first segment having the estimated integer pel motion through half/quarter pel motion estimation at step S402b.
While step S402b is performed, the segment-unit-coding module estimates an integer pel motion for the second segment of the image frame at step S402a. Steps S402a and step S402b are performed in parallel at the same time.
Next, the macroblock-unit-coding module codes each of macroblocks within the second segment, having the estimated integer pel motion, through half/quarter pel motion estimation at step S404b.
While step S404b is performed, the segment-unit-coding module estimates an integer pel motion for the third segment of the image frame and, at the same time, generates a half/quarter pel image for the first segment, coded at step S402b, at step S404a.
Steps S404a and S404b are performed in parallel, preferably at the same time.
After an integer pel motion is completed up to the third segment of the current image frame at step S406, the segment-unit-coding module divides a next input image frame into three segments, estimates an integer pel motion for the first segment of the current image frame and, at the same time, generates a half/quarter pel image for the first segment, coded at step S404b, at step S408a.
While step S406 is performed, the macroblock-unit-coding module codes each of macroblocks within the third segment, having the estimated integer pel motion, through half/quarter pel motion estimation at step S408b.
The segment-unit-coding module and the macroblock-unit-coding module perform the above process until the input of a corresponding image signal is completed.
FIG. 5 is a flowchart illustrating a coding method in the case where an image frame is divided into three segments according to the present invention.
Referring to FIG. 5, one image frame consists of an M number of macroblocks, and one segment consists of an M/3 number of macroblocks.
The moving image coding apparatus is divided into the macroblock-unit-coding module 500 and the segment-unit-coding module 550, and coding is executed in the two coding modules in parallel.
That is, the macroblock-unit-coding module 500 performs coding, and the segment-unit-coding module 550 estimates an integer pel motion 560 and generates a half/quarter pel image 570 independently, preferably at the same time.
The macroblock-unit-coding module 500 sequentially codes (510) a first macroblock to an (M/3)th macroblock included in the first segment and, after coding each of the macroblocks included in the first segment, codes (520) macroblocks included in the second segment.
A method of the macroblock-unit-coding module 500 coding each of the macroblocks is described below. The integer pel motion estimation unit estimates motion vectors for the respective macroblocks of the first segment, having the estimated integer pel motion, in units of half/quarter pels.
Next, the macroblock-unit-coding module transforms and quantizes each of the macroblocks having its motions estimated in units of half/quarter pels and performs variable length coding on the quantized macroblocks.
However, the integer pel motion estimation results and a half/quarter pel image have to be executed before the macroblock-unit-coding module is executed.
To this end, the integer pel motion estimation unit estimates an integer pel motion for a next segment which is temporally by one segment earlier than a segment coded in the macroblock-unit-coding module.
Furthermore, since interpolation has to be applied to an image with respect to coded segments, the half/quarter pel image generation unit generates a half/quarter pel image for a segment coded immediately before a segment now being coded.
Consequently, while the macroblock-unit-coding module codes (500) the first segment of the current image frame in units of macroblocks, the integer pel motion estimation unit estimates (562) an integer pel motion for the second segment of the current image frame and the half/quarter pel image generation unit generates (572) a half/quarter pel image for the third segment of the previous image frame.
After the above process is completed, the macroblock-unit-coding module codes (520) each of the macroblocks within the second segment of the current image frame, the integer pel motion estimation unit estimates (564) an integer pel motion for the third segment of the current image frame, and the half/quarter pel image generation unit generates (574) a half/quarter pel image for the first segment of the current image frame.
Next, the macroblock-unit-coding module codes (530) each of macroblocks within the third segment of the current image frame, the integer pel motion estimation unit estimates (566) an integer pel motion for the first segment of a next image frame, and the half/quarter pel image generation unit generates (576) a half/quarter pel image for the second segment of the current image frame.
As described above, while the second segment is coded in units of macroblocks, the estimation of an integer pel motion of the current image frame for the third segment and the generation of the half/quarter pel image for the first segment of the current image frame are performed at the same time. The time taken for coding is a maximum value from among “a macroblock-unit coding time”, “an integer pel motion estimation time”, and “a half/quarter pel image generation time”.
As described above, the processes of estimating an integer pel motion, coding each of macroblocks, and generating a half/quarter pel image are sequentially performed on each of the segments. Meanwhile, all the processes are sequentially performed on each of the segments, but are divided and processed. Accordingly, a process for a next segment can be initiated before all the processes are performed on one segment, and thus operations for a plurality of segments are performed in parallel at the same time.
As described above, a person having ordinary skill in the art to which the present invention pertains will understand that the present invention may be implemented in various detailed forms without changing the technical spirit or indispensable characteristics of the present invention. It will be understood that the above-described embodiments are illustrative and not limitative from all aspects. The scope of the present invention is defined by the appended claims rather than the detailed description, and the present invention should be construed to cover all modifications or variations induced from the meaning and scope of the appended claims and their equivalents.
INDUSTRIAL APPLICABILITY
As described above, the moving image coding apparatus and method according to the present invention is suitable for moving image coding techniques, enabling high-speed coding because it uses parallel coding in coding an image frame.

Claims (13)

The invention claimed is:
1. A moving image coding apparatus for coding an input image frame, comprising:
a segment-unit-coding module for receiving the input image frame as a segment-unit image by dividing the input image frame into N number of segments, estimating an integer pel motion, and generating an image therefrom, coded by a macroblock-unit-coding module, as a 1/n pixel image; and
a macroblock-unit-coding module for receiving the segment-unit image, having the estimated integer pel motion, from the segment-unit-coding module, in units of macroblocks and coding each of the macroblocks through 1/n pixel motion estimation for each macroblock,
wherein the segment-unit-coding module estimates an integer pixel motion for a next segment in advance before the macroblock-unit-coding module codes the image of the macroblock units having the estimated integer pixel motion, and
wherein while the operation of the macroblock-unit-coding module for coding the image of the macroblock units having the estimated integer pixel motion proceeds the operation of the segment-unit-coding module for estimating an integer pixel motion for a next segment and for generating a 1/n pixel image for a previous segment image occurs independently and simultaneously and with both operations performed in parallel, and
wherein the operations of the segment-unit-coding module and the macroblock-unit-coding module are performed on different segments of the image frame in parallel.
2. The moving image coding apparatus according to claim 1, wherein the segment-unit-coding module comprises:
an integer pel motion estimation unit for estimating a motion vector of an integer pixel unit for a current image frame of segment units;
a motion estimation information storage unit for storing information estimated by the integer pel motion estimation unit; and
a 1/n pixel image generation unit for generating an image, coded by the macroblock-unit-coding module, as the 1/n pixel image.
3. The moving image coding apparatus according to claim 2, wherein the 1/n pixel image generation unit generates the 1/n pixel image by enlarging a reference image frame n times in order to estimate a motion for the image, coded by the macroblock-unit-coding module, at an interval of a 1/n pixel.
4. The moving image coding apparatus according to claim 2, wherein the macroblock-unit-coding module comprises:
a 1/n pixel motion estimation unit for estimating a 1/n pixel motion for the current macroblock image using the integer pel motion estimation information, stored in the motion estimation information storage unit, and the 1/n pixel image generated by the 1/n pixel image generation unit;
a transform/quantization unit for transforming and quantizing the macroblock image estimated by the 1/n pixel motion estimation unit;
an entropy coding unit for coding the image transformed and quantized by the transform/quantization unit; and
an inverse transform/inverse quantization unit for inverse-transforming and inverse-quantizing the image transformed and quantized by the transform/quantization unit.
5. The moving image coding apparatus according to claim 4, wherein the macroblock-unit-coding module further comprises an intra-mode estimation unit for selecting an optimal intra-estimation mode by performing estimation from adjacent and surrounding blocks when coding the macroblock image in an intra mode and for acquiring a compensation image, subjected to intra-mode estimation, from previously coded and adjacent macroblocks using the selected intra-estimation mode.
6. A method of a moving image coding apparatus coding an image frame, the method comprising the steps of:
(a) dividing the image frame into N number of segments;
(b) estimating an integer pel motion for segments of the image frame using a segment-unit-coding module;
(c) coding the segments of the image frame, having the estimated integer pixel motion, in units of macroblocks through 1/n pixel motion estimation for each of the macroblocks within each unit segment using a macroblock-unit-coding module; and
(d) generating a 1/n pixel image for the segments of the image frame, coded in units of the macroblocks by the segment-unit-coding module,
wherein in a case where an integer pixel motion for a nth inputted segment is estimated in step (b) and a 1/n pixel image for segments, inputted a predetermined number of times earlier than the nth inputted segment, is generated in step (d), the steps (b) and (d) are performed by the segment-unit-coding module independently and simultaneously,
wherein the steps (b), (c) and (d) are performed in parallel and simultaneously, and
wherein the steps (b), (c), and (d) are performed on different segments of the image frame in parallel.
7. The method according to claim 6, wherein the step (c) comprises steps of:
estimating a motion vector of a 1/n pixel unit using the 1/n pixel image generated through the 1/n pixel motion estimation for each of the macroblocks within one unit segment having the integer pixel motion estimated by the segment-unit-coding module;
quantizing each of the macroblocks having a motion estimated in unit of the 1/n pixel; and
performing variable length coding on the quantized macroblocks.
8. The method according to claim 6, wherein generating a 1/n pixel image for a previous segment of the image frame in the step (d) comprises generating an image obtained by enlarging a reference image frame of the segments within the image frame, generated by the macroblock-unit-coding module, n times.
9. A method of a moving image coding apparatus coding an image frame, the method comprising the steps of:
(a) dividing the image frame into N number of unit segments
(b) estimating an integer pel motion for a first segment of a first input image frame;
(c) coding each of macroblocks within the first segment, having the estimated integer pixel motion, through 1/n pixel motion estimation while simultaneously estimating an integer pixel motion for a second unit segment of the image frame;
(d) coding each of macroblocks within the second segment, having the estimated integer pixel motion, through 1/n pixel motion estimation, estimates an integer pixel motion for a third segment of the image frame, and generates a 1/n pixel image for the coded first segment simultaneously
wherein the steps (d), estimates an integer pixel motion for a third segment of the image frame, and generates a 1/n pixel image for the coded first segment independently and simultaneously, and
wherein the steps (b), (c), and (d) are performed on different segments of the image frame in parallel.
10. The method according to claim 9, wherein generating the 1/n pixel image for the coded first segment-in the step (a) (d) comprises generating an image obtained by enlarging a reference image frame of the first segment, coded in units of the macroblocks, n times.
11. A non-transitory recording medium on which a program for coding an image frame is recorded, the program is configured to execute the method of claim 6.
12. A non-transitory recording medium on which a program for coding an image frame is recorded, the program is configured to execute the method of claim 6 comprising the steps:
(a) estimating an integer pel motion and a 1/n pixel motion for a third segment of a current image frame consisting of N segments inputted in units of segments while simultaneously generating a 1/n pixel image for a coded first segment of the current image frame; and
coding a second segment of the current image frame, having an integer pixel motion and a 1/n motion estimated, in units of macroblocks.
13. A moving image coding apparatus for coding an input image frame, comprising:
one or more processors including a segment-unit-coding module for (i) determining a number of segments which are included in the input image frame, (ii) dividing, based on the determined number of the segments, the input image frame into a plurality of N segments, the input image frame being divided into rectangular segments by at least one vertical boundary line, and (iii) estimating an integer pel motion, and
wherein one of the processors includes a macroblock-unit-coding module for encoding each of the segments by a block unit and generating an image after coding as a 1/n pixel image,
wherein the plurality of the segments are encoded independently of each other,
wherein each of the segments is encoded using at least one of motion prediction or intra prediction,
wherein the motion prediction relating to at least one block included in a first segment is performed independently of a second segment other than the first segment in the input image frame, the first segment being one of the plurality of the segments,
wherein the block has one of a first motion vector of an integer pixel unit, a second motion vector of a half pixel unit and a third motion vector of a quarter pixel unit and
wherein the integer pixel motion estimation for a segment and sub-pixel motion estimation for other segments are performed in parallel and simultaneously.
US14/811,157 2008-08-11 2009-05-11 Moving image coding device and method Active 2030-05-15 USRE47004E1 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US14/811,157 USRE47004E1 (en) 2008-08-11 2009-05-11 Moving image coding device and method

Applications Claiming Priority (5)

Application Number Priority Date Filing Date Title
KR10-2008-0078410 2008-08-11
KR1020080078410A KR101085963B1 (en) 2008-08-11 2008-08-11 Apparatus and Method for encoding video
US14/811,157 USRE47004E1 (en) 2008-08-11 2009-05-11 Moving image coding device and method
PCT/KR2009/002462 WO2010018916A1 (en) 2008-08-11 2009-05-11 Moving image coding device and method
US12/737,764 US8705609B2 (en) 2008-08-11 2009-05-11 Moving image coding device and method

Publications (1)

Publication Number Publication Date
USRE47004E1 true USRE47004E1 (en) 2018-08-21

Family

ID=41669047

Family Applications (3)

Application Number Title Priority Date Filing Date
US14/811,157 Active 2030-05-15 USRE47004E1 (en) 2008-08-11 2009-05-11 Moving image coding device and method
US12/737,764 Ceased US8705609B2 (en) 2008-08-11 2009-05-11 Moving image coding device and method
US14/811,535 Active 2030-05-15 USRE48451E1 (en) 2008-08-11 2009-05-11 Moving image coding device and method

Family Applications After (2)

Application Number Title Priority Date Filing Date
US12/737,764 Ceased US8705609B2 (en) 2008-08-11 2009-05-11 Moving image coding device and method
US14/811,535 Active 2030-05-15 USRE48451E1 (en) 2008-08-11 2009-05-11 Moving image coding device and method

Country Status (5)

Country Link
US (3) USRE47004E1 (en)
EP (10) EP2953365B1 (en)
KR (1) KR101085963B1 (en)
CN (1) CN102150431A (en)
WO (1) WO2010018916A1 (en)

Families Citing this family (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR101673186B1 (en) 2010-06-09 2016-11-07 삼성전자주식회사 Apparatus and method of processing in parallel of encoding and decoding of image data by using correlation of macroblock
CN103297777A (en) * 2013-05-23 2013-09-11 广州高清视信数码科技股份有限公司 Method and device for increasing video encoding speed
KR102476204B1 (en) 2017-10-19 2022-12-08 삼성전자주식회사 Multi-codec encoder and multi-codec encoding system including the same
US10764603B2 (en) 2018-12-31 2020-09-01 Alibaba Group Holding Limited Resolution-adaptive video coding

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5488419A (en) * 1992-03-13 1996-01-30 Matsushita Electric Industrial Co., Ltd. Video compression coding and decoding with automatic sub-pixel frame/field motion compensation
US5929940A (en) 1995-10-25 1999-07-27 U.S. Philips Corporation Method and device for estimating motion between images, system for encoding segmented images
US20050123039A1 (en) 2003-12-06 2005-06-09 Samsung Electronics Co., Ltd. Motion estimation method for motion picture encoding and recording medium having program recorded thereon to implement the motion estimation method
US20050207496A1 (en) * 2004-03-17 2005-09-22 Daisaku Komiya Moving picture coding apparatus
US20060013317A1 (en) 2000-05-08 2006-01-19 Jani Lainema Method for encoding and decoding video information, a motion compensated video encoder and a coresponding decoder
EP1675406A2 (en) 2004-12-21 2006-06-28 STMicroelectronics, Inc. Method and system for fast implementation of subpixel interpolation
US20070165716A1 (en) * 2006-01-13 2007-07-19 Shinji Kitamura Signal processing device, image capturing device, network camera system and video system
US20080117971A1 (en) * 2006-11-21 2008-05-22 Avinash Ramachandran Motion refinement engine with a plurality of cost calculation methods for use in video encoding and methods for use therewith

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
AU2003208559A1 (en) * 2002-04-11 2003-10-20 Koninklijke Philips Electronics N.V. Motion estimation unit and method of estimating a motion vector
KR20060004060A (en) * 2004-07-08 2006-01-12 삼성전자주식회사 Motion estimation method based on multi resolution, and recording medium storing a program to implement thereof

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5488419A (en) * 1992-03-13 1996-01-30 Matsushita Electric Industrial Co., Ltd. Video compression coding and decoding with automatic sub-pixel frame/field motion compensation
US5929940A (en) 1995-10-25 1999-07-27 U.S. Philips Corporation Method and device for estimating motion between images, system for encoding segmented images
US20060013317A1 (en) 2000-05-08 2006-01-19 Jani Lainema Method for encoding and decoding video information, a motion compensated video encoder and a coresponding decoder
US20050123039A1 (en) 2003-12-06 2005-06-09 Samsung Electronics Co., Ltd. Motion estimation method for motion picture encoding and recording medium having program recorded thereon to implement the motion estimation method
CN1627825A (en) 2003-12-08 2005-06-15 三星电子株式会社 Motion estimation method for motion picture encoding
US20050207496A1 (en) * 2004-03-17 2005-09-22 Daisaku Komiya Moving picture coding apparatus
EP1675406A2 (en) 2004-12-21 2006-06-28 STMicroelectronics, Inc. Method and system for fast implementation of subpixel interpolation
US20070165716A1 (en) * 2006-01-13 2007-07-19 Shinji Kitamura Signal processing device, image capturing device, network camera system and video system
US20080117971A1 (en) * 2006-11-21 2008-05-22 Avinash Ramachandran Motion refinement engine with a plurality of cost calculation methods for use in video encoding and methods for use therewith

Non-Patent Citations (19)

* Cited by examiner, † Cited by third party
Title
Cheng Lu, "Motion Estimation of Multi-Frame Block Matching in the Super-resolution Restoration", Natural Sciences Journal of Harbin Normal University, vol. 22, No. 6, 2006, pp. 9-12.
Eric Debes et al., "Efficient Exploitation of Parallelism in General Purpose Processor Based Systems for Image and Video Processing Applications," Visual Communications and Image Processing 2002, Proceedings of SPIE vol. 4671, pp. 667-678.
European Examination Report, European Application No. 09806778.8, dated Jun. 13, 2017, 6 pages.
Fuldseth, A. et al., "Tiles," Document: JCTVC-F335, Joint Collaborative Team on Video Coding (JTC-VC) of ITU-T SG16 WP3 and ISO/IEC JTC1/SC29/WG11, 6th Meeting, Torino, IT, Jul. 14-22, 2011, 15 pages.
Kim, J-M. et al., "Multithread Video Coding Processor for the Videophone," IEEE Signal Processing Society Workshop on VLSI Signal Processing, VIII, 1995, IEEE, Sep. 16, 1995, pp. 470-480.
Sebastian Fluegel et al., "A Highly Parallel Sub-Pel Accurate Motion Estimator for H.264," 2006 IEEE, pp. 387-390.
Steven Ge et al., "Efficient Multithreading Implementation of H.264 Encoder on Intel Hyper-Threading Architectures," Information, Communications and Signal Processing, 2003 and Fourth Pacific Rim Conference on Multimedia. Proceedings of the 2003 Joint Conference of the Fourth International Conference on, 2003, pp. 469-473 vol. 1. *
Thomas Wiegand et al., "Overview of the H.264/AVC Video Coding Standard," IEEE Transactions on Circuits and Systems for Video Technology, vol. 13, No. 7, Jul. 2003, pp. 560-576. *
United States Office Action, U.S. Appl. No. 14/811,416, dated Jul. 18, 2017, 17 pages.
United States Office Action, U.S. Appl. No. 14/811,416, dated Jun. 17, 2016, 14 pages.
United States Office Action, U.S. Appl. No. 14/811,450, dated Jul. 6, 2016, 15 pages.
United States Office Action, U.S. Appl. No. 14/811,450, dated Jun. 15, 2017. 17 pages.
United States Office Action, U.S. Appl. No. 14/811,468, dated Jul. 11, 2016, 15 pages.
United States Office Action, U.S. Appl. No. 14/811,485, dated Jul. 28, 2016, 15 pages.
United States Office Action, U.S. Appl. No. 14/811,511, dated Jun. 21, 2016, 15 pages.
United States Office Action, U.S. Appl. No. 14/811,535, dated Jun. 9, 2016, 17 pages.
United States Office Action, U.S. Appl. No. 14/811,545, dated Jul. 28, 2016, 17 pages.
United States Office Action, U.S. Appl. No. 14/811,554, dated Jul. 29, 2016, 17 pages.
United States Office Action, U.S. Appl. No. 14/811,570, dated Jul. 11, 2016, 17 pages.

Also Published As

Publication number Publication date
EP2953364B1 (en) 2018-07-04
EP2330826A1 (en) 2011-06-08
EP2953362A1 (en) 2015-12-09
USRE48451E1 (en) 2021-02-23
EP2953363A1 (en) 2015-12-09
EP2330826A4 (en) 2013-06-12
EP2953358A1 (en) 2015-12-09
EP2330826B1 (en) 2018-12-05
EP2953361A1 (en) 2015-12-09
US20110135002A1 (en) 2011-06-09
EP2953359A1 (en) 2015-12-09
US8705609B2 (en) 2014-04-22
EP2953365A1 (en) 2015-12-09
EP2953364A1 (en) 2015-12-09
EP2953363B1 (en) 2018-07-04
KR20100019717A (en) 2010-02-19
EP2953362B1 (en) 2018-07-11
EP2953360A1 (en) 2015-12-09
CN102150431A (en) 2011-08-10
EP2947883A1 (en) 2015-11-25
WO2010018916A1 (en) 2010-02-18
KR101085963B1 (en) 2011-11-22
EP2953365B1 (en) 2018-07-04

Similar Documents

Publication Publication Date Title
US10051273B2 (en) Video decoder and video decoding method
US20160080769A1 (en) Encoding system using motion estimation and encoding method using motion estimation
USRE47004E1 (en) Moving image coding device and method

Legal Events

Date Code Title Description
MAFP Maintenance fee payment

Free format text: PAYMENT OF MAINTENANCE FEE, 8TH YEAR, LARGE ENTITY (ORIGINAL EVENT CODE: M1552); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

Year of fee payment: 8