JP4564599B2 - Method and apparatus for improved video coding using a zero block predictor module - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates generally to image processing, and more particularly to a method and system for improved encoding of video data.
[0002]
[Prior art]
Image compression and decompression is used in a wide range of applications including video conferencing systems, video telephony and movie transmission. The traditional approach in these applications has been to use dedicated hardware for video coding or image compression and decompression. Video coding processes are computationally expensive and slow for general purpose or general purpose hardware, requiring the use of dedicated hardware, typically a digital signal processor (DSP). is there.
As a result, the widespread use of these applications has been hampered by the costs associated with effectively using the special hardware needed to achieve good performance. However, these applications, especially video conferencing, are expected to become a necessity for the desktop in the next few years. Improvements to the video coder and video coding process are necessary to realize this expectation.
[0003]
International Telecommunications Union (ITU) standard for video coding and encoding at very low bit rates, such as 28 Kbits per second. A diagram illustrating a video coding process according to H.263 (hereinafter referred to as ITU / H.263 standard) is shown in FIG.
[0004]
The video encoder shown in FIG. 1 includes a color conversion module 12, a motion detector module 14, a motion compensation module 16, a conversion module 18, a quantization module 20, and a coding module 22. A feedback module 24 is also included, which includes an inverse quantization module 30, an inverse transform module 28, and a frame reconstruction module 26.
[0005]
The video decoder as shown in FIG. 2 performs the reverse process of the video coder, and includes a bitstream decoding module 40, an inverse quantization module 30, an inverse transform module 28, an inverse motion compensation module 42, and frame reconstruction. Module 26.
[0006]
In the video coding process, the motion compensation performed by the motion compensation module 16 is the most time consuming operation stage. The transformation and quantization operation steps performed by the transformation module 18 and the quantization module 20, respectively, are also operation steps that are expensive to perform.
[0007]
However, applications that require sufficient video processing to get good performance and quality and make image processing a necessity in a desktop computer environment, given the fact that the processor speed doubles every two years. It is possible to solve only the software to reduce the cost associated with the software.
[0008]
In order to overcome the computational requirements of the various stages, the video processing application of the prior art system requires a dedicated DSP to perform the expensive stages in performing the various calculations faster. Use. Using dedicated hardware weakens current video conferencing systems. Designing new hardware as video coding standards change and evolve is expensive and time consuming, and further substantially increases the cost of the resulting system. The large costs associated with dedicated hardware not only provide a barrier to image processing applications that are a necessity for desktops, but they also run counter to the recent trend of hardware / software solutions that use open systems. Also become.
[0009]
[Problems to be solved by the invention]
Therefore, there is a need for a method and system that overcomes the limitations and weaknesses of configuring current video processing applications. In particular, a method and system for video encoding that is computationally more efficient than that of the prior art and enables a low-cost general purpose or general-purpose DSP configuration or software-only solution Need.
[0010]
[Means for Solving the Problems]
The present invention includes a video coding process that can be configured with a low cost general purpose DSP or as a software-only solution with a general purpose microprocessor to obtain acceptable performance. The present invention modifies the overall video coding process and increases its optimization capability to reduce the overall video coding computation time so that a low cost DSP solution or a software only solution can be used. To do. Both of these allow for acceptable performance on today's desktop CPUs.
[0011]
One feature of the present invention includes optimization of the motion detection stage.
Other features of the invention include optimization of the motion compensation stage.
Yet another feature of the present invention includes adding a zero block prediction step.
These and other features of the present invention will become apparent to those skilled in the art from the following detailed description of the invention, with reference to the accompanying drawings.
[0012]
DETAILED DESCRIPTION OF THE INVENTION
The video coder according to the present invention as shown in FIG. 3 includes several improvements over the prior art video coder and video coding process shown in FIG. Improvements in the video coder according to the invention as shown in FIG. 3 give improved performance over prior art video coders and video coding processes. The video coder and video coding process according to the present invention are considered useful in any application using video coding techniques.
[0013]
The present invention consists of a C ++ using a Microsoft Visual C ++ compiler on a workstation containing a Pentium processor. However, other languages and hardware configurations will be apparent to those skilled in the art.
[0014]
The overall structure of a video coder according to the present invention is shown in FIG. As mentioned above, the motion compensation and conversion stages are particularly time consuming stages in a typical video coding process. The present invention provides an improvement to the motion detection stage as performed by the modified motion detector module 50. This motion detector module 50 according to the present invention determines whether the motion compensation stage performed by the modified motion compensation module 52 should be bypassed. This modified motion compensation module 52 also includes improvements to the motion compensation stage over that of the prior art motion compensation module 16.
[0015]
The video coding process according to the present invention also includes adding a zero block prediction stage performed by the zero block prediction module 54. This zero block predictor module 54 determines whether to bypass the transform and quantization steps. Each of these features is described in detail below.
[0016]
In addition to the embodiment of the present invention shown in FIG. 3, it is contemplated that other embodiments are possible that also provide improvements to the prior art video coder. One other embodiment of a video coder according to the present invention is shown in FIG. 4 and includes a modified motion compensation module 52. FIG. 5 illustrates yet another embodiment of a video coder according to the present invention, which includes a modified motion detector module 50 and a zero block predictor module 54.
[0017]
The operation of the video encoder shown in FIG. 3 is shown in FIG. At block 60, the color conversion module 12 generates a data signal from the video image signal input at 51. This data signal is then input to a modified motion detector module 50 as indicated by block 62, which determines whether motion has been detected in a given image. Once the motion is determined at decision block 64, the modified motion compensation module 52 compensates for the motion, as indicated by block 66, and operation continues at block 68. If the motion is not determined at decision block 64, the motion compensation phase performed by the modified motion compensation module 52 is bypassed and operation at block 68 continues.
[0018]
At block 68, the zero block predictor module 54 is currently processing the data signal after the transform and quantization steps made by the transform module 18 and the quantization module 20, respectively. Macro block Is zero Macro block Determine whether to occur. At decision block 70, the zero value Macro block Is predicted, the transformation and quantization stage is bypassed and operation at block 78 continues.
[0019]
At decision block 70, the zero value Macro block If not predicted, operation continues at block 72 where the data signal is converted by the conversion module 18. In the present invention, the conversion module 18 performs individual cosine conversion, but it is conceivable that other conversions may be performed. The transformed data signal is then quantized by the quantization module 20 at block 74. At block 76, the data signal of the currently processed image is used to generate an interpolated image, which is fed back to the modified motion compensation module 52 at block 66.
[0020]
The quantized data signal generated at block 74 is encoded at block 78 for further processing by applications incorporating or using the video coder of the present invention.
[0021]
In the motion detection stage performed by the modified motion detection module 50 (this operation is shown in detail in FIG. 7), the currently processed image Macro block Are classified as either moving or stationary. these Macro block Modified motion detection module 50 to classify Macro block Determines whether there has been movement based on a predetermined movement criterion. The modified motion detector block 50 is then currently processing using the predetermined motion criteria. Macro block And in the same position of the previous image Macro block And is currently processing Macro block An interpolation image is generated by the feedback module 24 when is in the current image.
[0022]
Of the currently processed image Macro block And the same position in the previous image Macro block The pixel-to-pixel absolute difference between is calculated. If the difference between the pixel of the image currently being processed and the pixel at the same position in the previous image is greater than or equal to a predetermined threshold, the pixel Macro block Are classified as moving. If the number of pixels classified as having motion is greater than or equal to the second predetermined threshold, Macro block Is classified as moving, otherwise Macro block Are classified as stationary. Macro block If is stationary, the motion compensation phase is completely skipped and the calculation as shown in FIG. 6 continues.
[0023]
Returning to FIG. 7, at decision block 110, the modified motion detector module 50 stores the first frame of data representing the current image. Macro block (Ie if it is an INTRA coded block) Macro block Is classified as stationary at block 134 and processing by the modified detector module 50 ends. At decision block 110, the current Macro block The first being processed Macro block If not, operation at block 112 continues such that the counter and maximum data signal indicator are started. Then currently processing Macro block Operation continues at block 114 such that the first pixel of the second pixel is replenished.
[0024]
Currently processing at decision block 116 Macro block If the signal value of the current pixel at is greater than the current value of the maximum data signal indicator, the value of the maximum data signal indicator is replaced with that signal value at block 118 and the operation at block 120 Followed. If, at decision block 116, the current pixel signal value is not greater than the current value of the maximum data signal indicator, the operation at block 120 continues and the modified motion detector module 50 Determine the difference between the signal value of the current pixel and the signal value of the pixel at the same position in the previous image.
[0025]
If at decision block 122 the difference is greater than the first predetermined threshold, the counter is advanced at block 124 and operation at decision block 126 continues. If it is determined at decision block 122 that the difference is not greater than the first threshold, the operation at decision block 126 continues and is currently being processed. Macro block If there are more pixels in the block, the next pixel is replenished at block 128 and the operation at decision block 116 continues. At decision block 126, the current Macro block Once all of the pixels have been processed, a determination is made at decision block 130 as to whether the counter is greater than or equal to the second predetermined threshold.
[0026]
At decision block 130, if the counter is greater than or equal to the second predetermined threshold, Macro block Are classified as having motion at block 132. At decision block 130, if the counter is not greater than the second predetermined threshold, Macro block Are classified as stationary at block 134. Then, the operation of the modified motion detector module 50 stops.
[0027]
Returning to FIG. 6, as indicated by decision block 64, the modified motion detector module 50 is Macro block Is classified as having motion, the motion compensation stage performed by the modified motion compensation module 52 is not bypassed. This modified motion compensation module 52 operates as shown in FIG.
[0028]
Both the prior art video coder motion compensation module 16 and the modified motion compensation module 52 according to the present invention both provide optimal alignment of previous images. Macro block Use a search procedure to find the best match of the interpolated ones given together Macro block A half-pel search is performed around. When the full search block matching procedure is used in the motion compensation module 16 of the prior art video coder, the motion compensation operation stage is orders of magnitude more expensive than the other operation stages of the video coding process (additional and multiples). ) Therefore, to reduce the complexity of this motion compensation stage, the modified motion compensation module of the present invention includes several improvements.
[0029]
As indicated by block 80 in FIG. 8, the modified motion compensation module 52 is currently processing the first time. Macro block From the previous image in the same position as the previous image Macro block Replenish. This modified motion compensation module 52 is then replenished at block 82 from the previous image. Macro block And currently processing the current image Macro block Determine the sum of absolute differences between and.
[0030]
At decision block 84, the currently processed image Macro block And the same position in the previous image Macro block If the sum of the absolute difference between is considered to be below a certain threshold, no further search is performed and the previous image is replenished Macro block Processing continues at block 92 such that is selected as the optimal matching block.
[0031]
In decision block 84, if the sum is greater than or equal to a predetermined threshold, the previous image is replenished. Macro block The process of block 86 continues such that eight neighboring points around the center of are determined. The modified motion compensation module 52 is then currently processing at block 88. Macro block And each of the previous images centered around one of the eight neighboring points Macro block Determine the sum of absolute differences between pixels.
[0032]
This centrally biased orthogonal search algorithm (C-OSA) for block matching used by the motion compensation module 52 of the present invention was published in IEEE Bulletin ICASSSP, 1987, 25.4.1-25.4.4. A. It is an optimization of the orthogonal search algorithm (OSA) described in the literature entitled “Effective Block Matching Algorithm for Motion Compensated Coding” by Puri et al. The C-OSA used in the modified motion compensation module 52 in the present invention is supplemented with the previous image. Macro block Perform a quick search on a set of eight neighboring points (one pixel + or − along the diagonal in each direction) centered around the search point at.
[0033]
In the central bias search procedure used in the modified motion compensation module 52 of the present invention, if at decision block 90 the best match occurs in the middle, no further search is performed and the best match. Centered around the point Macro block Processing continues at block 92 such that is selected. However, if it is determined at decision block 90 that the best match occurred at one of eight neighboring points, processing at decision block 94 continues.
[0034]
At decision block 94, the best match is an orthogonal point {not a point that is a + or -p pixel in either direction from the center point (p is equal to 7), ie, one of the points along the diagonal} If so, the search procedure performed by the modified motion compensation module of the present invention continues as in the case of the OSA procedure described in Puri et al. However, if it is determined at decision block 94 that the best match occurs at one other than one of the eight neighboring points, a new set of eight neighboring points is selected as having the best matching point as the center point. Is done. The best match among this new set of eight neighboring points is returned to block 100 as a result. Note that in a new set of eight neighboring points, not all eight points need be recalculated. Some of them have already been calculated before.
[0035]
Note that the technique performed by this modified motion compensation module 52 is typically suitable for image motion compensation, such as when the motion between successive images is relatively small, such as video conferencing applications. I want to be. The C-OSA procedure used by the modified motion compensation module 52 of the present invention includes a quick test to find the optimal matching block in a very small search area. In the worst case, the modified motion compensation module 52 of the present invention performs a complete orthogonal search that is less complex and yields good results. The complexity of the C-OSA used in the modified motion compensation module 52 of the present invention is 16 + 4 log 2w (where w is the pixel search distance, typically plus or minus 15 pixels).
[0036]
Finally, as indicated by block 102, the modified motion compensation module 52 of the present invention provides a half-pel motion compensation search with the OSA procedure described in the above document by Puri et al. This half-pel search is performed on the interpolated image in the plus or minus one pixel search area in all directions and along the diagonal. Surprisingly, this required a time that exceeded a substantial portion of the total motion compensation time used by the modified motion compensation module 52 because the full pel search described above was already quite effective. Applying the OSA procedure described in the above document by Puri et al. To a half-pel search significantly reduces the number of search points. Also, the search area for boundary blocks is smaller because some of them are not within the image limits, which also saves computation time.
[0037]
The results obtained from the C-OSA and OSA procedures given to the full pel and half pel search, respectively, by the modified motion compensation module 52 closely approach the exhaustive search without noticeable degradation in image quality, compared to the exhaustive search. The result of speeding up motion compensation by orders of magnitude was obtained. Several ITU / H.B. Images containing “Miss America” images. For the H.263 reference test image, motion compensation is done in less than 12% of the total computation time with the signal-to-noise ratio decreasing to less than 0.1 dB.
[0038]
Returning to FIG. 6, in a typical image sequence, for example, in an ITU test image sequence, no more than 30% of INTER coded blocks had non-zero values after the quantization stage at 74, but all blocks had 72 There was no need to go through an expensive conversion step. Furthermore, the non-zero value block has only one or two non-zero coefficients. To take advantage of this result, a zero block prediction stage at 70 performed by the zero block predictor module 54 is included after the motion compensation stage at 66.
[0039]
The modified motion detector module 50 as described above is Macro block Are classified as either moving or stationary. The modified motion detector module 50 is also capable of generating maximum signal values and overall Macro block Record the number of still pixels in. Based on the quantized values that change as the process proceeds, this information is converted after the transformation and quantization stages. Macro block Is used by the zero block predictor module 50 to determine or predict whether or not all have zero values. Once a zero value block is predicted, the transform and quantization steps at 72 and 74, respectively, are skipped and a zero value block is generated.
[0040]
The operation of the zero block predictor module 54 is illustrated in FIG. At block 140, the prediction is set to “none”. In decision block 142, in the first frame Macro block Is processed, the processing in the zero block predictor module 54 ends. Otherwise, at block 144, the zero block predictor module 54 Macro block Gives a heuristic using the maximum signal value, the number of still pixels, and the current quantized value to determine whether to generate a zero-value block after the transform and quantization steps. When a block of zero values is predicted at decision block 146, the prediction is set to “Yes” at block 148, and processing at the zero block predictor module 54 ends.
[0041]
The heuristics used by the zero block predictor module 54 are only given to the INTER coding block. On average, this saves about 30% of the blocks from going through the transform and quantization stages. However, because the heuristics used by the zero block predictor module 54 are conservative, 40% of the blocks that would have had zero value blocks have gone through expensive transformation and quantization steps. become. However, it is contemplated that the portion of the block predicted by the zero block predictor module 54 as a zero value block may be increased as the heuristics used are improved.
[0042]
Before this optimization takes place, the transformation and quantization stage consumes about 30% of the total processing time. This was reduced to about 18% after the above optimization was given. It is believed that this processing time can be further reduced given an improved heuristic that predicts zero value blocks. The overall video coding process of the present invention is described in ITU H.264. The H.263 standard is increased about 5-6 times over the video coding process. Thus, the video coding process according to the present invention is computationally more efficient and can be configured using a low cost DSP or software-only solution.
[0043]
The described techniques and concepts in the present invention have been developed specifically for the video coding process used by video conferencing systems. However, it is believed that the present invention is equally applicable to any application that uses video coding techniques for image compression.
[0044]
Although the invention and its advantages have been described in detail, it should be understood that various changes, substitutions and alterations can be made without departing from the spirit and scope of the invention as defined by the claims. is there.
[0045]
The following items are further disclosed with respect to the above description.
(1) In the video encoder,
A color conversion module that accepts an input video signal and generates a first set of data signals representing a current image including a plurality of pixels;
A motion detector module coupled to the color conversion module and generating a classification signal for classifying the current image as moving or stationary according to the first set of data signals;
In response to the classification signal generated by the motion detector module and a second set of data signals representing the previous image generated by a feedback module, the current image motion is determined using a center bias orthogonal search technique. A modified motion compensation module to compensate; and
A calculation module for generating a digitized data signal from the first set of data signals in response to the modified motion compensation module;
The feedback module for generating the second set of data signals for input to the motion compensation module in response to the digitized data signal;
A coding module for generating an encoded data signal in response to the digitized data signal;
A video encoder comprising:
[0046]
(2) In the video encoder,
A color conversion module that accepts an input video signal and generates a first set of data signals representing a current image including a plurality of pixels;
A classification signal coupled to the color conversion module for classifying the current image as moving or stationary according to the first set of data signals; a maximum value signal; and a stationary signal A motion detector module that generates a counter signal indicating how many of the pixels classified as
A modified motion compensation module that compensates for motion of the current image in response to the classification signal generated by the motion detector module and a second set of data signals representing the previous image generated by a feedback module And, according to the first set of data signals, the maximum value signal, and the counter signal, a zero block signal that indicates whether the calculation for the first set of data signals generates non-zero data The generated zero block predictor module; and
A calculation module for generating a digitized data signal from the first set of data signals in response to the zero block signal;
The feedback module for generating the second set of data signals for input to the motion compensation module in response to the digitized data signal;
A coding module for generating an encoded data signal in response to the digitized data signal;
A video encoder comprising:
[0047]
(3) In the video encoder,
A color conversion module that accepts an input video signal and generates a first set of data signals representing a current image including a plurality of pixels;
A classification signal coupled to the color conversion module for classifying the current image as moving or stationary according to the first set of data signals; a maximum value signal; and a stationary signal A modified motion detector module that generates a counter signal indicating how many of the pixels classified as
In response to the classification signal generated by the modified motion detector module and a second set of data signals representing the previous image generated by a feedback module, the current image using a center bias orthogonal search technique A modified motion compensation module that compensates for the motion of
In response to the first set of data signals, the maximum value signal, and the counter signal, a zero block signal is generated that indicates whether the calculation for the first set of data signals generates non-zero data. A zero block predictor module;
A calculation module for generating a digitized data signal from the first set of data signals in response to the zero block signal;
The feedback module for generating the second set of data signals for input to the motion compensation module in response to the compressed data signals;
A coding module for generating an encoded data signal in response to the digitized data signal;
A video encoder comprising:
[0048]
(4) The video encoder according to item 3, wherein the changed motion compensation module is operable to perform a full pel search according to the central bias orthogonal search technique.
(5) The video encoder according to item 3, wherein the changed motion compensation module is operable to perform a half-pel search according to an orthogonal search technique.
(6) The present invention includes a video encoder that is optimized to be configured as software that can be executed using a general purpose DSP or in a general purpose microprocessor. The present invention includes a modified motion detector module (50) that classifies the currently processed block of images as moving or stationary. The blocks classified as having motion are then processed by a modified motion compensation module (52) that uses a central bias orthogonal search procedure to align the blocks with previously processed images. The zero block detector module (54) determines whether the block generates a zero-valued block after being processed by the transform module (18) and the quantization module (20). If a zero-valued block is predicted, the transform module (18) and quantization module (20) are bypassed.
[Brief description of the drawings]
FIG. 1 is a block diagram illustrating a prior art video encoder.
FIG. 2 is a block diagram illustrating a prior art video decoder.
FIG. 3 shows a first embodiment of a video encoder according to the invention.
FIG. 4 shows a second embodiment of a video encoder according to the invention.
FIG. 5 shows a third embodiment of a video encoder according to the invention.
FIG. 6 is a flowchart showing the operation of the video encoder according to the present invention.
FIG. 7 is a flow diagram illustrating the operation of a modified motion detector module according to the present invention.
FIG. 8 is a flowchart illustrating the operation of a modified motion compensation module according to the present invention.
FIG. 9 is a block diagram illustrating the operation of a zero block predictor module according to the present invention.
[Explanation of symbols]
50 Modified motion detector module
52 Modified motion compensation module
54 Zero Block Predictor Module

Claims (5)

In video encoder,
A color conversion module that accepts an input video signal and generates a first set of data signals, such as macroblocks, representing a current image including a plurality of pixels;
A classification signal that is coupled to the color conversion module and classifies the current image as moving or stationary according to the first set of data signals, and the pixel signal value is large A modified motion detector module that generates a maximum value signal indicator that is replaced with a maximum signal value when being done, and a counter that indicates how many of the plurality of pixels have been classified as stationary ;
In response to a second set of data signals representing an image before the upper Symbol Ru generated by the classification signal and the feedback module generated by the modified motion detector module, is changed to compensate for the motion of the current image A motion compensation module,
Upper Symbol maximum value signal indicator, and in response to the counter, the zero block predictor module whether calculations for the first set of data signals to generate a zero value data to generate the finger Shimesuru zero block signal When,
A calculation module for generating a digitized data signal from the first set of data signals in response to the zero block signal;
The feedback module for generating the second set of data signals for input to the motion compensation module in response to the digitized data signal;
A coding module for generating an encoded data signal in response to the digitized data signal;
A video encoder comprising:   The video encoder of claim 1, wherein the modified motion compensation module is operable to perform motion compensation of a current image in accordance with the central bias orthogonal search technique.   The video encoder of claim 2, wherein the modified motion compensation module is operable to perform a full pixel search according to the central bias orthogonal search technique.   The video encoder of claim 2, wherein the modified motion compensation module is operable to perform a half-pixel search according to an orthogonal search technique. The video encoder of claim 2, wherein the modified motion compensation module is
Means for calculating an absolute difference between a data signal from a set of neighboring points in each direction around a point to be searched in a previous macroblock in the previous image and a macroblock of the current data signal; If the optimum match is obtained at the center point, the center point is selected as the optimum point, and if the optimum point is not obtained, it is separated from the neighboring points in the direction not along the diagonal line. Whether or not to match at an orthogonal point at a certain distance, and if optimal matching is performed at the orthogonal point, the optimal matching is selected according to the orthogonal search technique, and optimal when matching is optimal near the orthogonal point. A video encoder comprising the calculating means for selecting a point near a matching point as the center point.

Images