WO2004025964A2 - Manipulation of video compression - Google Patents

Manipulation of video compression Download PDF

Info

Publication number
WO2004025964A2
WO2004025964A2 PCT/GB2003/003991 GB0303991W WO2004025964A2 WO 2004025964 A2 WO2004025964 A2 WO 2004025964A2 GB 0303991 W GB0303991 W GB 0303991W WO 2004025964 A2 WO2004025964 A2 WO 2004025964A2
Authority
WO
WIPO (PCT)
Prior art keywords
pictures
sequence
quality
picture
coding
Prior art date
Application number
PCT/GB2003/003991
Other languages
French (fr)
Other versions
WO2004025964A3 (en
Inventor
Violet Snell
Michael James Knee
Original Assignee
Snell & Wilcox Limited
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Snell & Wilcox Limited filed Critical Snell & Wilcox Limited
Priority to DE10393211T priority Critical patent/DE10393211T5/en
Priority to AU2003269142A priority patent/AU2003269142A1/en
Priority to GB0504586A priority patent/GB2410852B/en
Publication of WO2004025964A2 publication Critical patent/WO2004025964A2/en
Publication of WO2004025964A3 publication Critical patent/WO2004025964A3/en

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/503—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal prediction
    • H04N19/51—Motion estimation or motion compensation
    • H04N19/577—Motion compensation with bidirectional frame interpolation, i.e. using B-pictures
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/103—Selection of coding mode or of prediction mode
    • H04N19/105—Selection of the reference unit for prediction within a chosen coding or prediction mode, e.g. adaptive choice of position and number of pixels used for prediction
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/102—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or selection affected or controlled by the adaptive coding
    • H04N19/124—Quantisation
    • H04N19/126—Details of normalisation or weighting functions, e.g. normalisation matrices or variable uniform quantisers
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/15—Data rate or code amount at the encoder output by monitoring actual compressed data size at the memory before deciding storage at the transmission buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/134—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the element, parameter or criterion affecting or controlling the adaptive coding
    • H04N19/146—Data rate or code amount at the encoder output
    • H04N19/152—Data rate or code amount at the encoder output by measuring the fullness of the transmission buffer
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/10—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding
    • H04N19/169—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding
    • H04N19/17—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object
    • H04N19/172—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/30—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability
    • H04N19/31—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/50—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding
    • H04N19/587—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using predictive coding involving temporal sub-sampling or interpolation, e.g. decimation or subsequent interpolation of pictures in a video sequence
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N19/00—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals
    • H04N19/60—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding
    • H04N19/61—Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using transform coding in combination with predictive coding

Definitions

  • This invention is directed to the compression of picture sequences, and in particular aspects to the manipulation of compression techniques in controlling the relative quality of pictures of an image sequence.
  • the invention consists in one aspect in a system for compressing a picture sequence, comprising: means for compressing the picture sequence to give a compressed sequence wherein a first set of pictures have a high coding quality, and wherein a second set of pictures, constituent pictures of which are intermediate pictures of the first set, have a lower coding quality; and downstream, means for separating the first set of pictures from the sequence, and playing the first set of pictures as a separate high quality picture sequence.
  • the invention consists in a method of compressing a picture sequence in which a first set of pictures occurs in a regular pattern in the sequence, and pictures of a second set of pictures occur intermediate pictures of the first set of pictures, comprising coding the first set of pictures to a high coding quality, and coding the second set of pictures to quality determined by remaining bit-rate resources.
  • the first set of pictures are coded independently of said second intermediate set of pictures, with all pictures preferably being intra coded. In this way the sequence is compressed with very little overall loss of quality.
  • the compression algorithm is MPEG based.
  • the bit-rate of the second intermediate set of pictures is controlled by varying quantizer weighting matrix values.
  • said quantizer weighting matrix values are a linear combination of the values of a flat weighting matrix and a default, non-flat matrix.
  • a fixed number of bits are generated for the period covered by one picture of the first set of pictures and the associated intermediate pictures.
  • the invention provides a method of video signal compression, comprising the steps of receiving a video signal, performing a transform on the video signal, and quantizing the transformed signal, wherein the step of quantizing comprises applying a variable quantization matrix, the variation of which matrix controlled by a parameter relating to the compression bit rate.
  • the parameter is dependent upon the number of bits required for a current picture.
  • the method comprises determining the number of bits required for a current picture according to a regular pattern.
  • the invention provides a compressed signal comprising a first sequence of high quality pictures interleaved with a second sequence of lower quality pictures, the first sequence being separable from the signal to form a video signal, the second sequence of pictures being for use in editing or further processing of the video signal.
  • Figure 1 is a diagram illustrating apparatus according to an embodiment of one aspect of the invention
  • Figure 2 to 6 illustrate picture sequences which can be derived from an input sequence according to aspects of the invention
  • Figure 7 is a diagram illustrating the function of a method according to an embodiment of another aspect of the invention.
  • Figure 1 shows a high definition 72 Hz image capture device 102 which provides an input video signal to a compression stage 104.
  • the video signal is compressed according to a control signal 106 which identifies key frames that are to be coded to a higher quality than that of other frames.
  • key frames are identified every third frame.
  • the compressed output is then transmitted to be received by a decompression stage 108, and subsequently a further processing stage 110. These act to select and decompress only the key frames and output these frames as a 24 Hz high definition video signal.
  • Stages 108 and 110 can optionally output an additional 72Hz signal as will be described in relation to Figures 2 to 6.
  • the particular example of use of the invention described hereafter concerns the capture of high-frame-rate, high-definition image sequences in an environment where there are limitations in sampling, storage or transmission bandwidth or in processing complexity. It describes a method of compressing high-frame-rate, high-definition image sequences with very little loss of quality. The method ensures that the highest possible quality is retained for a temporally subsampled version of the input sequence, while allowing some loss of quality for intermediate frames according to the available coded bit-rate.
  • the invention also concerns the use of picture signals having a high quality at a low frame rate together with those having a lower quality at a higher frame rate, as a means for reducing the overall sampling, storage or transmission bandwidth of the picture information, as will be described.
  • the different quality of the two kinds of image is expressed in terms of resolution.
  • a high-definition 25 Hz sequence is multiplexed with a standard-definition 100 Hz sequence.
  • Motion information may be derived from both sequences and used for motion- compensated conversion of the high-definition sequence to the higher frame rate or indeed to any other frame rate.
  • FIG. 2 illustrates one embodiment of the invention in use, in which we begin with a high-definition (e.g. 1920 pixels by 1080 lines) picture signal with a frame rate of 72 Hz and with a 4:2:2 luminance and colour-difference sampling structure (represented by input sequence of frames 202) as used in the most common digital television sampling standards such as SMPTE 274M.
  • a high-definition e.g. 1920 pixels by 1080 lines
  • FIG. 1 illustrates one embodiment of the invention in use, in which we begin with a high-definition (e.g. 1920 pixels by 1080 lines) picture signal with a frame rate of 72 Hz and with a 4:2:2 luminance and colour-difference sampling structure (represented by input sequence of frames 202) as used in the most common digital television sampling standards such as SMPTE 274M.
  • bit rate of such a signal is about 2.4 Gbit s.
  • bit-rate limit of, say, 500 Mbit/s.
  • -A low-frame-rate (say 24 Hz) version of the input signal may be independently decoded with zero or negligible loss of quality
  • the remaining input frames are transmitted with some quality loss, which should nevertheless be minimized.
  • the decoding of these frames may be allowed to depend upon that of the 24 Hz frames, because subsequent processing of those frames may also depend on the 24 Hz frames.
  • Input sequence 202 is therefore coded to produce coded sequence 204.
  • "Key" frames (every third) are coded to a high quality as indicated by solid arrows 206 (and by the resulting coded frames having darker shading) whereas intermediate frames are coded to a lesser quality as indicated by dashed arrows 208 (and resulting lighter shaded coded frames).
  • the frames which are in between the "key” frames need not be sub-sampled - they are simply coded at a lower quality.
  • the method also preserves the key frames to that degree of quality required at the output. For example, here the output requires a 24Hz high definition signal.
  • the total bit rate assigned to the remaining, intermediate frames will depend on that required for the high quality frames, and the remaining bit rate allowed by the transmission channel used.
  • these intermediate frames are not be coded in dependence, in terms of picture content, upon the key frames, in the sense that normal P and B frames of MPEG are dependent upon prediction from nearby I frames.
  • the coded sequence preferably consists entirely of Intra coded frames (ie frames coded without reference to any other frames in the sequence), however the quality of these Intra coded frames differs. In certain cases though, it may be necessary to use prediction coding for the intermediate frames, for example where a particular bit rate constraint demands.
  • any of a number of known compression techniques could be used to perform the actual compression, once the key frames are chosen and the quality level set, for example an established compression scheme such as MPEG-2 or JPEG could be used. In this case, there would be a small but possibly significant loss of quality to all input frames, not just the intermediates.
  • a particular embodiment makes use of MPEG-2 for the compression of the source picture sequence using a novel method for ensuring that the highest possible quality is retained for the low-frame-rate signal, which in this example is every third frame.
  • This can be done by encoding this "key frame" as an l-frame with a very fine quantizer and a flat quantizer weighting matrix.
  • the remaining bits are allocated to the other two "intermediate" frames, which are also coded as l-frames.
  • the key frames are separated from the intermediate frames, to produce high quality 24Hz sequence 210.
  • FIG. 3 illustrates an alternative way of creating a 72 Hz browse track.
  • Only intermediate frames need be extracted from sequence 204, as shown by sequence 218.
  • This essentially provides the desired browse track but with "holes" at each third frame.
  • the "holes" are filled by interpolating (possibly using motion compensation) from neighbouring intermediate frames, as illustrated by arrows 220, to produce a compete browse track 222.
  • Figures 5 and 6 illustrate further advantages of the present invention.
  • Figure 5 shows a first method for producing an output sequence at a frequency which is not an integer multiple of the capture rate.
  • the output sequence 504 is derived by interpolating the input sequence 502 which has been coded according to the invention.
  • Output frames are interpolated from the frames which are temporally closest to the respective output frame as indicated by solid arrows 514. It can be seen that for some output frames this will involve one key frame and one intermediate frame (for example at frame 506) while in other cases this will be two intermediate frames (eg frame 508).
  • Output frames such as 506 will still obtain a quality benefit from one key frame (in this case frame 510). While the same is not automatically true for frames such as 508, use of the key frames can still be made, by adding texture to the output frame from key frames which are temporally close to the output frame. This is indicated by dashed arrows 512.
  • Figure 6 shows an alternative method for producing an output sequence at a frequency which is not an integer multiple of the capture rate.
  • the key frames 610 of the input which has been coded according to the invention
  • the output is interpolated from these key frames using conventional methods as illustrated by arrows 612.
  • the intermediate frames are employed, for the purpose of determining motion vectors to be used in the interpolation of the key frames.
  • the conventional way to control the bit-rate in MPEG-2 coding is to vary the quantizer scale parameter. Unfortunately, at high quality levels the control available is very coarse.
  • f be the value of the elements of a flat quantizer weighting matrix.
  • Wjj be the elements of a typical quantizer weighting matrix designed to balance the visibility of quantizing noise across the spectrum of the picture (an example being the MPEG-2 default intra matrix).
  • a be a bit-rate control parameter.
  • the variation of the control parameter implements a cross-fade between a flat matrix and a subjectively lossless one.
  • a value ⁇ - 0 corresponds to the flat matrix and ⁇ - 1 to the default matrix.
  • a value of ⁇ between 0 and 1 gives a matrix which leads to subjectively lossless coding in normal viewing conditions, while a value ⁇ > 1 may lead to some subjective loss.
  • Figure 7 illustrates how the weighting matrix may vary according to the control parameter. In the graph, a one-dimensional section of the matrix is plotted for simplicity.
  • the value of the control parameter ⁇ can be calculated using an existing model of the relationship between ⁇ and the number of bits generated.
  • the number of bits actually used can serve in calculating a correction to the model for subsequent intermediate frames.
  • the bit-rate control may be run using a first-in first-out buffer as is well known. Alternatively, the control may aim to generate a fixed number of bits for each group of three frames, so that the 24 Hz picture may be extracted more easily.
  • well-known fallback methods of bit-rate control may be employed to ensure that the bit-rate target is met and that the coding quality is acceptable both for the key frames and for the intermediate frames.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Compression Or Coding Systems Of Tv Signals (AREA)
  • Television Signal Processing For Recording (AREA)
  • Compression, Expansion, Code Conversion, And Decoders (AREA)
  • Color Television Systems (AREA)

Abstract

A method of compressing a picture sequence in which a first set of pictures in the sequence, and a second set of pictures interleaved with the first set of pictures in the sequence are identified, and in which the first set of pictures are coded to a high coding quality and the second set of pictures are coded to a quality determined by the remaining bitrate resources. Downstream the first set of pictures can be extracted to produce a separate high quality picture sequence. The compression coding is preferably MPEG based, and control of the coding quality may be achieved by varying quantizer weighting matrix values in a linear combination of a flat matrix and a default non-flat matrix.

Description

MANIPULATION OF VIDEO COMPRESSION
This invention is directed to the compression of picture sequences, and in particular aspects to the manipulation of compression techniques in controlling the relative quality of pictures of an image sequence.
Methods for video sequence compression are well known to the art, and amongst these, techniques exist for varying the coding quality.
It is an object of the present invention to provide novel and flexible techniques for manipulating the compression of image sequences.
Accordingly, the invention consists in one aspect in a system for compressing a picture sequence, comprising: means for compressing the picture sequence to give a compressed sequence wherein a first set of pictures have a high coding quality, and wherein a second set of pictures, constituent pictures of which are intermediate pictures of the first set, have a lower coding quality; and downstream, means for separating the first set of pictures from the sequence, and playing the first set of pictures as a separate high quality picture sequence. In another aspect, the invention consists in a method of compressing a picture sequence in which a first set of pictures occurs in a regular pattern in the sequence, and pictures of a second set of pictures occur intermediate pictures of the first set of pictures, comprising coding the first set of pictures to a high coding quality, and coding the second set of pictures to quality determined by remaining bit-rate resources.
Suitably, the first set of pictures are coded independently of said second intermediate set of pictures, with all pictures preferably being intra coded. In this way the sequence is compressed with very little overall loss of quality. Advantageously, the compression algorithm is MPEG based. In one form of the invention, the bit-rate of the second intermediate set of pictures is controlled by varying quantizer weighting matrix values. Suitably, said quantizer weighting matrix values are a linear combination of the values of a flat weighting matrix and a default, non-flat matrix. Advantageously, a fixed number of bits are generated for the period covered by one picture of the first set of pictures and the associated intermediate pictures.
In yet another aspect, the invention provides a method of video signal compression, comprising the steps of receiving a video signal, performing a transform on the video signal, and quantizing the transformed signal, wherein the step of quantizing comprises applying a variable quantization matrix, the variation of which matrix controlled by a parameter relating to the compression bit rate.
Preferably, the parameter is dependent upon the number of bits required for a current picture. Suitably, the method comprises determining the number of bits required for a current picture according to a regular pattern.
In still another aspect, the invention provides a compressed signal comprising a first sequence of high quality pictures interleaved with a second sequence of lower quality pictures, the first sequence being separable from the signal to form a video signal, the second sequence of pictures being for use in editing or further processing of the video signal.
The invention will now be described by way of example with reference to the accompanying drawings, in which:
Figure 1 is a diagram illustrating apparatus according to an embodiment of one aspect of the invention;
Figure 2 to 6 illustrate picture sequences which can be derived from an input sequence according to aspects of the invention;
Figure 7 is a diagram illustrating the function of a method according to an embodiment of another aspect of the invention.
Figure 1 shows a high definition 72 Hz image capture device 102 which provides an input video signal to a compression stage 104. Here the video signal is compressed according to a control signal 106 which identifies key frames that are to be coded to a higher quality than that of other frames. In Figure 1 , key frames are identified every third frame. The compressed output is then transmitted to be received by a decompression stage 108, and subsequently a further processing stage 110. These act to select and decompress only the key frames and output these frames as a 24 Hz high definition video signal. Stages 108 and 110 can optionally output an additional 72Hz signal as will be described in relation to Figures 2 to 6.
The particular example of use of the invention described hereafter concerns the capture of high-frame-rate, high-definition image sequences in an environment where there are limitations in sampling, storage or transmission bandwidth or in processing complexity. It describes a method of compressing high-frame-rate, high-definition image sequences with very little loss of quality. The method ensures that the highest possible quality is retained for a temporally subsampled version of the input sequence, while allowing some loss of quality for intermediate frames according to the available coded bit-rate.
The invention also concerns the use of picture signals having a high quality at a low frame rate together with those having a lower quality at a higher frame rate, as a means for reducing the overall sampling, storage or transmission bandwidth of the picture information, as will be described. In the prior art, the different quality of the two kinds of image is expressed in terms of resolution. Thus, for example in German patent DE-A-41 32 359, a high-definition 25 Hz sequence is multiplexed with a standard-definition 100 Hz sequence. Motion information may be derived from both sequences and used for motion- compensated conversion of the high-definition sequence to the higher frame rate or indeed to any other frame rate.
The prior art does not necessarily deliver the required combination of the highest possible quality at the low frame rate together with an optimum trade-off between quality, bit-rate and processing complexity for the higher-frame-rate version of the sequence. For example, the method described in DE-A-41 32 359 relies on motion compensated up-conversion to restore the full resolution of the image at the higher frame rate. Figure 2 illustrates one embodiment of the invention in use, in which we begin with a high-definition (e.g. 1920 pixels by 1080 lines) picture signal with a frame rate of 72 Hz and with a 4:2:2 luminance and colour-difference sampling structure (represented by input sequence of frames 202) as used in the most common digital television sampling standards such as SMPTE 274M. With 8-bit sampling, the bit rate of such a signal is about 2.4 Gbit s. Let us suppose that at some point in the subsequent processing chain there is a bit-rate limit of, say, 500 Mbit/s. We require a method of compressing the signal into the output bit- rate with the following properties:
-A low-frame-rate (say 24 Hz) version of the input signal may be independently decoded with zero or negligible loss of quality
-The remaining input frames are transmitted with some quality loss, which should nevertheless be minimized. The decoding of these frames may be allowed to depend upon that of the 24 Hz frames, because subsequent processing of those frames may also depend on the 24 Hz frames.
Input sequence 202 is therefore coded to produce coded sequence 204. "Key" frames (every third) are coded to a high quality as indicated by solid arrows 206 (and by the resulting coded frames having darker shading) whereas intermediate frames are coded to a lesser quality as indicated by dashed arrows 208 (and resulting lighter shaded coded frames). Advantageously, the frames which are in between the "key" frames need not be sub-sampled - they are simply coded at a lower quality. The method also preserves the key frames to that degree of quality required at the output. For example, here the output requires a 24Hz high definition signal. Typically, the total bit rate assigned to the remaining, intermediate frames will depend on that required for the high quality frames, and the remaining bit rate allowed by the transmission channel used. Preferably, these intermediate frames are not be coded in dependence, in terms of picture content, upon the key frames, in the sense that normal P and B frames of MPEG are dependent upon prediction from nearby I frames. In this way the coded sequence preferably consists entirely of Intra coded frames (ie frames coded without reference to any other frames in the sequence), however the quality of these Intra coded frames differs. In certain cases though, it may be necessary to use prediction coding for the intermediate frames, for example where a particular bit rate constraint demands.
Any of a number of known compression techniques could be used to perform the actual compression, once the key frames are chosen and the quality level set, for example an established compression scheme such as MPEG-2 or JPEG could be used. In this case, there would be a small but possibly significant loss of quality to all input frames, not just the intermediates.
A particular embodiment makes use of MPEG-2 for the compression of the source picture sequence using a novel method for ensuring that the highest possible quality is retained for the low-frame-rate signal, which in this example is every third frame. This can be done by encoding this "key frame" as an l-frame with a very fine quantizer and a flat quantizer weighting matrix. The remaining bits are allocated to the other two "intermediate" frames, which are also coded as l-frames. As mentioned above however, in certain cases it may be necessary to code these frames as B-frames with or without motion compensation. Returning to Figure 2, downstream of the coding process, the key frames are separated from the intermediate frames, to produce high quality 24Hz sequence 210.
Turning now to figures 3 and 4, different methods of producing a 72 Hz browse track to accompany the key frame sequence are described. In Figure 3, intermediate frames are decoded from sequence 204 in a standard fashion as indicated by the solid arrows 212, while the key frames are decoded and downconverted in an appropriate fashion, as indicated by dashed arrows 214, to provide every third frame in the browse track 216. Alternatively Figure 4 illustrates an alternative way of creating a 72 Hz browse track. Here, only intermediate frames need be extracted from sequence 204, as shown by sequence 218. This essentially provides the desired browse track but with "holes" at each third frame. In this particular example the "holes" are filled by interpolating (possibly using motion compensation) from neighbouring intermediate frames, as illustrated by arrows 220, to produce a compete browse track 222. Figures 5 and 6 illustrate further advantages of the present invention.
Figure 5 shows a first method for producing an output sequence at a frequency which is not an integer multiple of the capture rate. Here the output sequence 504 is derived by interpolating the input sequence 502 which has been coded according to the invention. Output frames are interpolated from the frames which are temporally closest to the respective output frame as indicated by solid arrows 514. It can be seen that for some output frames this will involve one key frame and one intermediate frame (for example at frame 506) while in other cases this will be two intermediate frames (eg frame 508). Output frames such as 506 will still obtain a quality benefit from one key frame (in this case frame 510). While the same is not automatically true for frames such as 508, use of the key frames can still be made, by adding texture to the output frame from key frames which are temporally close to the output frame. This is indicated by dashed arrows 512.
Figure 6 shows an alternative method for producing an output sequence at a frequency which is not an integer multiple of the capture rate. Here only the key frames 610 of the input (which has been coded according to the invention) are used and the output is interpolated from these key frames using conventional methods as illustrated by arrows 612. In a variation on the method illustrated in Figure 6, the intermediate frames are employed, for the purpose of determining motion vectors to be used in the interpolation of the key frames. In many circumstances it will be necessary to reduce the coding quality of these intermediate frames in order to meet the bit-rate target. The conventional way to control the bit-rate in MPEG-2 coding is to vary the quantizer scale parameter. Unfortunately, at high quality levels the control available is very coarse. For example, changing the quantizer scale from 1 to 2 will quadruple the quantizing noise power and will typically reduce the number of bits generated for a given area of the picture by 40%. In the present invention, finer control of the bit rate and coding quality is achieved by modifying the quantizer weighting matrix. There now follows an example of how such a modified matrix may be derived, in the case of intra-frame coding:
Let f be the value of the elements of a flat quantizer weighting matrix. Let Wjj be the elements of a typical quantizer weighting matrix designed to balance the visibility of quantizing noise across the spectrum of the picture (an example being the MPEG-2 default intra matrix). Let a be a bit-rate control parameter.
Then the values ι//7 of the matrix actually used for the frame being coded are given by uij = { - )f + αwij
Thus, the variation of the control parameter implements a cross-fade between a flat matrix and a subjectively lossless one. A value α - 0 corresponds to the flat matrix and α - 1 to the default matrix. A value of α between 0 and 1 gives a matrix which leads to subjectively lossless coding in normal viewing conditions, while a value α > 1 may lead to some subjective loss. Figure 7 illustrates how the weighting matrix may vary according to the control parameter. In the graph, a one-dimensional section of the matrix is plotted for simplicity.
For a given intermediate frame, the value of the control parameter α can be calculated using an existing model of the relationship between α and the number of bits generated. The number of bits actually used can serve in calculating a correction to the model for subsequent intermediate frames. The bit-rate control may be run using a first-in first-out buffer as is well known. Alternatively, the control may aim to generate a fixed number of bits for each group of three frames, so that the 24 Hz picture may be extracted more easily. For both the key frames and the intermediate frames, well-known fallback methods of bit-rate control may be employed to ensure that the bit-rate target is met and that the coding quality is acceptable both for the key frames and for the intermediate frames.
It will be appreciated by those skilled in the art that the invention has been described by way of example only, and that a wide variety of alternative approaches may be adopted.

Claims

1. A system for compressing a picture sequence, comprising: means for compressing the picture sequence to give a compressed sequence wherein a first set of pictures have a high coding quality, and wherein a second set of pictures, constituent pictures of which are intermediate pictures of the first set, have a lower coding quality; and downstream, means for separating the first set of pictures from the sequence, and playing the first set of pictures as a separate high quality picture sequence.
2. A system according to Claim 1 , in which the first set of pictures is coded independently of said second intermediate set of pictures.
3. A system according to Claim 1 or Claim 2, in which the compression algorithm used is MPEG based.
4. A system according to any of the Claims 1 to 3, in which the quality of coding of the second intermediate set of pictures is determined by remaining bit-rate resources.
5. A system according to Claim 4, in which the bit-rate of the second intermediate set of pictures is controlled by varying quantizer weighting matrix values.
6. A system according to Claim 5, in which said quantizer weighting matrix values are a linear combination of the values of a flat weighting matrix and a default, non-flat matrix.
7. A system according to any previous claim, in which a fixed number of bits are generated for the period covered by one picture of the first set of pictures and the associated intermediate pictures.
8. A system according to any previous claim, further including means for outputting a picture sequence, having a higher frame rate but of a lower quality than said separate high quality picture sequence.
9. A method of compressing a picture sequence in which a first set of pictures occurs in a regular pattern in the sequence, and pictures of a second set of pictures occur intermediate pictures of the first set of pictures, comprising coding the first set of pictures to a high coding quality, and coding the second set of pictures to a quality determined by remaining bit-rate resources.
10. A method according to Claim 9, in which the first set of pictures is coded independently of said second intermediate set of pictures.
11. A method according to Claim 9 or Claim 10, in which all of said pictures are intra coded.
12. A method according to any one of Claims 9 to 11 in which the compression algorithm is MPEG based.
13. A method according to any one of Claims 9 to 12, in which the bit- rate of the second intermediate set of pictures is controlled by varying quantizer weighting matrix values.
14. A method according to Claim 13, in which said quantizer weighting matrix values are a linear combination of the values of a flat weighting matrix and a default, non-flat matrix.
15. A method according to any one of Claims 9 to 14, in which a fixed number of bits are generated for the period covered by one picture of the first set of pictures and the associated intermediate pictures.
16. Video processing apparatus for compressing a picture sequence comprising: means for dividing the picture sequence into sets, each set containing a key picture and one or more intermediate pictures; a first coder for compressing said key pictures at a first coding quality; means for determining bit rate resources available for the intermediate pictures; and a second coder for compressing the intermediate pictures at a second coding quality dependent on the available bitrate resources, said second coding quality being lower than the first coding quality.
17. Video processing apparatus according to Claim 16, wherein said first and second coders operate independently.
18. Video processing apparatus according to Claims 16 or 17, in which all pictures are compressed using intra coding.
19. Video processing apparatus according to any one of Claims 16 to 18, wherein the remaining bit rate resources are determined such that a constant number of bits are generated for each set of pictures.
20. A method of video signal compression, comprising the steps of receiving a video signal, performing a transform on the video signal, and quantizing the transformed signal, wherein the step of quantizing comprises applying a variable quantization matrix, the variation of which matrix controlled by a parameter relating to the compression bit rate.
21. A method according to Claim 20, wherein the parameter is dependent upon the number of bits required for a current picture.
22. A method according to Claim 20 or Claim 21 , comprising determining the number of bits required for a current picture according to a regular pattern.
23. A method according to any one of claims 20 to 22, wherein said quantization matrix is a linear combination of a flat weighting matrix and a default, non-flat matrix.
24. A compressed signal comprising a first sequence of high quality pictures interleaved with a second sequence of lower quality pictures, the first sequence being separable from the signal to form a video signal, the second sequence of pictures being for use in editing or further processing of the video signal.
PCT/GB2003/003991 2002-09-12 2003-09-12 Manipulation of video compression WO2004025964A2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
DE10393211T DE10393211T5 (en) 2002-09-12 2003-09-12 Handling Video Compression
AU2003269142A AU2003269142A1 (en) 2002-09-12 2003-09-12 Manipulation of video compression
GB0504586A GB2410852B (en) 2002-09-12 2003-09-12 Manipulation of video compression

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
GB0221146A GB2393060B (en) 2002-09-12 2002-09-12 Manipulation of video compression
GB0221146.4 2002-09-12

Publications (2)

Publication Number Publication Date
WO2004025964A2 true WO2004025964A2 (en) 2004-03-25
WO2004025964A3 WO2004025964A3 (en) 2004-07-29

Family

ID=9943919

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/GB2003/003991 WO2004025964A2 (en) 2002-09-12 2003-09-12 Manipulation of video compression

Country Status (4)

Country Link
AU (1) AU2003269142A1 (en)
DE (1) DE10393211T5 (en)
GB (2) GB2393060B (en)
WO (1) WO2004025964A2 (en)

Cited By (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006098226A1 (en) 2005-03-14 2006-09-21 Matsushita Electric Industrial Co., Ltd. Encoding device and dynamic image recording system having the encoding device
EP2136566A1 (en) * 2007-04-16 2009-12-23 Kabushiki Kaisha Toshiba Image encoding and image decoding method and device
ES2350664A1 (en) * 2008-09-01 2011-01-26 Fundacion Cetena Method of bit rate control in digital video compression systems based on MJPEG
WO2013158293A1 (en) * 2012-04-19 2013-10-24 Vid Scale, Inc. System and method for error-resilient video coding

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4132359A1 (en) * 1991-09-27 1993-04-01 Grundig Emv METHOD FOR RECORDING MOVING IMAGES FOR A FOLLOWING, MOTION-INFORMATION-BASED SIGNAL PROCESSING
WO1999009747A1 (en) * 1997-08-14 1999-02-25 Samsung Electronics Co., Ltd. Video transmission apparatus employing intra-frame-only video compression that is mpeg-2 compatible
US5915069A (en) * 1995-09-27 1999-06-22 Sony Corporation Apparatus and method for recording a video signal on a record medium
WO2001022735A1 (en) * 1999-09-21 2001-03-29 Hewlett-Packard Company Video communication using multiple streams
FR2811187A1 (en) * 2000-07-03 2002-01-04 Telediffusion De France Tdf Public place/road traffic/site surveillance having digital video sequences with predictive/non predictive images with adaptive time coding varying image step sizes following alert state.

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
EP0514663A3 (en) * 1991-05-24 1993-07-14 International Business Machines Corporation An apparatus and method for motion video encoding employing an adaptive quantizer
US5384598A (en) * 1992-10-20 1995-01-24 International Business Machines Corporation System and method for frame differencing video compression and decompression with frame rate scalability
CA2105209A1 (en) * 1993-08-31 1995-03-01 Limin Wang Bit rate control for hybrid dpcm/dct video codec
US5731837A (en) * 1996-01-25 1998-03-24 Thomson Multimedia, S.A. Quantization circuitry as for video signal compression systems
US6480541B1 (en) * 1996-11-27 2002-11-12 Realnetworks, Inc. Method and apparatus for providing scalable pre-compressed digital video with reduced quantization based artifacts
US6118822A (en) * 1997-12-01 2000-09-12 Conexant Systems, Inc. Adaptive entropy coding in adaptive quantization framework for video signal coding systems and processes
JPH11196422A (en) * 1997-12-16 1999-07-21 Texas Instr Inc <Ti> Encode method of image signal

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE4132359A1 (en) * 1991-09-27 1993-04-01 Grundig Emv METHOD FOR RECORDING MOVING IMAGES FOR A FOLLOWING, MOTION-INFORMATION-BASED SIGNAL PROCESSING
US5915069A (en) * 1995-09-27 1999-06-22 Sony Corporation Apparatus and method for recording a video signal on a record medium
WO1999009747A1 (en) * 1997-08-14 1999-02-25 Samsung Electronics Co., Ltd. Video transmission apparatus employing intra-frame-only video compression that is mpeg-2 compatible
WO2001022735A1 (en) * 1999-09-21 2001-03-29 Hewlett-Packard Company Video communication using multiple streams
FR2811187A1 (en) * 2000-07-03 2002-01-04 Telediffusion De France Tdf Public place/road traffic/site surveillance having digital video sequences with predictive/non predictive images with adaptive time coding varying image step sizes following alert state.

Non-Patent Citations (3)

* Cited by examiner, † Cited by third party
Title
KEESMAN G ET AL: "BIT-RATE CONTROL FOR MPEG ENCODERS" SIGNAL PROCESSING. IMAGE COMMUNICATION, ELSEVIER SCIENCE PUBLISHERS, AMSTERDAM, NL, vol. 6, no. 6, 1 February 1995 (1995-02-01), pages 545-560, XP000491860 ISSN: 0923-5965 *
SONG BIN ET AL: "Control strategy of subjective video quality over error-prone channels*" COMMUNICATION TECHNOLOGY PROCEEDINGS, 2000. WCC - ICCT 2000. INTERNATIONAL CONFERENCE ON, vol. 1, 21 August 2000 (2000-08-21), pages 962-965, XP010526892 *
WENGER S: "Temporal scalability using P-pictures for low-latency applications" MULTIMEDIA SIGNAL PROCESSING, 1998 IEEE SECOND WORKSHOP ON REDONDO BEACH, CA, USA 7-9 DEC. 1998, PISCATAWAY, NJ, USA,IEEE, US, 7 December 1998 (1998-12-07), pages 559-564, XP010318327 ISBN: 0-7803-4919-9 *

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2006098226A1 (en) 2005-03-14 2006-09-21 Matsushita Electric Industrial Co., Ltd. Encoding device and dynamic image recording system having the encoding device
EP1860886A1 (en) * 2005-03-14 2007-11-28 Matsushita Electric Industrial Co., Ltd. Encoding device and dynamic image recording system having the encoding device
EP1860886A4 (en) * 2005-03-14 2011-08-03 Panasonic Corp Encoding device and dynamic image recording system having the encoding device
EP2136566A1 (en) * 2007-04-16 2009-12-23 Kabushiki Kaisha Toshiba Image encoding and image decoding method and device
EP2136566A4 (en) * 2007-04-16 2012-07-18 Toshiba Kk Image encoding and image decoding method and device
ES2350664A1 (en) * 2008-09-01 2011-01-26 Fundacion Cetena Method of bit rate control in digital video compression systems based on MJPEG
WO2013158293A1 (en) * 2012-04-19 2013-10-24 Vid Scale, Inc. System and method for error-resilient video coding
US9191671B2 (en) 2012-04-19 2015-11-17 Vid Scale, Inc. System and method for error-resilient video coding

Also Published As

Publication number Publication date
GB2410852A (en) 2005-08-10
DE10393211T5 (en) 2005-10-13
GB2393060A (en) 2004-03-17
GB2410852B (en) 2006-01-04
AU2003269142A1 (en) 2004-04-30
GB2393060B (en) 2006-01-25
GB0504586D0 (en) 2005-04-13
GB0221146D0 (en) 2002-10-23
WO2004025964A3 (en) 2004-07-29
AU2003269142A8 (en) 2004-04-30

Similar Documents

Publication Publication Date Title
JP3888597B2 (en) Motion compensation coding apparatus and motion compensation coding / decoding method
EP1497990B1 (en) Method and device for indicating quantizer parameters in a video coding system
US8300688B2 (en) Method for video transcoding with adaptive frame rate control
KR100934290B1 (en) MPEG-2 4: 2: 2-Method and Architecture for Converting a Profile Bitstream to a Main-Profile Bitstream
US6658157B1 (en) Method and apparatus for converting image information
EP1725042A1 (en) Fade frame generating for MPEG compressed video data
EP1445962A1 (en) Adaptive variable-length coding and decoding methods for image data
US9071844B2 (en) Motion estimation with motion vector penalty
US20100172593A1 (en) Image encoding apparatus, image encoding method, and image encoding program
AU2005201439A1 (en) High-fidelity transcoding
JPH09512410A (en) Video signal transcoding method and apparatus
JP2002199402A (en) System for transcoding discrete cosine transform coded signals, and method related thereto
US20060072667A1 (en) Transcoder for a variable length coded data stream
US6795498B1 (en) Decoding apparatus, decoding method, encoding apparatus, encoding method, image processing system, and image processing method
KR100549208B1 (en) Overhead data processor in a memory efficient image processing system
JP3948266B2 (en) Moving picture coding apparatus, coding method, decoding apparatus, decoding method, and moving picture code string transmission method
US6040875A (en) Method to compensate for a fade in a digital video input sequence
JP2000115765A (en) Video signal coder
WO2004025964A2 (en) Manipulation of video compression
EP0899961A1 (en) Pipelined processing for moving picture compression bit rate control
JP4399794B2 (en) Image coding apparatus and image coding method
KR100627494B1 (en) Moving picture encoder and method for c0ding moving picture using the same
KR20010104058A (en) Adaptive quantizer according to DCT mode in MPEG2 encoder
JPH07107464A (en) Picture encoding device and decoding device
JP2005507620A (en) compression

Legal Events

Date Code Title Description
AK Designated states

Kind code of ref document: A2

Designated state(s): AE AG AL AM AT AU AZ BA BB BG BR BY BZ CA CH CN CO CR CU CZ DE DK DM DZ EC EE EG ES FI GB GD GE GH GM HR HU ID IL IN IS JP KE KG KP KR KZ LC LK LR LS LT LU LV MA MD MG MK MN MW MX MZ NI NO NZ OM PG PH PL PT RO RU SC SD SE SG SK SL SY TJ TM TN TR TT TZ UA UG US UZ VC VN YU ZA ZM ZW

AL Designated countries for regional patents

Kind code of ref document: A2

Designated state(s): GH GM KE LS MW MZ SD SL SZ TZ UG ZM ZW AM AZ BY KG KZ MD RU TJ TM AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HU IE IT LU MC NL PT RO SE SI SK TR BF BJ CF CG CI CM GA GN GQ GW ML MR NE SN TD TG

121 Ep: the epo has been informed by wipo that ep was designated in this application
ENP Entry into the national phase in:

Ref document number: 0504586

Country of ref document: GB

Kind code of ref document: A

Free format text: PCT FILING DATE = 20030912

RET De translation (de og part 6b)

Ref document number: 10393211

Country of ref document: DE

Date of ref document: 20051013

Kind code of ref document: P

WWE Wipo information: entry into national phase

Ref document number: 10393211

Country of ref document: DE

122 Ep: pct application non-entry in european phase
NENP Non-entry into the national phase in:

Ref country code: JP

WWW Wipo information: withdrawn in national office

Ref document number: JP

REG Reference to national code

Ref country code: DE

Ref legal event code: 8607