GB2207829A - Transform coding a video signal - Google Patents

Abstract

Transform coding is used for encoding blocks of a video signal which have changed since an earlier field. Higher sequency transform coefficients are omitted. The omitted coefficients are those coefficients which, if the group were arranged, ordered as to horizontal and vertical sequency, in a rectangular matrix with the zero sequency coefficient at one corner representing the origin of x, y coordinates, x representing the position along a row and y the position along a column, would have an x coordinate less than or equal to that of the highest column, and a y coordinate less than or equal to that of the highest row, in respect of which the sum of the absolute values of the coefficients, or of an increasing substantially monotonic function of the absolute values of the coefficients, in the row or column exceeds a second threshold value which is related to the number of blocks selected in that field. Additionally a number of unchanged blocks are chosen for transform coding (replenishment). <IMAGE>

Description

TRANSFORM CODING This invention relates to a video coder and decoder particularly though not exclusively for electronically represented moving images, such as those used in videophone systems.

The transmission of video signals over telephone lines requires a very efficient coding scheme because only limited bandwidth is available in the channel. In videophones the usual image, the head and shoulders of a person talking, exhibits little movement between one frame and the next; or put another way there is a high frame-to-frame correlation. The technique of 'conditional replenishment' takes advantage of the resulting signal redundancy to improve coding efficiency: instead of transmitting data for every element of every frame, data concerning only those elements that change between frames is sent.

This technique will now be described in more detail.

A reference image is stored at both transmitting and receiving ends of the system and each new frame of the image is compared with the reference image at the transmitter in order to determine the difference between every picture element of the two. Unless the difference in respect of a particular element is sufficiently great the original value of that element is returned to the reference image store. However, when the difference exceeds a threshold the new value, as well as replacing the existing value in the reference image store, is also sent along with its address to the receiver. At the receiver the values of the picture elements are updated (replenished) as they are received.

This scheme has the disadvantage that, for every replenished picture element, the new value that is sent to the receiver must be accompanied by its address. Coding efficiency can be further improved by using a block based scheme, in which the image is divided into blocks of picture elements and the replenishment decision is made on the basis of a comparison between corresponding blocks exceeding a variable threshold. Such a scheme is described in European Patent Application EP-A-0084270 (CLI). A block marked for transmission is transform-coded, for instance by applying a discrete cosine transform (DCT). In general a block of NxM picture elements will be transformed into an array of NxM coefficients. The resulting array of coefficients is then quantised, by dividing each coefficient by a variable quantisation factor.The coefficients relating to the highest frequency components of the picture elements of the block are disregarded - that is, those zero value components that have predictive mean values below a preselected value. And finally the remaining quantised coefficients are coded in variable length code, for instance Huffman Code. In fact several variable length codebooks are available, and a decision about which is chosen to be used for a particular coefficient is made depending on the statistics of the coefficients.

In such coding schemes data is generated at an irregular rate at the transmitter and an output buffer memory is used to ensure that transmission takes place at a uniform data rate.

The-fullness of the output buffer is used to determine the variable threshold and the quantisation factor. And in such a manner that when the buffer fills (as a result of a high level of activity in the picture) the variable threshold is raised and replenishment becomes concentrated around the areas of the image that move most; and the quantisation factor is increased so that spatial resolution is reduced. Conversely, when image activity is low and the buffer empties, the variable threshold may be reduced (even to zero when replenishment may be triggered by background noise) in order that a minimum amount of data will be maintained in the buffer. One drawback of this is that some elements of the image may not be replenished for a comparatively long period of time.

A coder for two dimensional data having a successive field format including: comparison means for determining a measure of the extent to which each of a plurality of blocks of the said data within a field has changed relative to the corresponding block of an earlier field; selector means for selecting those later blocks in respect of which the measure is greater than a threshold value; counter means for counting the number of blocks selected in each field; converter means for converting each of the selected blocks to a two dimensional group of transform coefficients; analysis means for identifying for each group those coefficients which, if the group were arranged, ordered as to horizontal and vertical sequency, in a rectangular matrix with the zero sequency coefficient at one corner representing the origin of x, y coordinates, x representing the position along a row and y the position along a column, would have an x coordinate less than or equal to that of the highest (sequency) column, and a y coordinate less than or equal to that of the highest row, in respect of which the sum of the absolute values of the coefficients, or of an increasing substantially monotonic function of the absolute values of the coefficients, in that row or column exceeds a second threshold value which is related to the number of blocks selected in that field; and encoding means for encoding the identified coefficients.

The invention will now be described by way of example with reference to the accompanying drawings in which: Figure 1 shows a schematic block diagram of a coder according to an embodiment of the invention; Figure 2a shows the arrangement of the picture elements in the frame store of the coder shown in Figure 1; Figure 2b shows the arrangement of the data blocks in the frame store shown in Figure 2a; Figure 3a shows the coefficients produced by the discrete cosine transform of the coder shown in Figure 1 arranged in order of row and column sequency in an array; Figure 3b shows an active zone of the coefficient array shown in Figure 3a; Figure 3c shows the allocation of variable length coders to the coefficients of the array shown in Figure 3a; and Figure 4 shows a schematic block diagram of a decoder according to an embodiment of the invention.

Referring now to Figures 1, 2 and 3 each frame of an image generated by a camera 1 is loaded into a frame store 2. The frame store is shown in greater detail in Figure 2(a) where it can be seen each frame comprises 64 by 64 luminance, or Y, picture elements (pels) and 32 by 32 of both U and V chrominance pels. Each frame is sub-divided into blocks for processing purposes, each block comprises 8 by 8 pels. This is illustrated in Figure 2(b) where it can be seen that there are 64 Y blocks and 16 of both U and V blocks.

As a new frame is being written into (a page in) the store 2 the processing front moves down nine lines behind the field capture front, so processing of the first row of Y blocks starts once line 9 of the field is reached by the capture front. Each block of data is filtered by a non-linear temporal filter 3 (of Figure 1), of the type using a look-up table. This filtering process reduces noise. The look-up process also produces a modified fourth power law pel difference (between pels of the present block and the last block in that position to be transmitted) output. This output is summed over the block and compared with a pre-set constant threshold, in change circuit 4, to identify changed blocks.The block output from the filter is stored via a selector illustrated as a switch 4a controlled by the output of the change circuit 4, in a further store 5 (where it overwrites the previously stored value), if that block is identified as having changed.

The number of changed blocks in each frame is also counted by a counter 6 to provide an indication of picture activity for use later.

In parallel with this identification of changed blocks four Y blocks and one of each of U and V blocks in every frame are selected for transmission by forcing circuit 7.

For this purpose every block is numbered and a block is marked for transmission if its number (modulo 16) equals that of a forced up-date counter (not shown), which is incremented in scrambled order to avoid a visible up-date front. The output of the filter for each block marked for transmission is also entered in the store 5. This cyclic forced up-date ensures that maximum spatial resolution is achieved and mops up errors.

A Discrete Cosine Transform (DCT) sub-routine circuit 8 then reads data from each block to be changed (in UVY order; U and V first to allow for more efficient use of the processor time at the decoder, as will be discussed later). Then the data is transformed to give, in respect of each block, sixtyfour coefficients. The coefficients may be arranged, as shown in Figure 3(a), in order of row and column sequency in an array. (Sequency in a given direction is the number of sign changes in the corresponding row or column of the basis matrix of the transformation used). In the array shown the upper left coefficient is the DC (average) coefficient and the other coefficients represent increasingly higher frequency components of the spatial domain block the further rightwards and downwards from the DC value they are.

The coefficients are then analysed 9 to determine the "active zone" - so called because it is represented by a zone of coefficients within the ordered array and usually excludes some coefficients that represent high frequency spatial domain information. This may be determined, as a function of activity in the block and the percentage changed area in the frame, as follows: A block quantiser control parameter is derived 10; in the case of changed blocks it is related to the number of changed Y blocks, and in the case of forced up-date blocks it is set to zero (giving maximum resolution). The coefficients making up each row and column of the ordered array are then tested from the bottom and right, ie in order of those having highest sequency first, to see if the sum of squares of the coefficients in that row, or column, exceeds the control parameter.All rows and columns that fail to yield a sum that exceeds the parameter, up to, but not including, the first that exceeds it, are considered to lie outside the active zone. Or, in other words, the coefficients having row sequency greater than that of the highest sequency column or column sequency greater than that of the highest row, the coefficients of which when squared and summed exceed the parameter are considered to lie outside the active zone.

It should be appreciated that test is not limited merely to the sum of squares. Any other increasing substantially monotonic function of the absolute values of the coefficients may be used. In a particular preferred embodiment a look-up table is used to relate the coefficients' values to values which are summed and compared with the parameter.

The active zone is usually a shrunk version of the ordered array; with the shrinkage on the bottom and right side. Figure 3(b) illustrates an active zone of X columns and Y rows.

The process excludes rows and columns with all zero coefficients, and for a control parameter greater than zero, ie. when there is a high number of changed blocks in a frame, may also exclude some rows and columns having only a few small valued coefficients. Coefficients lying outside the active zone are disregarded, so that the number of zero and small value coefficients that have to be further processed and transmitted is reduced, and coding efficiency is increased. A minimum permitted active zone size, for example 3 by 3 coefficients may be set.

The active zone of coefficients is scanned from bottom right to top left and each coefficient is quantised and coded, by variable length coder (VLC) 11, according to one of several variable length codebooks. The codebooks are matched to the measured probability distribution of the coefficients. In a preferred embodiment there are four variable length codebooks selected according to the position of the coefficients within the ordered array as shown in Figure 3c where each array position is matched with a digit from 1 to 4 representing one of the four codebooks. The DC coefficient is linear quantised.

The coded coefficients and other data, such as a map of the blocks which are to be updated and the data defining the size of the active zone XV, are then assembled into frames and passed to an output FIFO buffer 12 where the data rate equalised to that of the serial output line 14 before the bits are transmitted by transmitter 13.

In the event that all the AC coefficients have zero value, so that the active zone is of minimum size and all the active zone coefficient use their shortest code words, the transmission time remains greater than the time required to transform and code the next block of data so the output buffer 12 cannot underflow.

Since the threshold used to determine whether a block of data has changed sufficiently for it to be transmitted remains constant, no matter how much activity there is in the picture, then as the picture becomes more active so more blocks are marked for transmission and the frame rate falls. In order to prevent the picture update rate falling excessively the control parameter is increased in dependence on the number of changed blocks per frame that are counted, so that the active zone size is reduced and each block is represented by a shorter amount of data.

This means that as picture activity increases spatial resolution is traded-off against maintaining an acceptably high frame rate. Conversely when activity in the picture falls the control parameter is reduced and spatial resolution increases.

At the decoder, shown in Figure 4, coded data from the serial line channel 14 is received at receiver 15, and stored in FIFO buffer store 16, where a bit-wise search is conducted through the data to-find the beginning of a frame.

The blocks to be updated are identified from the map and inverse block quantiser 17 reads the X and Y active zone sizes so that the variable length codes are decoded, in VLC decoder 18, in the same order as they are scanned at the coder. The variable length codes are partly decoded by tree following algorithms, since the direct table look-up approach requires large tables which demand a large memory overhead.

The decoded arrays are then transformed back to the spatial domain by inverse discrete cosine transformer 19.

The inverse DCT takes a similar time to the forward DCT, but the variable length decoding and inverse DCT together always take less time than that required to transmit the data for a block; so buffer 16 is always emptied faster than it can be filled, preventing overflow. If the store empties completely the decoder waits until another byte is available.

U and V frames are decoded at 32 by 32 pel resolution, but then are linearly interpolated by interpolator 20 to 64 pels vertically before'display, to reduce the visibility of coding errors, particularly at block edges.

Since this interpolation is performed before the Y data is decoded a slight backlog of data builds up in the buffer 16 during the interpolation but this backlog is usually cleared before the last Y block is reached so no additional overall delay is introduced.

Several checksums (not shown) may be distributed through the data frame. If any of the checksums fail to be validated the frame is aborted, in which case the search starts for the next frame. Any false starts will be caught in this way by the first checksum.

Once a complete frame has been successfully received it is passed to a frame store and displayed after the next field blanking interval 21. Frames dropped due to errors may leave incorrect blocks, for example remnant edges of moving objects, in the displayed picture. These are progressively over written with correct data by the forced up-date mode of the coder. To the decoder these forced up-date blocks are indistinguishable from normal changed blocks.

The cyclic forced up-date would ensure that the entire picture was up-dated in 1 to 3 seconds, when a 64 kbits/s channel is available, depending on the level of picture activity.

The video coder and decoder described provide very high coding efficiency; and subjectively acceptable results have been obtained from a videophone system having a picture display of 64 by 64 pels using a 64 Kbits/sec channel when using this coding scheme.

In an alternative embodiment the coder does not maintain a constant threshold to determine whether a block of data has changed significantly, but varies the threshold in dependence upon the number of changed blocks counted in the previous frame. The threshold is raised when picture activity in the preceding frame is high, and lowered when picture activity in the preceding frame is low. This negative feedback, with a one frame period delay, maintains an additional control on the overall frame rate.

Claims (8)

1. A coder for two dimensional data having a successive field format including: comparison means for determining a measure of the extent to which each of a plurality of blocks of the said data within a field has changed relative to the corresponding block of an earlier field; selector means for selecting those later blocks in respect of which the measure is greater than a threshold value; counter means for counting the number of blocks selected in each field; converter means for converting each of the selected blocks to a two dimensional group of transform coefficients; analysis means for identifying for each group those coefficients which, if the group were arranged, ordered as to horizontal and vertical sequency, in a rectangular matrix with the zero sequency coefficient at one corner representing the origin of x, y coordinates, x representing the position along a row and y the position along a column, would have an x coordinate less than or equal to that of the highest column, and a y coordinate less than or equal to that of the highest row, in respect of which the sum of the absolute values of the coefficients, or of an increasing substantially monotonic function of the absolute values of the coefficients, in that row or column exceeds a second threshold value which is related to the number of blocks selected in that field; and encoding means for encoding the identified coefficients.

2. A coder according to Claim 1 in which the converter means also converts a plurality of further blocks from each field to the transform domain, the plurality being chosen in scrambled order such that in a finite number of successive fields one complete field, and only one complete field, of blocks is chosen.

3. A coder according to Claim 1 or 2 in which the encoding means comprises a variable length coder having several codebooks which are allocated to coefficients according to their sequency.

4. A coder according to Claim 1, 2 or 3 in which the encoder also encodes a parameter representing the row and column sequency of that identified coefficient having the highest sequency.

5. A decoder for decoding two -dimensional transform coded data such as may be produced by a coder according to Claim 4 including: receiver means for receiving encoded data and an encoded parameter representing the row and column sequency of that encoded data coefficient having the highest sequency; inverse encoding means for decoding the data in order of reverse sequency; and inverse transformation means for transforming the decoded data into the time domain.

6. A decoder according to Claim 5 in which the inverse encoding means decodes the data at least partly according to tree following algorithms.

7. A coder substantially as hereinbefore described with reference to figures 1, 2 or 3 of the accompanying drawings.

8. A decoder substantially as hereinbefore described with reference to figure 4 of the accompanying drawings.